2024-09-21T03:54:01.855 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:54:01.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:54:01.967 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678 branch: wip-guits-main-2024-09-20-2310 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: gabrioux@ibm.com first_in_suite: false job_id: '7913678' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-20-2310 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: mon pg warn min per osd: 0 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-guits-main-2024-09-20-2310 sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3363 sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-20-2310 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8MRvxB9KO8eRD0P4Uw2xEN2eJS/YsZuhg5vaOS0P9TiPea2em/gAWvY9Z6T+uaCAF969LXV7nqXEOHGr0EwA8= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU6doYUyLO3OAH7LaVrZGEi5LCbl0c/XDwTmjdkyML7kWZaTAdJCp9uGOgN7fTkX968G5kyAW0zKI51qBRLdLo= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/gabrioux/src/git.ceph.com_ceph-c_wip-guits-main-2024-09-20-2310/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_03:44:43 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-21T03:54:01.967 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa; will attempt to use it 2024-09-21T03:54:01.968 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks 2024-09-21T03:54:01.968 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:54:01.970 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:54:01.989 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2' 2024-09-21T03:54:01.990 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:54:01.990 INFO:teuthology.packaging:ref: None 2024-09-21T03:54:01.990 INFO:teuthology.packaging:tag: None 2024-09-21T03:54:01.990 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-20-2310 2024-09-21T03:54:01.990 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T03:54:01.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-20-2310 2024-09-21T03:54:02.126 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5103.gf36283e7 2024-09-21T03:54:02.128 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:54:02.134 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:54:02.134 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:54:02.178 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:54:02.198 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:54:02.207 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:54:02.233 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:48:52.200676', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8MRvxB9KO8eRD0P4Uw2xEN2eJS/YsZuhg5vaOS0P9TiPea2em/gAWvY9Z6T+uaCAF969LXV7nqXEOHGr0EwA8='} 2024-09-21T03:54:02.255 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:48:52.201842', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU6doYUyLO3OAH7LaVrZGEi5LCbl0c/XDwTmjdkyML7kWZaTAdJCp9uGOgN7fTkX968G5kyAW0zKI51qBRLdLo='} 2024-09-21T03:54:02.255 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:54:02.261 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-21T03:54:02.261 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-21T03:54:02.261 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:54:02.315 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f814fe60280>, signals=[15]) 2024-09-21T03:54:02.315 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:54:02.321 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:54:02.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T03:54:02.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:54:02.400 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T03:54:02.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:54:02.476 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:54:02.483 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-21T03:54:02.501 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-21T03:54:02.502 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-21T03:54:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-21T03:54:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-21T03:54:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-21T03:54:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:54:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:54:02.560 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:54:02.561 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-21T03:54:02.588 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-21T03:54:02.606 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-21T03:54:02.607 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:54:02.665 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:54:02.666 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:54:02.666 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:54:02.666 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:54:02.666 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-21T03:54:02.688 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:54:02.697 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:54:02.703 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:54:02.703 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:54:02.705 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:54:02.721 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:54:02.727 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:54:02.728 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:54:02.762 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:54:03.044 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:54:03.099 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:54:03.114 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:54:03.114 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T03:54:03.114 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:54:03.115 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:54:03.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:54:03.132 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-21T03:54:03.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:54:03.133 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-21T03:54:03.190 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:54:03.190 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-503.el9.x86_64 2024-09-21T03:54:03.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-09-21T03:54:03.192 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:54:03.192 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-503.el9.x86_64 2024-09-21T03:54:03.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-09-21T03:54:06.250 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.3 MB 00:01 2024-09-21T03:54:08.672 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.3 MB 00:03 2024-09-21T03:54:16.048 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-21T03:54:20.307 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T03:54:22.041 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 19 kB 00:00 2024-09-21T03:54:23.279 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T03:54:28.726 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T03:54:28.739 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 977 kB/s | 20 MB 00:21 2024-09-21T03:54:30.616 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:54:30.616 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:54:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T03:54:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-21T03:54:30.703 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:54:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 79 M 2024-09-21T03:54:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 126 M 2024-09-21T03:54:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T03:54:30.819 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 381 kB/s | 36 kB 00:00 2024-09-21T03:54:31.222 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-21T03:54:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-21T03:54:31.948 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-21T03:54:31.948 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:54:31.948 INFO:teuthology.orchestra.run.smithi110.stdout:Total 63 MB/s | 79 MB 00:01 2024-09-21T03:54:32.631 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T03:54:32.975 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T03:54:32.975 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T03:54:33.165 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T03:54:33.909 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T03:54:33.910 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T03:54:35.516 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-21T03:54:36.356 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T03:54:36.403 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T03:54:37.066 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:54:37.148 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:54:39.897 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:54:40.041 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:54:41.875 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T03:54:43.774 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:54:43.774 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:54:43.870 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T03:54:43.879 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-21T03:54:43.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:54:43.881 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 79 M 2024-09-21T03:54:43.881 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 126 M 2024-09-21T03:54:43.881 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T03:54:44.468 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 64 kB/s | 36 kB 00:00 2024-09-21T03:54:44.857 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:54:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:54:45.427 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-21T03:54:45.961 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-21T03:54:46.661 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-21T03:54:46.661 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:54:46.661 INFO:teuthology.orchestra.run.smithi148.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-21T03:54:47.342 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T03:54:47.671 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T03:54:47.671 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T03:54:48.639 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T03:54:48.639 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T03:54:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:54:51.100 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T03:54:51.816 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:54:51.903 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:54:54.685 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:54:54.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:54:59.688 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:54:59.800 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:28.349 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:28.389 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:28.692 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:28.692 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:55:28.692 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:55:28.692 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:55:29.547 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:29.547 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:55:29.547 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T03:55:29.547 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:29.548 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:55:29.548 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:55:29.548 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:55:29.548 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:55:29.548 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T03:55:29.934 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:55:30.026 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:30.027 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:55:30.027 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:55:30.027 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:55:30.028 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:55:30.028 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-21T03:55:30.028 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:55:30.028 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:55:30.088 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:30.088 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:55:30.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 82.el9 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-09-21T03:55:30.127 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8274670 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:55:30.128 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:55:30.129 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:55:30.129 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-09-21T03:55:30.129 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T03:55:30.129 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:55:30.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:55:30.130 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:55:30.130 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:55:30.816 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-21T03:55:32.269 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:55:32.306 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-21T03:55:32.316 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-21T03:55:32.331 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.bJnrNct5Jz 2024-09-21T03:55:32.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bJnrNct5Jz 2024-09-21T03:55:32.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.bJnrNct5Jz 2024-09-21T03:55:32.550 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.bJnrNct5Jz is 6KB 2024-09-21T03:55:32.599 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.bJnrNct5Jz 2024-09-21T03:55:32.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:55:32.690 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:55:32.690 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:55:32.690 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:55:32.690 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:55:32.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:55:32.842 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-21T03:55:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:43.013 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:55:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:55:43.391 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:55:44.292 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:55:44.293 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T03:55:44.626 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:55:44.708 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:55:44.708 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:55:44.708 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:55:44.708 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:55:44.708 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-21T03:55:44.708 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:55:44.709 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:55:44.768 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:55:44.768 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:55:44.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-09-21T03:55:44.800 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-09-21T03:55:44.801 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-09-21T03:55:44.801 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-09-21T03:55:44.801 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 82.el9 2024-09-21T03:55:44.801 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-09-21T03:55:44.801 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8274670 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-09-21T03:55:44.802 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T03:55:44.803 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:55:44.804 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:55:44.805 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:55:44.805 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:55:45.478 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-09-21T03:55:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:55:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-09-21T03:55:46.846 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-21T03:55:46.862 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.AHpmN25M8c 2024-09-21T03:55:46.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AHpmN25M8c 2024-09-21T03:55:46.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.AHpmN25M8c 2024-09-21T03:55:47.081 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.AHpmN25M8c is 6KB 2024-09-21T03:55:47.129 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.AHpmN25M8c 2024-09-21T03:55:47.144 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:55:47.216 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:55:47.216 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:55:47.216 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:55:47.216 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:55:47.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:55:47.372 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-09-21T03:56:02.860 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-21T03:56:02.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:56:02.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:56:17.388 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-21T03:56:17.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T03:56:17.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:56:21.359 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-21T03:56:30.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:56:30.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:56:52.079 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-21T03:57:01.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T03:57:01.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:57:19.471 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-21T03:57:30.380 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:57:31.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T03:57:31.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:57:31.887 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-21T03:57:32.390 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T03:57:32.390 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:57:32.390 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-21T03:57:32.408 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:57:32.408 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:57:32.408 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:57:32.408 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:57:33.409 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-21T03:57:33.409 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:57:33.504 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-09-21T03:57:33.525 DEBUG:teuthology.parallel:result is None 2024-09-21T03:57:42.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:57:42.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:57:42.854 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-21T03:57:43.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T03:57:43.242 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:57:43.242 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-21T03:57:43.265 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:57:43.265 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:57:43.265 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:57:43.265 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:57:44.266 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-21T03:57:44.266 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T03:57:44.334 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-21T03:57:44.352 DEBUG:teuthology.parallel:result is None 2024-09-21T03:57:44.352 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:57:44.365 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:57:44.365 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:57:44.367 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:57:44.388 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:57:44.400 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:57:44.427 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:57:44.427 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:57:44.430 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:57:44.541 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:57:44.547 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:57:44.547 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:57:44.550 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:57:44.580 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:57:44.581 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:57:44.591 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:57:44.592 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:57:44.594 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:57:44.600 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:57:44.601 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:57:44.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:57:44.663 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:57:44.672 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:57:44.672 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:57:44.705 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:57:44.721 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:57:44.801 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:57:44.851 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:57:44.936 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:57:44.976 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T03:57:44.976 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:57:45.041 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:57:45.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:57:45.145 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:57:45.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:57:45.274 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T03:57:45.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:57:45.341 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-21T03:57:45.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-21T03:57:45.391 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:57:45.427 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:57:45.942 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:57:45.950 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:57:45.950 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:57:45.960 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:57:45.969 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:57:45.969 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-21T03:57:46.018 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-21T03:57:46.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-21T03:57:46.069 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-21T03:57:46.071 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:57:46.071 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-21T03:57:46.093 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-21T03:57:46.094 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-21T03:57:46.127 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-21T03:57:46.128 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:57:46.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:57:46.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:46.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:57:46.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:46.196 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:57:46.196 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:57:46.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:57:46.274 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:57:46.290 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:57:46.299 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:57:46.300 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T04:01:32.101 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-09-21T04:01:32.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T04:01:32.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:01:32.182 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-21T04:01:32.254 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T04:01:32.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T04:01:32.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:01:32.333 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-21T04:01:32.410 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T04:01:32.410 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T04:01:32.420 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T04:01:32.420 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T04:01:32.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:01:32.423 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T04:01:32.423 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:01:32.450 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T04:01:32.464 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T04:01:32.480 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T04:01:32.488 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-21T04:01:32.514 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T04:01:32.519 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-21T04:01:32.528 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T04:01:32.530 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-21T04:01:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T04:01:32.558 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T04:01:32.581 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:01:32.584 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:01:32.584 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T04:01:32.585 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.585 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.585 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.585 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:01:32.627 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:01:32.627 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-21T04:01:32.627 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.627 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:01:32.629 INFO:teuthology.run_tasks:Running task install... 2024-09-21T04:01:32.637 DEBUG:teuthology.task.install:project ceph 2024-09-21T04:01:32.637 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'}} 2024-09-21T04:01:32.637 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'} 2024-09-21T04:01:32.637 INFO:teuthology.task.install:Using flavor: default 2024-09-21T04:01:32.642 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T04:01:32.642 INFO:teuthology.task.install:extra packages: [] 2024-09-21T04:01:32.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'tag': None, 'wait_for_package': False} 2024-09-21T04:01:32.643 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:01:32.643 INFO:teuthology.packaging:ref: None 2024-09-21T04:01:32.643 INFO:teuthology.packaging:tag: None 2024-09-21T04:01:32.643 INFO:teuthology.packaging:branch: reef 2024-09-21T04:01:32.643 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:01:32.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-21T04:01:32.645 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'tag': None, 'wait_for_package': False} 2024-09-21T04:01:32.645 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:01:32.645 INFO:teuthology.packaging:ref: None 2024-09-21T04:01:32.646 INFO:teuthology.packaging:tag: None 2024-09-21T04:01:32.646 INFO:teuthology.packaging:branch: reef 2024-09-21T04:01:32.646 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:01:32.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-21T04:01:32.902 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/ 2024-09-21T04:01:32.902 INFO:teuthology.task.install.rpm:Package version is 18.2.4-871.g6a1bdf85 2024-09-21T04:01:32.946 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/ 2024-09-21T04:01:32.946 INFO:teuthology.task.install.rpm:Package version is 18.2.4-871.g6a1bdf85 2024-09-21T04:01:33.026 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T04:01:33.026 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:01:33.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T04:01:33.056 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T04:01:33.056 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:01:33.056 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T04:01:33.057 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T04:01:33.057 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:01:33.057 INFO:teuthology.packaging:ref: None 2024-09-21T04:01:33.057 INFO:teuthology.packaging:tag: None 2024-09-21T04:01:33.057 INFO:teuthology.packaging:branch: reef 2024-09-21T04:01:33.057 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:01:33.058 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T04:01:33.083 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T04:01:33.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:01:33.083 INFO:teuthology.packaging:ref: None 2024-09-21T04:01:33.084 INFO:teuthology.packaging:tag: None 2024-09-21T04:01:33.084 INFO:teuthology.packaging:branch: reef 2024-09-21T04:01:33.084 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:01:33.084 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T04:01:33.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T04:01:33.150 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T04:01:33.200 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T04:01:33.225 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T04:01:33.264 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-21T04:01:33.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-21T04:01:33.290 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-21T04:01:33.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-21T04:01:33.492 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-21T04:01:33.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-09-21T04:01:33.533 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-21T04:01:33.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw 2024-09-21T04:01:34.069 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 296 kB/s | 77 kB 00:00 2024-09-21T04:01:34.107 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 291 kB/s | 77 kB 00:00 2024-09-21T04:01:34.345 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 49 kB/s | 11 kB 00:00 2024-09-21T04:01:34.400 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-21T04:01:34.620 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T04:01:34.675 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T04:01:37.284 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.3 MB 00:02 2024-09-21T04:01:38.337 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.3 MB 00:03 2024-09-21T04:01:39.029 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-21T04:01:40.254 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-21T04:01:44.447 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-21T04:01:44.519 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-21T04:01:45.839 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 115 kB/s | 19 kB 00:00 2024-09-21T04:01:45.920 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 114 kB/s | 19 kB 00:00 2024-09-21T04:01:46.958 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-21T04:01:47.435 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-21T04:01:52.281 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T04:01:52.774 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-21T04:01:54.507 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 7.7 M 2024-09-21T04:01:54.517 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.3 M 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.0 M 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 5.1 M 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 18 M 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 25 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 711 k 2024-09-21T04:01:54.518 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 477 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 4.5 M 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 45 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 129 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 161 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T04:01:54.519 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 320 k 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 300 k 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 99 k 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout:Install 28 Packages 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 2 Packages 2024-09-21T04:01:54.520 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:01:54.521 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 53 M 2024-09-21T04:01:54.521 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:01:55.040 INFO:teuthology.orchestra.run.smithi110.stdout:(1/30): ceph-base-18.2.4-871.g6a1bdf85.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-21T04:01:55.067 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:01:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:01:55.080 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:01:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:01:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 7.7 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.3 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.0 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 5.1 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 18 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 25 k 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T04:01:55.081 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 711 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 477 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 4.5 M 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 45 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 129 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 161 k 2024-09-21T04:01:55.082 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 320 k 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 300 k 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 99 k 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:Install 28 Packages 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-09-21T04:01:55.083 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:01:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 53 M 2024-09-21T04:01:55.084 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:01:55.107 INFO:teuthology.orchestra.run.smithi110.stdout:(2/30): ceph-selinux-18.2.4-871.g6a1bdf85.el9.x 377 kB/s | 25 kB 00:00 2024-09-21T04:01:55.207 INFO:teuthology.orchestra.run.smithi110.stdout:(3/30): libcephfs2-18.2.4-871.g6a1bdf85.el9.x86 7.0 MB/s | 711 kB 00:00 2024-09-21T04:01:55.282 INFO:teuthology.orchestra.run.smithi110.stdout:(4/30): libradosstriper1-18.2.4-871.g6a1bdf85.e 6.2 MB/s | 477 kB 00:00 2024-09-21T04:01:55.399 INFO:teuthology.orchestra.run.smithi110.stdout:(5/30): ceph-radosgw-18.2.4-871.g6a1bdf85.el9.x 9.5 MB/s | 7.7 MB 00:00 2024-09-21T04:01:55.583 INFO:teuthology.orchestra.run.smithi110.stdout:(6/30): ceph-common-18.2.4-871.g6a1bdf85.el9.x8 18 MB/s | 18 MB 00:01 2024-09-21T04:01:55.608 INFO:teuthology.orchestra.run.smithi110.stdout:(7/30): python3-ceph-argparse-18.2.4-871.g6a1bd 216 kB/s | 45 kB 00:00 2024-09-21T04:01:55.684 INFO:teuthology.orchestra.run.smithi110.stdout:(8/30): librgw2-18.2.4-871.g6a1bdf85.el9.x86_64 11 MB/s | 4.5 MB 00:00 2024-09-21T04:01:55.709 INFO:teuthology.orchestra.run.smithi110.stdout:(9/30): python3-ceph-common-18.2.4-871.g6a1bdf8 1.0 MB/s | 129 kB 00:00 2024-09-21T04:01:55.717 INFO:teuthology.orchestra.run.smithi148.stdout:(1/30): ceph-base-18.2.4-871.g6a1bdf85.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-21T04:01:55.734 INFO:teuthology.orchestra.run.smithi110.stdout:(10/30): python3-cephfs-18.2.4-871.g6a1bdf85.el 1.3 MB/s | 161 kB 00:00 2024-09-21T04:01:55.759 INFO:teuthology.orchestra.run.smithi148.stdout:(2/30): ceph-selinux-18.2.4-871.g6a1bdf85.el9.x 608 kB/s | 25 kB 00:00 2024-09-21T04:01:55.759 INFO:teuthology.orchestra.run.smithi110.stdout:(11/30): python3-rados-18.2.4-871.g6a1bdf85.el9 4.1 MB/s | 320 kB 00:00 2024-09-21T04:01:55.785 INFO:teuthology.orchestra.run.smithi110.stdout:(12/30): python3-rbd-18.2.4-871.g6a1bdf85.el9.x 3.9 MB/s | 300 kB 00:00 2024-09-21T04:01:55.810 INFO:teuthology.orchestra.run.smithi110.stdout:(13/30): python3-rgw-18.2.4-871.g6a1bdf85.el9.x 1.3 MB/s | 99 kB 00:00 2024-09-21T04:01:55.826 INFO:teuthology.orchestra.run.smithi148.stdout:(3/30): libcephfs2-18.2.4-871.g6a1bdf85.el9.x86 10 MB/s | 711 kB 00:00 2024-09-21T04:01:55.893 INFO:teuthology.orchestra.run.smithi148.stdout:(4/30): libradosstriper1-18.2.4-871.g6a1bdf85.e 7.0 MB/s | 477 kB 00:00 2024-09-21T04:01:55.894 INFO:teuthology.orchestra.run.smithi110.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-09-21T04:01:55.919 INFO:teuthology.orchestra.run.smithi110.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 799 kB/s | 107 kB 00:00 2024-09-21T04:01:55.944 INFO:teuthology.orchestra.run.smithi110.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T04:01:55.986 INFO:teuthology.orchestra.run.smithi110.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-21T04:01:56.012 INFO:teuthology.orchestra.run.smithi110.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T04:01:56.037 INFO:teuthology.orchestra.run.smithi110.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T04:01:56.087 INFO:teuthology.orchestra.run.smithi110.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-21T04:01:56.104 INFO:teuthology.orchestra.run.smithi110.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.7 MB/s | 26 kB 00:00 2024-09-21T04:01:56.121 INFO:teuthology.orchestra.run.smithi110.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-21T04:01:56.126 INFO:teuthology.orchestra.run.smithi148.stdout:(5/30): librgw2-18.2.4-871.g6a1bdf85.el9.x86_64 19 MB/s | 4.5 MB 00:00 2024-09-21T04:01:56.146 INFO:teuthology.orchestra.run.smithi110.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-21T04:01:56.171 INFO:teuthology.orchestra.run.smithi110.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T04:01:56.197 INFO:teuthology.orchestra.run.smithi110.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T04:01:56.227 INFO:teuthology.orchestra.run.smithi148.stdout:(6/30): ceph-radosgw-18.2.4-871.g6a1bdf85.el9.x 8.2 MB/s | 7.7 MB 00:00 2024-09-21T04:01:56.239 INFO:teuthology.orchestra.run.smithi110.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-21T04:01:56.252 INFO:teuthology.orchestra.run.smithi148.stdout:(7/30): python3-ceph-argparse-18.2.4-871.g6a1bd 360 kB/s | 45 kB 00:00 2024-09-21T04:01:56.285 INFO:teuthology.orchestra.run.smithi148.stdout:(8/30): python3-ceph-common-18.2.4-871.g6a1bdf8 2.2 MB/s | 129 kB 00:00 2024-09-21T04:01:56.289 INFO:teuthology.orchestra.run.smithi110.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T04:01:56.311 INFO:teuthology.orchestra.run.smithi148.stdout:(9/30): python3-cephfs-18.2.4-871.g6a1bdf85.el9 2.7 MB/s | 161 kB 00:00 2024-09-21T04:01:56.344 INFO:teuthology.orchestra.run.smithi148.stdout:(10/30): python3-rados-18.2.4-871.g6a1bdf85.el9 5.3 MB/s | 320 kB 00:00 2024-09-21T04:01:56.369 INFO:teuthology.orchestra.run.smithi148.stdout:(11/30): python3-rbd-18.2.4-871.g6a1bdf85.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-21T04:01:56.389 INFO:teuthology.orchestra.run.smithi110.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-21T04:01:56.395 INFO:teuthology.orchestra.run.smithi148.stdout:(12/30): python3-rgw-18.2.4-871.g6a1bdf85.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-21T04:01:56.456 INFO:teuthology.orchestra.run.smithi110.stdout:(29/30): librados2-18.2.4-871.g6a1bdf85.el9.x86 15 MB/s | 3.3 MB 00:00 2024-09-21T04:01:56.515 INFO:teuthology.orchestra.run.smithi110.stdout:(30/30): librbd1-18.2.4-871.g6a1bdf85.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-21T04:01:56.519 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:01:56.520 INFO:teuthology.orchestra.run.smithi110.stdout:Total 27 MB/s | 53 MB 00:02 2024-09-21T04:01:56.570 INFO:teuthology.orchestra.run.smithi148.stdout:(13/30): ceph-common-18.2.4-871.g6a1bdf85.el9.x 14 MB/s | 18 MB 00:01 2024-09-21T04:01:56.588 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:01:56.623 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:01:56.623 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:01:56.631 INFO:teuthology.orchestra.run.smithi148.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 410 kB/s | 107 kB 00:00 2024-09-21T04:01:56.664 INFO:teuthology.orchestra.run.smithi148.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 482 kB/s | 45 kB 00:00 2024-09-21T04:01:56.706 INFO:teuthology.orchestra.run.smithi148.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 515 kB/s | 160 kB 00:00 2024-09-21T04:01:56.731 INFO:teuthology.orchestra.run.smithi148.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-21T04:01:56.757 INFO:teuthology.orchestra.run.smithi148.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-09-21T04:01:56.782 INFO:teuthology.orchestra.run.smithi148.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T04:01:56.841 INFO:teuthology.orchestra.run.smithi148.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.6 MB/s | 308 kB 00:00 2024-09-21T04:01:56.866 INFO:teuthology.orchestra.run.smithi148.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-21T04:01:56.891 INFO:teuthology.orchestra.run.smithi148.stdout:(22/30): fmt-8.1.1-5.el9.x86_64.rpm 695 kB/s | 111 kB 00:00 2024-09-21T04:01:56.916 INFO:teuthology.orchestra.run.smithi148.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-21T04:01:56.942 INFO:teuthology.orchestra.run.smithi148.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T04:01:56.975 INFO:teuthology.orchestra.run.smithi148.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T04:01:57.009 INFO:teuthology.orchestra.run.smithi148.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-21T04:01:57.059 INFO:teuthology.orchestra.run.smithi148.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-21T04:01:57.151 INFO:teuthology.orchestra.run.smithi148.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T04:01:57.210 INFO:teuthology.orchestra.run.smithi148.stdout:(29/30): librados2-18.2.4-871.g6a1bdf85.el9.x86 17 MB/s | 3.3 MB 00:00 2024-09-21T04:01:57.295 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:01:57.296 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:01:57.301 INFO:teuthology.orchestra.run.smithi148.stdout:(30/30): librbd1-18.2.4-871.g6a1bdf85.el9.x86_6 12 MB/s | 3.0 MB 00:00 2024-09-21T04:01:57.307 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:01:57.307 INFO:teuthology.orchestra.run.smithi148.stdout:Total 24 MB/s | 53 MB 00:02 2024-09-21T04:01:57.376 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:01:57.410 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:01:57.411 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:01:58.041 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:01:58.041 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:01:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:01:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-21T04:01:58.845 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-21T04:01:58.895 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-21T04:01:59.275 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-21T04:01:59.327 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:01:59.419 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:01:59.454 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:01:59.503 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 6/32 2024-09-21T04:01:59.529 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:01:59.550 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-21T04:01:59.581 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-21T04:01:59.592 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:01:59.621 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-21T04:01:59.631 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-21T04:01:59.664 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x 9/32 2024-09-21T04:01:59.730 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 10/32 2024-09-21T04:01:59.765 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:01:59.831 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:01:59.936 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-21T04:01:59.972 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-21T04:02:00.001 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-21T04:02:00.013 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-21T04:02:00.054 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-21T04:02:00.055 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:02:00.081 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-21T04:02:00.117 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-21T04:02:00.150 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:02:00.160 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-21T04:02:00.248 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 6/32 2024-09-21T04:02:00.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:02:00.337 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:02:00.373 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-21T04:02:00.416 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x 9/32 2024-09-21T04:02:00.458 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-21T04:02:00.482 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 10/32 2024-09-21T04:02:00.496 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 20/32 2024-09-21T04:02:00.509 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:02:00.575 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:02:00.587 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 20/32 2024-09-21T04:02:00.618 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 21/32 2024-09-21T04:02:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-21T04:02:00.680 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-21T04:02:00.708 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-21T04:02:00.757 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-21T04:02:00.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-21T04:02:00.816 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-21T04:02:00.852 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-21T04:02:00.886 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-21T04:02:01.057 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-21T04:02:01.082 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 24/32 2024-09-21T04:02:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 24/32 2024-09-21T04:02:01.190 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-21T04:02:01.232 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 20/32 2024-09-21T04:02:01.255 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 25/32 2024-09-21T04:02:01.285 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86 26/32 2024-09-21T04:02:01.322 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 20/32 2024-09-21T04:02:01.353 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 21/32 2024-09-21T04:02:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-21T04:02:01.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-21T04:02:01.851 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 24/32 2024-09-21T04:02:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 24/32 2024-09-21T04:02:02.024 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 25/32 2024-09-21T04:02:02.053 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86 26/32 2024-09-21T04:02:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:03.424 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 28/32 2024-09-21T04:02:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 28/32 2024-09-21T04:02:04.083 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T04:02:04.083 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T04:02:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:04.234 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:04.275 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 27/32 2024-09-21T04:02:04.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 28/32 2024-09-21T04:02:04.962 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 28/32 2024-09-21T04:02:04.962 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T04:02:04.962 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T04:02:04.962 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:04.997 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:13.880 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:13.880 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-21T04:02:13.880 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-21T04:02:13.880 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-21T04:02:13.881 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-21T04:02:13.881 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-21T04:02:13.881 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-21T04:02:13.881 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-21T04:02:13.881 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/32 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/32 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T04:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T04:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-21T04:02:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:15.570 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/32 2024-09-21T04:02:15.595 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/32 2024-09-21T04:02:15.595 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:15.595 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T04:02:15.595 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:02:15.595 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:02:15.596 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:15.597 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T04:02:15.680 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-21T04:02:15.681 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:15.847 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:15.847 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/32 2024-09-21T04:02:15.847 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/32 2024-09-21T04:02:15.847 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 6/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x 8/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86 9/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 10/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 12/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 13/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-21T04:02:15.848 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-21T04:02:15.849 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:15.850 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-21T04:02:15.850 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 6/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 7/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x 8/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86 9/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 10/32 2024-09-21T04:02:16.823 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 11/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 12/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 13/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-21T04:02:16.824 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-21T04:02:16.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/32 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T04:02:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T04:02:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T04:02:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:16.930 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:17.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-test 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T04:02:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T04:02:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T04:02:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:17.791 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:25 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:18.022 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:18.022 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:02:18.022 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 40 M 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 166 k 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-21T04:02:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:18.024 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 40 M 2024-09-21T04:02:18.024 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 164 M 2024-09-21T04:02:18.024 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:18.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-09-21T04:02:18.318 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-21T04:02:18.360 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T04:02:18.385 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): libcephsqlite-18.2.4-871.g6a1bdf85.el9.x 724 kB/s | 166 kB 00:00 2024-09-21T04:02:18.716 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:26 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:18.849 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 40 M 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 166 k 2024-09-21T04:02:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:18.851 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 40 M 2024-09-21T04:02:18.852 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 164 M 2024-09-21T04:02:18.852 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:19.261 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): libcephsqlite-18.2.4-871.g6a1bdf85.el9.x 585 kB/s | 166 kB 00:00 2024-09-21T04:02:19.486 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): ceph-test-18.2.4-871.g6a1bdf85.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-21T04:02:19.488 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:19.488 INFO:teuthology.orchestra.run.smithi110.stdout:Total 27 MB/s | 40 MB 00:01 2024-09-21T04:02:19.515 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:19.534 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:19.534 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 474 kB/s | 304 kB 00:00 2024-09-21T04:02:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 166 kB/s | 64 kB 00:00 2024-09-21T04:02:19.850 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:19.850 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:20.324 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:20.353 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): ceph-test-18.2.4-871.g6a1bdf85.el9.x86_6 29 MB/s | 40 MB 00:01 2024-09-21T04:02:20.355 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:20.355 INFO:teuthology.orchestra.run.smithi148.stdout:Total 27 MB/s | 40 MB 00:01 2024-09-21T04:02:20.383 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:20.402 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:20.402 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:20.404 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T04:02:20.443 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T04:02:20.471 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/4 2024-09-21T04:02:20.721 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:20.721 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:21.199 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:21.289 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T04:02:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T04:02:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/4 2024-09-21T04:02:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/4 2024-09-21T04:02:24.261 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/4 2024-09-21T04:02:25.068 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/4 2024-09-21T04:02:25.144 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/4 2024-09-21T04:02:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/4 2024-09-21T04:02:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/4 2024-09-21T04:02:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/4 2024-09-21T04:02:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:26.558 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:26.906 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph 2024-09-21T04:02:27.069 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/4 2024-09-21T04:02:27.069 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/4 2024-09-21T04:02:27.069 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/4 2024-09-21T04:02:27.070 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T04:02:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T04:02:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:27.386 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:27.488 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:35 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 6.5 k 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 2.1 M 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 1.5 M 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 246 k 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 4.7 M 2024-09-21T04:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 17 M 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T04:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T04:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout:Install 33 Packages 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 29 M 2024-09-21T04:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 104 M 2024-09-21T04:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:27.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-09-21T04:02:28.205 INFO:teuthology.orchestra.run.smithi110.stdout:(1/33): ceph-18.2.4-871.g6a1bdf85.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-21T04:02:28.270 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:36 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:28.372 INFO:teuthology.orchestra.run.smithi110.stdout:(2/33): ceph-mgr-18.2.4-871.g6a1bdf85.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-21T04:02:28.404 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 6.5 k 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 2.1 M 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 1.5 M 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 246 k 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 4.7 M 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 17 M 2024-09-21T04:02:28.407 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T04:02:28.408 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T04:02:28.409 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Install 33 Packages 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 29 M 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 104 M 2024-09-21T04:02:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:28.439 INFO:teuthology.orchestra.run.smithi110.stdout:(3/33): ceph-mds-18.2.4-871.g6a1bdf85.el9.x86_6 5.9 MB/s | 2.1 MB 00:00 2024-09-21T04:02:28.531 INFO:teuthology.orchestra.run.smithi110.stdout:(4/33): ceph-mon-18.2.4-871.g6a1bdf85.el9.x86_6 14 MB/s | 4.7 MB 00:00 2024-09-21T04:02:28.557 INFO:teuthology.orchestra.run.smithi110.stdout:(5/33): ceph-mgr-modules-core-18.2.4-871.g6a1bd 2.1 MB/s | 246 kB 00:00 2024-09-21T04:02:28.723 INFO:teuthology.orchestra.run.smithi110.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T04:02:28.774 INFO:teuthology.orchestra.run.smithi110.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T04:02:28.941 INFO:teuthology.orchestra.run.smithi110.stdout:(8/33): ceph-osd-18.2.4-871.g6a1bdf85.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-21T04:02:28.966 INFO:teuthology.orchestra.run.smithi110.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 705 kB/s | 135 kB 00:00 2024-09-21T04:02:28.970 INFO:teuthology.orchestra.run.smithi148.stdout:(1/33): ceph-18.2.4-871.g6a1bdf85.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-21T04:02:29.050 INFO:teuthology.orchestra.run.smithi110.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-21T04:02:29.083 INFO:teuthology.orchestra.run.smithi110.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 2.3 MB/s | 1.2 MB 00:00 2024-09-21T04:02:29.108 INFO:teuthology.orchestra.run.smithi110.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 747 kB/s | 125 kB 00:00 2024-09-21T04:02:29.162 INFO:teuthology.orchestra.run.smithi148.stdout:(2/33): ceph-mgr-18.2.4-871.g6a1bdf85.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-21T04:02:29.212 INFO:teuthology.orchestra.run.smithi148.stdout:(3/33): ceph-mds-18.2.4-871.g6a1bdf85.el9.x86_6 5.8 MB/s | 2.1 MB 00:00 2024-09-21T04:02:29.304 INFO:teuthology.orchestra.run.smithi148.stdout:(4/33): ceph-mgr-modules-core-18.2.4-871.g6a1bd 2.6 MB/s | 246 kB 00:00 2024-09-21T04:02:29.342 INFO:teuthology.orchestra.run.smithi110.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 134 kB/s | 35 kB 00:00 2024-09-21T04:02:29.371 INFO:teuthology.orchestra.run.smithi148.stdout:(5/33): ceph-mon-18.2.4-871.g6a1bdf85.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-21T04:02:29.443 INFO:teuthology.orchestra.run.smithi110.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 153 kB/s | 51 kB 00:00 2024-09-21T04:02:29.488 INFO:teuthology.orchestra.run.smithi148.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T04:02:29.493 INFO:teuthology.orchestra.run.smithi110.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 872 kB/s | 43 kB 00:00 2024-09-21T04:02:29.519 INFO:teuthology.orchestra.run.smithi110.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-21T04:02:29.530 INFO:teuthology.orchestra.run.smithi148.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T04:02:29.552 INFO:teuthology.orchestra.run.smithi110.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-21T04:02:29.563 INFO:teuthology.orchestra.run.smithi148.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-09-21T04:02:29.577 INFO:teuthology.orchestra.run.smithi110.stdout:(18/33): python3-mako-1.1.4-6.el9.noarch.rpm 326 kB/s | 172 kB 00:00 2024-09-21T04:02:29.603 INFO:teuthology.orchestra.run.smithi110.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-09-21T04:02:29.636 INFO:teuthology.orchestra.run.smithi110.stdout:(20/33): python3-toml-0.10.2-6.el9.noarch.rpm 142 kB/s | 42 kB 00:00 2024-09-21T04:02:29.662 INFO:teuthology.orchestra.run.smithi110.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 451 kB/s | 23 kB 00:00 2024-09-21T04:02:29.695 INFO:teuthology.orchestra.run.smithi110.stdout:(22/33): python3-jaraco-classes-3.2.1-5.el9.noa 150 kB/s | 18 kB 00:00 2024-09-21T04:02:29.721 INFO:teuthology.orchestra.run.smithi110.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 384 kB/s | 20 kB 00:00 2024-09-21T04:02:29.739 INFO:teuthology.orchestra.run.smithi148.stdout:(9/33): ceph-osd-18.2.4-871.g6a1bdf85.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-21T04:02:29.746 INFO:teuthology.orchestra.run.smithi110.stdout:(24/33): python3-jaraco-functools-3.5.0-2.el9.n 181 kB/s | 19 kB 00:00 2024-09-21T04:02:29.781 INFO:teuthology.orchestra.run.smithi148.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-09-21T04:02:29.783 INFO:teuthology.orchestra.run.smithi110.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 541 kB/s | 46 kB 00:00 2024-09-21T04:02:29.806 INFO:teuthology.orchestra.run.smithi148.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 516 kB/s | 125 kB 00:00 2024-09-21T04:02:29.808 INFO:teuthology.orchestra.run.smithi110.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 903 kB/s | 79 kB 00:00 2024-09-21T04:02:29.834 INFO:teuthology.orchestra.run.smithi110.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-21T04:02:29.859 INFO:teuthology.orchestra.run.smithi110.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-21T04:02:29.884 INFO:teuthology.orchestra.run.smithi110.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T04:02:29.889 INFO:teuthology.orchestra.run.smithi148.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.4 MB/s | 215 kB 00:00 2024-09-21T04:02:29.918 INFO:teuthology.orchestra.run.smithi110.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-21T04:02:29.943 INFO:teuthology.orchestra.run.smithi110.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T04:02:29.969 INFO:teuthology.orchestra.run.smithi110.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-21T04:02:30.002 INFO:teuthology.orchestra.run.smithi110.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-21T04:02:30.007 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:30.007 INFO:teuthology.orchestra.run.smithi110.stdout:Total 12 MB/s | 29 MB 00:02 2024-09-21T04:02:30.090 INFO:teuthology.orchestra.run.smithi148.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 122 kB/s | 35 kB 00:00 2024-09-21T04:02:30.131 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:30.169 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:30.170 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:30.265 INFO:teuthology.orchestra.run.smithi148.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 136 kB/s | 51 kB 00:00 2024-09-21T04:02:30.290 INFO:teuthology.orchestra.run.smithi148.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-09-21T04:02:30.316 INFO:teuthology.orchestra.run.smithi148.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 865 kB/s | 43 kB 00:00 2024-09-21T04:02:30.349 INFO:teuthology.orchestra.run.smithi148.stdout:(17/33): python3-mako-1.1.4-6.el9.noarch.rpm 303 kB/s | 172 kB 00:00 2024-09-21T04:02:30.383 INFO:teuthology.orchestra.run.smithi148.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 5.4 MB/s | 358 kB 00:00 2024-09-21T04:02:30.408 INFO:teuthology.orchestra.run.smithi148.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 182 kB/s | 11 kB 00:00 2024-09-21T04:02:30.442 INFO:teuthology.orchestra.run.smithi148.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-21T04:02:30.467 INFO:teuthology.orchestra.run.smithi148.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 449 kB/s | 23 kB 00:00 2024-09-21T04:02:30.492 INFO:teuthology.orchestra.run.smithi148.stdout:(22/33): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-09-21T04:02:30.518 INFO:teuthology.orchestra.run.smithi148.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 264 kB/s | 19 kB 00:00 2024-09-21T04:02:30.543 INFO:teuthology.orchestra.run.smithi148.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-21T04:02:30.568 INFO:teuthology.orchestra.run.smithi148.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 927 kB/s | 46 kB 00:00 2024-09-21T04:02:30.582 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:30.582 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:30.593 INFO:teuthology.orchestra.run.smithi148.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-21T04:02:30.619 INFO:teuthology.orchestra.run.smithi148.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-21T04:02:30.644 INFO:teuthology.orchestra.run.smithi148.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-21T04:02:30.669 INFO:teuthology.orchestra.run.smithi148.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-21T04:02:30.694 INFO:teuthology.orchestra.run.smithi148.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-21T04:02:30.720 INFO:teuthology.orchestra.run.smithi148.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-21T04:02:30.745 INFO:teuthology.orchestra.run.smithi148.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T04:02:30.770 INFO:teuthology.orchestra.run.smithi148.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-21T04:02:30.775 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:30.775 INFO:teuthology.orchestra.run.smithi148.stdout:Total 12 MB/s | 29 MB 00:02 2024-09-21T04:02:30.899 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:30.937 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:30.938 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:31.333 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:31.333 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:31.789 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:31.839 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-21T04:02:31.870 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-21T04:02:31.904 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-21T04:02:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-21T04:02:32.082 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-21T04:02:32.132 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-21T04:02:32.167 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-21T04:02:32.205 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-21T04:02:32.248 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-21T04:02:32.282 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-21T04:02:32.331 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-21T04:02:32.375 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-21T04:02:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-21T04:02:32.456 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-21T04:02:32.538 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-21T04:02:32.602 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:32.679 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-21T04:02:32.681 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-21T04:02:32.716 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-21T04:02:32.750 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-21T04:02:32.755 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-21T04:02:32.809 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-21T04:02:32.851 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-21T04:02:32.865 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-21T04:02:32.936 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-21T04:02:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-21T04:02:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-21T04:02:32.993 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-21T04:02:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-21T04:02:33.042 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-21T04:02:33.077 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-21T04:02:33.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-21T04:02:33.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-21T04:02:33.176 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-21T04:02:33.195 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-21T04:02:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-21T04:02:33.232 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-21T04:02:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-21T04:02:33.301 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-21T04:02:33.342 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-21T04:02:33.392 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-21T04:02:33.431 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-21T04:02:33.493 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-21T04:02:33.547 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-21T04:02:33.585 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-21T04:02:33.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-21T04:02:33.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-21T04:02:33.724 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.n 28/33 2024-09-21T04:02:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-21T04:02:33.759 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/33 2024-09-21T04:02:33.785 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/33 2024-09-21T04:02:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T04:02:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:02:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:02:33.786 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:33.805 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-21T04:02:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-21T04:02:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-21T04:02:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-21T04:02:34.240 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-21T04:02:34.287 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-21T04:02:34.402 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-21T04:02:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.n 28/33 2024-09-21T04:02:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/33 2024-09-21T04:02:34.615 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 29/33 2024-09-21T04:02:34.616 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:34.616 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T04:02:34.616 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:02:34.616 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:02:34.616 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:35.201 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/33 2024-09-21T04:02:35.228 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/33 2024-09-21T04:02:35.228 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:35.228 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T04:02:35.229 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:02:35.229 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:02:35.229 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:35.669 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/33 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/33 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:02:35.693 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 32/33 2024-09-21T04:02:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 32/33 2024-09-21T04:02:35.927 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:35.927 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T04:02:35.927 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:02:35.927 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:02:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:35.991 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 33/33 2024-09-21T04:02:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/33 2024-09-21T04:02:36.050 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 30/33 2024-09-21T04:02:36.050 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:36.050 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T04:02:36.051 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:02:36.051 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:02:36.051 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:36.499 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/33 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 31/33 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:02:36.523 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:36.732 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 32/33 2024-09-21T04:02:36.756 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 32/33 2024-09-21T04:02:36.756 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:36.757 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T04:02:36.757 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:02:36.757 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:02:36.757 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:36.820 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 33/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 33/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/33 2024-09-21T04:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.n 6/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-21T04:02:37.814 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-21T04:02:37.815 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 33/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 3/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 4/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 5/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.n 6/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-21T04:02:38.605 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-21T04:02:38.606 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-21T04:02:38.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T04:02:38.764 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T04:02:38.765 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:38.766 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:39.100 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-base 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T04:02:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:39.485 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:39.690 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:47 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:39.780 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:02:39.822 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:39.824 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:02:39.824 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:39.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-09-21T04:02:39.888 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install cephadm 2024-09-21T04:02:40.467 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:48 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:40.467 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:48 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:40.556 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:02:40.596 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:40.597 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 226 k 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:40.598 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 226 k 2024-09-21T04:02:40.599 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 217 k 2024-09-21T04:02:40.599 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:40.599 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:40.599 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:02:40.599 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:40.665 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-09-21T04:02:40.828 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm-18.2.4-871.g6a1bdf85.el9.noarch.rpm 986 kB/s | 226 kB 00:00 2024-09-21T04:02:40.829 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:40.829 INFO:teuthology.orchestra.run.smithi110.stdout:Total 979 kB/s | 226 kB 00:00 2024-09-21T04:02:40.830 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:40.834 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:40.834 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:40.837 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:40.838 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:40.904 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:41.244 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:49 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:41.283 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:41.375 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:41.375 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:41.378 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:41.378 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:02:41.378 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:41.378 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:41.378 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 226 k 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 226 k 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 217 k 2024-09-21T04:02:41.379 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:41.613 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-18.2.4-871.g6a1bdf85.el9.noarch.rpm 957 kB/s | 226 kB 00:00 2024-09-21T04:02:41.614 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:41.614 INFO:teuthology.orchestra.run.smithi148.stdout:Total 951 kB/s | 226 kB 00:00 2024-09-21T04:02:41.614 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:41.619 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:41.619 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:41.622 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:41.622 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:41.721 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:41.970 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:42.189 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T04:02:42.218 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:42.596 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:42.778 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:50 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:42.858 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/1 2024-09-21T04:02:42.859 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:42.859 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:42.859 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:42.859 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:42.859 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:42.916 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 142 k 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:42.918 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:02:42.918 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:42.918 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 142 k 2024-09-21T04:02:42.918 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 395 k 2024-09-21T04:02:42.918 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:43.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T04:02:43.127 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-immutable-object-cache-18.2.4-871.g6a1bdf8 684 kB/s | 142 kB 00:00 2024-09-21T04:02:43.128 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:43.128 INFO:teuthology.orchestra.run.smithi110.stdout:Total 677 kB/s | 142 kB 00:00 2024-09-21T04:02:43.128 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:43.135 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:43.136 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:43.193 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:43.194 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:43.332 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:43.437 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:43.437 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T04:02:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:43.641 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:51 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:43.785 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:43.785 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:43.785 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 142 k 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 142 k 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 395 k 2024-09-21T04:02:43.786 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:43.994 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-18.2.4-871.g6a1bdf8 687 kB/s | 142 kB 00:00 2024-09-21T04:02:43.994 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:43.995 INFO:teuthology.orchestra.run.smithi148.stdout:Total 682 kB/s | 142 kB 00:00 2024-09-21T04:02:43.995 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:44.002 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:44.002 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:44.060 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:44.061 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:44.204 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:44.287 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:44.314 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:44.315 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:02:44.316 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T04:02:44.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:44.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:44.415 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:44.415 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:44.415 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:44.416 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:44.416 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:44.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr 2024-09-21T04:02:45.232 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:45.256 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.e 1/1 2024-09-21T04:02:45.257 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:45.257 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:45.257 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:02:45.257 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:45.257 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:45.322 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:02:45.364 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:45.365 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:02:45.365 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:45.429 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T04:02:45.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-09-21T04:02:46.013 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:46.069 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:46.145 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 3.5 M 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 24 k 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 16 k 2024-09-21T04:02:46.146 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout:Install 5 Packages 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.8 M 2024-09-21T04:02:46.147 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 87 M 2024-09-21T04:02:46.148 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:02:46.202 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:46.203 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:02:46.203 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:46.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T04:02:46.657 INFO:teuthology.orchestra.run.smithi110.stdout:(1/5): ceph-prometheus-alerts-18.2.4-871.g6a1bd 103 kB/s | 16 kB 00:00 2024-09-21T04:02:46.682 INFO:teuthology.orchestra.run.smithi110.stdout:(2/5): ceph-grafana-dashboards-18.2.4-871.g6a1b 132 kB/s | 24 kB 00:00 2024-09-21T04:02:46.732 INFO:teuthology.orchestra.run.smithi110.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 408 kB/s | 31 kB 00:00 2024-09-21T04:02:46.766 INFO:teuthology.orchestra.run.smithi110.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T04:02:46.849 INFO:teuthology.orchestra.run.smithi110.stdout:(5/5): ceph-mgr-dashboard-18.2.4-871.g6a1bdf85. 10 MB/s | 3.5 MB 00:00 2024-09-21T04:02:46.853 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:46.853 INFO:teuthology.orchestra.run.smithi110.stdout:Total 5.4 MB/s | 3.8 MB 00:00 2024-09-21T04:02:46.854 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:46.856 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:46.861 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:46.861 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:46.943 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:46.943 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:46.989 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:46.989 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:46.989 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:02:46.989 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 3.5 M 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 24 k 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 16 k 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout:Install 5 Packages 2024-09-21T04:02:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:46.991 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.8 M 2024-09-21T04:02:46.991 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 87 M 2024-09-21T04:02:46.991 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:47.183 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:47.281 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-21T04:02:47.297 INFO:teuthology.orchestra.run.smithi148.stdout:(1/5): ceph-grafana-dashboards-18.2.4-871.g6a1b 172 kB/s | 24 kB 00:00 2024-09-21T04:02:47.313 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-21T04:02:47.322 INFO:teuthology.orchestra.run.smithi148.stdout:(2/5): ceph-prometheus-alerts-18.2.4-871.g6a1bd 99 kB/s | 16 kB 00:00 2024-09-21T04:02:47.342 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.no 3/5 2024-09-21T04:02:47.364 INFO:teuthology.orchestra.run.smithi148.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-21T04:02:47.399 INFO:teuthology.orchestra.run.smithi148.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T04:02:47.558 INFO:teuthology.orchestra.run.smithi148.stdout:(5/5): ceph-mgr-dashboard-18.2.4-871.g6a1bdf85. 8.8 MB/s | 3.5 MB 00:00 2024-09-21T04:02:47.561 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:47.562 INFO:teuthology.orchestra.run.smithi148.stdout:Total 6.6 MB/s | 3.8 MB 00:00 2024-09-21T04:02:47.562 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:47.569 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:47.569 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:47.655 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:47.655 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:47.895 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:47.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-21T04:02:48.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-21T04:02:48.041 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.no 3/5 2024-09-21T04:02:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.n 4/5 2024-09-21T04:02:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 5/5 2024-09-21T04:02:49.091 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.n 4/5 2024-09-21T04:02:49.120 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 5/5 2024-09-21T04:02:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 5/5 2024-09-21T04:02:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.n 1/5 2024-09-21T04:02:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 2/5 2024-09-21T04:02:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.no 3/5 2024-09-21T04:02:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-21T04:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-21T04:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:02:50.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T04:02:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 5/5 2024-09-21T04:02:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.n 1/5 2024-09-21T04:02:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 2/5 2024-09-21T04:02:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.no 3/5 2024-09-21T04:02:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:50.457 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:02:50.599 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:50.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T04:02:50.732 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 7.4 M 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T04:02:50.734 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout:Install 12 Packages 2024-09-21T04:02:50.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:02:50.736 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 38 M 2024-09-21T04:02:50.736 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 212 M 2024-09-21T04:02:50.736 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:02:51.099 INFO:teuthology.orchestra.run.smithi110.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-21T04:02:51.199 INFO:teuthology.orchestra.run.smithi110.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-21T04:02:51.233 INFO:teuthology.orchestra.run.smithi110.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-21T04:02:51.254 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:02:51.375 INFO:teuthology.orchestra.run.smithi110.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.4-87 16 MB/s | 7.4 MB 00:00 2024-09-21T04:02:51.388 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 7.4 M 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T04:02:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-09-21T04:02:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:02:51.391 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 38 M 2024-09-21T04:02:51.391 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 212 M 2024-09-21T04:02:51.391 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:02:51.400 INFO:teuthology.orchestra.run.smithi110.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 109 kB/s | 18 kB 00:00 2024-09-21T04:02:51.442 INFO:teuthology.orchestra.run.smithi110.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 576 kB/s | 38 kB 00:00 2024-09-21T04:02:51.534 INFO:teuthology.orchestra.run.smithi110.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T04:02:51.601 INFO:teuthology.orchestra.run.smithi110.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.4 MB/s | 3.0 MB 00:00 2024-09-21T04:02:51.651 INFO:teuthology.orchestra.run.smithi110.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-21T04:02:51.768 INFO:teuthology.orchestra.run.smithi110.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-09-21T04:02:51.852 INFO:teuthology.orchestra.run.smithi110.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 11 MB/s | 4.9 MB 00:00 2024-09-21T04:02:51.873 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-21T04:02:51.932 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-21T04:02:52.032 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-09-21T04:02:52.082 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-21T04:02:52.132 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-21T04:02:52.225 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): ceph-mgr-diskprediction-local-18.2.4-87 15 MB/s | 7.4 MB 00:00 2024-09-21T04:02:52.227 INFO:teuthology.orchestra.run.smithi110.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-21T04:02:52.228 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:52.229 INFO:teuthology.orchestra.run.smithi110.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-21T04:02:52.358 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-09-21T04:02:52.467 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-21T04:02:52.542 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-09-21T04:02:52.580 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:02:52.592 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-21T04:02:52.606 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:02:52.606 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:02:52.667 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.2 MB/s | 4.9 MB 00:00 2024-09-21T04:02:53.009 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-21T04:02:53.010 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:02:53.010 INFO:teuthology.orchestra.run.smithi148.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-21T04:02:53.117 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:02:53.118 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:02:53.366 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:02:53.392 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:02:53.392 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:02:53.887 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:02:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:02:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:02:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T04:02:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:02:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:02:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T04:02:54.335 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T04:02:54.368 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T04:02:54.444 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T04:02:54.665 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:02:54.731 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T04:02:54.774 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:02:54.873 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:02:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T04:02:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T04:02:55.102 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T04:02:55.132 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T04:02:55.214 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T04:02:55.228 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T04:02:55.894 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T04:02:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T04:02:56.449 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:02:57.276 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:02:57.475 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T04:02:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 12/12 2024-09-21T04:02:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T04:02:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 12/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 12/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 1/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T04:03:00.059 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:03:00.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T04:03:00.511 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:00.512 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:00.761 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 12/12 2024-09-21T04:03:00.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf 1/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T04:03:00.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-rook 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T04:03:01.148 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:01.149 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:01.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-09-21T04:03:01.488 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:01.621 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:01.622 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:01.622 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:03:01.622 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 49 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T04:03:01.623 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout:Install 13 Packages 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:01.624 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.2 M 2024-09-21T04:03:01.625 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 26 M 2024-09-21T04:03:01.625 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:01.936 INFO:teuthology.orchestra.run.smithi110.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 164 kB/s | 19 kB 00:00 2024-09-21T04:03:01.961 INFO:teuthology.orchestra.run.smithi110.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 26 kB 00:00 2024-09-21T04:03:01.986 INFO:teuthology.orchestra.run.smithi110.stdout:(3/13): ceph-mgr-rook-18.2.4-871.g6a1bdf85.el9. 294 kB/s | 49 kB 00:00 2024-09-21T04:03:02.045 INFO:teuthology.orchestra.run.smithi110.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-21T04:03:02.070 INFO:teuthology.orchestra.run.smithi110.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-21T04:03:02.085 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:10 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:02.104 INFO:teuthology.orchestra.run.smithi110.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-21T04:03:02.138 INFO:teuthology.orchestra.run.smithi110.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 477 kB/s | 32 kB 00:00 2024-09-21T04:03:02.163 INFO:teuthology.orchestra.run.smithi110.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 247 kB/s | 14 kB 00:00 2024-09-21T04:03:02.188 INFO:teuthology.orchestra.run.smithi110.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-21T04:03:02.214 INFO:teuthology.orchestra.run.smithi110.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-21T04:03:02.220 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 49 k 2024-09-21T04:03:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T04:03:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.2 M 2024-09-21T04:03:02.223 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-09-21T04:03:02.224 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:02.256 INFO:teuthology.orchestra.run.smithi110.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.5 MB/s | 90 kB 00:00 2024-09-21T04:03:02.298 INFO:teuthology.orchestra.run.smithi110.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-21T04:03:02.323 INFO:teuthology.orchestra.run.smithi110.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 441 kB/s | 59 kB 00:00 2024-09-21T04:03:02.325 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:02.326 INFO:teuthology.orchestra.run.smithi110.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-21T04:03:02.391 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:02.406 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:02.406 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:02.492 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:02.493 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 299 kB/s | 26 kB 00:00 2024-09-21T04:03:02.547 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 173 kB/s | 19 kB 00:00 2024-09-21T04:03:02.598 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-09-21T04:03:02.623 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): ceph-mgr-rook-18.2.4-871.g6a1bdf85.el9. 264 kB/s | 49 kB 00:00 2024-09-21T04:03:02.655 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-21T04:03:02.682 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-09-21T04:03:02.716 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 580 kB/s | 54 kB 00:00 2024-09-21T04:03:02.733 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 418 kB/s | 32 kB 00:00 2024-09-21T04:03:02.758 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-21T04:03:02.791 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-21T04:03:02.833 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-21T04:03:02.859 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.5 MB/s | 220 kB 00:00 2024-09-21T04:03:02.892 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-21T04:03:02.895 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:02.895 INFO:teuthology.orchestra.run.smithi148.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-21T04:03:02.962 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:02.976 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:02.976 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:02.990 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:03.068 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:03.069 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:03.153 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T04:03:03.207 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T04:03:03.263 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T04:03:03.300 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T04:03:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T04:03:03.444 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T04:03:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T04:03:03.594 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:03.602 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T04:03:03.758 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T04:03:03.818 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T04:03:03.873 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T04:03:03.911 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T04:03:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T04:03:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T04:03:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T04:03:04.212 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T04:03:04.319 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T04:03:04.358 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T04:03:04.391 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T04:03:04.439 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T04:03:04.461 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 13/13 2024-09-21T04:03:04.927 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T04:03:04.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T04:03:05.001 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T04:03:05.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T04:03:05.071 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 13/13 2024-09-21T04:03:05.309 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 13/13 2024-09-21T04:03:05.309 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 1/13 2024-09-21T04:03:05.309 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T04:03:05.309 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T04:03:05.309 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T04:03:05.310 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T04:03:05.660 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T04:03:05.660 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T04:03:05.661 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T04:03:05.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T04:03:05.662 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:05.662 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 13/13 2024-09-21T04:03:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 1/13 2024-09-21T04:03:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T04:03:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T04:03:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T04:03:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T04:03:05.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T04:03:06.196 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T04:03:06.196 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:06.196 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:06.196 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T04:03:06.197 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T04:03:06.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:06.198 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:06.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T04:03:06.538 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:14 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:06.670 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 139 k 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T04:03:06.671 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 7.0 M 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 33 M 2024-09-21T04:03:06.672 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:06.977 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): ceph-mgr-cephadm-18.2.4-871.g6a1bdf85.el 629 kB/s | 139 kB 00:00 2024-09-21T04:03:07.029 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:15 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:07.044 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 10 MB/s | 548 kB 00:00 2024-09-21T04:03:07.069 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-21T04:03:07.094 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.0 MB/s | 74 kB 00:00 2024-09-21T04:03:07.162 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:07.163 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:03:07.163 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:18.2.4-871.g6a1bdf85.el9 ceph-noarch 139 k 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-09-21T04:03:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:07.165 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.0 M 2024-09-21T04:03:07.165 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 33 M 2024-09-21T04:03:07.165 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:07.435 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-21T04:03:07.460 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): ceph-mgr-cephadm-18.2.4-871.g6a1bdf85.el 674 kB/s | 139 kB 00:00 2024-09-21T04:03:07.519 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 987 kB/s | 58 kB 00:00 2024-09-21T04:03:07.544 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 315 kB/s | 248 kB 00:00 2024-09-21T04:03:07.569 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-21T04:03:07.703 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-21T04:03:07.706 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:07.706 INFO:teuthology.orchestra.run.smithi148.stdout:Total 13 MB/s | 7.0 MB 00:00 2024-09-21T04:03:07.776 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:07.787 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:07.787 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:07.886 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:07.887 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:07.911 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-21T04:03:07.913 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:07.914 INFO:teuthology.orchestra.run.smithi110.stdout:Total 5.6 MB/s | 7.0 MB 00:01 2024-09-21T04:03:07.986 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:07.997 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:07.998 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:08.096 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:08.098 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T04:03:08.394 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T04:03:08.574 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T04:03:08.622 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T04:03:08.950 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T04:03:09.033 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T04:03:09.103 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T04:03:09.134 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 6/6 2024-09-21T04:03:09.136 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T04:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T04:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T04:03:09.317 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 6/6 2024-09-21T04:03:09.881 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 6/6 2024-09-21T04:03:09.881 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/6 2024-09-21T04:03:09.881 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T04:03:09.881 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T04:03:09.882 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T04:03:09.882 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T04:03:10.127 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 6/6 2024-09-21T04:03:10.127 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 1/6 2024-09-21T04:03:10.127 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T04:03:10.127 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T04:03:10.127 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T04:03:10.128 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T04:03:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T04:03:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:10.146 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:10.147 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:10.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-09-21T04:03:10.390 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:18.2.4-871.g6a1bdf85.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:10.391 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:10.643 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-fuse 2024-09-21T04:03:10.949 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:18 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:11.081 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:11.081 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:11.081 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T04:03:11.081 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:11.081 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 853 k 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-09-21T04:03:11.082 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:11.083 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 932 k 2024-09-21T04:03:11.083 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 2.7 M 2024-09-21T04:03:11.083 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:11.227 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:11.355 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): ceph-fuse-18.2.4-871.g6a1bdf85.el9.x86_6 3.3 MB/s | 853 kB 00:00 2024-09-21T04:03:11.358 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 853 k 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T04:03:11.359 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:Install 2 Packages 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 932 k 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 2.7 M 2024-09-21T04:03:11.360 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:11.697 INFO:teuthology.orchestra.run.smithi110.stdout:(1/2): ceph-fuse-18.2.4-871.g6a1bdf85.el9.x86_6 3.0 MB/s | 853 kB 00:00 2024-09-21T04:03:11.846 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 107 kB/s | 80 kB 00:00 2024-09-21T04:03:11.847 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:11.848 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.2 MB/s | 932 kB 00:00 2024-09-21T04:03:11.861 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:11.881 INFO:teuthology.orchestra.run.smithi110.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-21T04:03:11.881 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:11.882 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.7 MB/s | 932 kB 00:00 2024-09-21T04:03:11.884 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:11.884 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:11.894 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:11.916 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:11.917 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:11.966 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:11.966 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:11.996 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:11.996 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:12.179 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:12.305 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T04:03:12.333 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T04:03:12.378 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/2 2024-09-21T04:03:12.397 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/2 2024-09-21T04:03:13.069 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/2 2024-09-21T04:03:13.069 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/2 2024-09-21T04:03:13.135 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2/2 2024-09-21T04:03:13.135 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/2 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:13.325 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:13.425 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:13.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados-devel 2024-09-21T04:03:13.701 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-09-21T04:03:14.196 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:22 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:14.302 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:22 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 126 k 2024-09-21T04:03:14.334 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:14.335 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:14.335 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:14.335 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:03:14.335 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:14.337 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 126 k 2024-09-21T04:03:14.337 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 456 k 2024-09-21T04:03:14.338 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:14.434 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 126 k 2024-09-21T04:03:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 126 k 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 456 k 2024-09-21T04:03:14.436 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:14.528 INFO:teuthology.orchestra.run.smithi110.stdout:librados-devel-18.2.4-871.g6a1bdf85.el9.x86_64. 656 kB/s | 126 kB 00:00 2024-09-21T04:03:14.529 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:14.529 INFO:teuthology.orchestra.run.smithi110.stdout:Total 652 kB/s | 126 kB 00:00 2024-09-21T04:03:14.529 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:14.535 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:14.535 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:14.597 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:14.597 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-18.2.4-871.g6a1bdf85.el9.x86_64. 567 kB/s | 126 kB 00:00 2024-09-21T04:03:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:Total 563 kB/s | 126 kB 00:00 2024-09-21T04:03:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:14.666 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:14.666 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:14.729 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:14.730 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:14.881 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:15.299 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:15.425 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:15.553 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:15.670 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:15.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs2 2024-09-21T04:03:15.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-09-21T04:03:16.386 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:16.475 INFO:teuthology.orchestra.run.smithi110.stdout:Package libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:16.493 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:16.520 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:16.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs-devel 2024-09-21T04:03:16.584 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:16.631 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:16.631 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:16.632 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:16.697 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-09-21T04:03:17.165 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:17.277 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:17.298 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:17.298 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 31 k 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 31 k 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 139 k 2024-09-21T04:03:17.299 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:17.408 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 31 k 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:17.410 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:03:17.411 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:17.411 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 31 k 2024-09-21T04:03:17.411 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 139 k 2024-09-21T04:03:17.411 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:17.458 INFO:teuthology.orchestra.run.smithi110.stdout:libcephfs-devel-18.2.4-871.g6a1bdf85.el9.x86_64 196 kB/s | 31 kB 00:00 2024-09-21T04:03:17.459 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:17.459 INFO:teuthology.orchestra.run.smithi110.stdout:Total 194 kB/s | 31 kB 00:00 2024-09-21T04:03:17.459 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:17.462 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:17.462 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:17.476 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:17.476 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:17.554 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-18.2.4-871.g6a1bdf85.el9.x86_64 215 kB/s | 31 kB 00:00 2024-09-21T04:03:17.555 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:17.555 INFO:teuthology.orchestra.run.smithi148.stdout:Total 213 kB/s | 31 kB 00:00 2024-09-21T04:03:17.555 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:17.558 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:17.559 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:17.568 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:17.568 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:17.654 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:18.112 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:18.315 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:18.344 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:18.344 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:18.506 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados2 2024-09-21T04:03:18.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-09-21T04:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:19.116 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:19.179 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:19.204 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:19.223 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:19.224 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:19.224 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:19.246 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:19.247 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:19.247 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:19.295 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librbd1 2024-09-21T04:03:19.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-09-21T04:03:19.876 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:19.896 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:19.964 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:19.985 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:20.008 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:20.009 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:20.009 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:20.028 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:20.029 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:20.029 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:20.072 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rados 2024-09-21T04:03:20.088 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-09-21T04:03:20.651 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:20.677 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:20.739 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:20.767 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:20.782 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:20.783 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:20.783 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:20.810 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:20.811 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:20.811 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:20.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rgw 2024-09-21T04:03:20.879 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-09-21T04:03:21.430 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:21.458 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:21.520 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:21.547 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:21.562 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:21.564 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:21.564 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:21.590 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:21.591 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:21.591 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:21.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-cephfs 2024-09-21T04:03:21.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-09-21T04:03:22.217 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:22.237 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:22.307 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:22.327 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:22.349 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:22.350 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:22.350 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:22.369 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:22.371 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:22.371 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:22.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rbd 2024-09-21T04:03:22.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-09-21T04:03:22.995 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:23.026 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:23.085 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:23.115 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 is already installed. 2024-09-21T04:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-21T04:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:23.158 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:23.159 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-21T04:03:23.159 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:23.185 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-fuse 2024-09-21T04:03:23.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-09-21T04:03:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:23.821 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:23.903 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 87 k 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:03:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:23.905 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 87 k 2024-09-21T04:03:23.905 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 227 k 2024-09-21T04:03:23.905 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:23.955 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 87 k 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:23.956 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:23.957 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:03:23.957 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:23.957 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 87 k 2024-09-21T04:03:23.957 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 227 k 2024-09-21T04:03:23.957 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-fuse-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 400 kB/s | 87 kB 00:00 2024-09-21T04:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout:Total 397 kB/s | 87 kB 00:00 2024-09-21T04:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:24.149 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 456 kB/s | 87 kB 00:00 2024-09-21T04:03:24.149 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:24.150 INFO:teuthology.orchestra.run.smithi148.stdout:Total 453 kB/s | 87 kB 00:00 2024-09-21T04:03:24.150 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:24.156 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:24.157 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:24.219 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:24.219 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:24.348 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:24.367 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:24.449 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:24.846 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:25.118 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:25.157 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:25.158 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:25.158 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:25.158 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:25.158 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:25.158 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:25.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-09-21T04:03:25.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-mirror 2024-09-21T04:03:25.955 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:25.999 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:26.090 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:26.090 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:26.090 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-21T04:03:26.090 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.0 M 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.0 M 2024-09-21T04:03:26.091 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 12 M 2024-09-21T04:03:26.092 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:26.135 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:26.135 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:26.135 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-21T04:03:26.135 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:26.135 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 3.0 M 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:26.136 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.0 M 2024-09-21T04:03:26.137 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 12 M 2024-09-21T04:03:26.137 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:26.426 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-21T04:03:26.427 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:26.427 INFO:teuthology.orchestra.run.smithi148.stdout:Total 8.9 MB/s | 3.0 MB 00:00 2024-09-21T04:03:26.427 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:26.436 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:26.436 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:26.490 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-mirror-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 8.5 MB/s | 3.0 MB 00:00 2024-09-21T04:03:26.491 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:26.491 INFO:teuthology.orchestra.run.smithi110.stdout:Total 8.4 MB/s | 3.0 MB 00:00 2024-09-21T04:03:26.491 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:26.501 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:26.501 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:26.515 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:26.515 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:26.582 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:26.582 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:03:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T04:03:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:03:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:03:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:27.003 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:27.058 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:03:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:28.026 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:28.026 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:28.026 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:28.026 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:28.027 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:28.027 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:28.166 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:28.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-09-21T04:03:28.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-nbd 2024-09-21T04:03:28.868 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:29.000 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 172 k 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-21T04:03:29.001 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 172 k 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 494 k 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T04:03:29.021 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 04:01:52 AM UTC. 2024-09-21T04:03:29.155 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:18.2.4-871.g6a1bdf85.el9 ceph 172 k 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-21T04:03:29.156 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:29.157 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 172 k 2024-09-21T04:03:29.157 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 494 k 2024-09-21T04:03:29.157 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T04:03:29.215 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 806 kB/s | 172 kB 00:00 2024-09-21T04:03:29.216 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:29.216 INFO:teuthology.orchestra.run.smithi148.stdout:Total 801 kB/s | 172 kB 00:00 2024-09-21T04:03:29.216 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T04:03:29.224 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T04:03:29.224 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T04:03:29.286 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T04:03:29.287 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T04:03:29.370 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-nbd-18.2.4-871.g6a1bdf85.el9.x86_64.rpm 810 kB/s | 172 kB 00:00 2024-09-21T04:03:29.370 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:03:29.371 INFO:teuthology.orchestra.run.smithi110.stdout:Total 803 kB/s | 172 kB 00:00 2024-09-21T04:03:29.371 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T04:03:29.378 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T04:03:29.378 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T04:03:29.444 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T04:03:29.444 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T04:03:29.457 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T04:03:29.552 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:29.614 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T04:03:29.717 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:30.004 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:30.154 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:30.258 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:30.258 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:30.258 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T04:03:30.259 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:30.259 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:03:30.259 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 1/1 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:18.2.4-871.g6a1bdf85.el9.x86_64 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:03:30.402 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T04:03:30.467 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:30.649 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:30.649 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:03:30.650 INFO:teuthology.packaging:ref: None 2024-09-21T04:03:30.650 INFO:teuthology.packaging:tag: None 2024-09-21T04:03:30.650 INFO:teuthology.packaging:branch: reef 2024-09-21T04:03:30.650 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:03:30.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-21T04:03:30.918 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T04:03:30.939 INFO:teuthology.orchestra.run.smithi110.stdout:18.2.4-871.g6a1bdf85.el9 2024-09-21T04:03:30.939 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-871.g6a1bdf85.el9 2024-09-21T04:03:30.940 INFO:teuthology.task.install:The correct ceph version 18.2.4-871.g6a1bdf85 is installed. 2024-09-21T04:03:30.941 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:03:30.941 INFO:teuthology.packaging:ref: None 2024-09-21T04:03:30.941 INFO:teuthology.packaging:tag: None 2024-09-21T04:03:30.941 INFO:teuthology.packaging:branch: reef 2024-09-21T04:03:30.942 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:03:30.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-21T04:03:31.201 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T04:03:31.225 INFO:teuthology.orchestra.run.smithi148.stdout:18.2.4-871.g6a1bdf85.el9 2024-09-21T04:03:31.226 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-871.g6a1bdf85.el9 2024-09-21T04:03:31.226 INFO:teuthology.task.install:The correct ceph version 18.2.4-871.g6a1bdf85 is installed. 2024-09-21T04:03:31.227 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T04:03:31.228 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:03:31.228 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T04:03:31.253 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:03:31.253 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T04:03:31.294 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T04:03:31.294 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:03:31.294 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T04:03:31.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T04:03:31.382 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:03:31.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T04:03:31.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T04:03:31.477 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T04:03:31.477 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:03:31.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T04:03:31.501 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T04:03:31.566 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:03:31.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T04:03:31.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T04:03:31.658 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T04:03:31.658 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:03:31.658 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T04:03:31.683 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T04:03:31.747 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:03:31.747 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T04:03:31.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T04:03:31.840 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:03:31.849 INFO:teuthology.task.print:**** done install task... 2024-09-21T04:03:31.849 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T04:03:31.956 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'global': {'mon pg warn min per osd': 0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'} 2024-09-21T04:03:31.956 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-09-21T04:03:31.956 INFO:tasks.cephadm:Cluster fsid is 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:03:31.956 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T04:03:31.956 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-21T04:03:31.957 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi110': '172.21.15.110', 'mon.smithi148': '172.21.15.148'} 2024-09-21T04:03:31.957 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T04:03:31.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-21T04:03:31.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-09-21T04:03:32.009 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-09-21T04:03:32.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:03:32.147 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83198/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5103-gf36283e7', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5103.gf36283e7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-2310/f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:53:49.938247', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-20-2310', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-20-2310/f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T04:03:32.404 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 6a1bdf854bb5803efd58957907442aa01f41a8fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-09-21T04:03:32.406 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/x86_64/flavors/default/cephadm 2024-09-21T04:03:32.406 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/x86_64/flavors/default/cephadm 2024-09-21T04:03:32.406 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:32.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 21 04:03 /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:32.895 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/6a1bdf854bb5803efd58957907442aa01f41a8fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:33.381 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 21 04:03 /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:33.381 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:33.401 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T04:03:33.427 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-09-21T04:03:33.427 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-21T04:03:33.443 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-21T04:03:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-21T04:03:35.066 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-21T04:04:27.904 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:04:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)", 2024-09-21T04:04:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de", 2024-09-21T04:04:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-21T04:04:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6" 2024-09-21T04:04:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-21T04:04:27.905 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:05:13.186 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)", 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de", 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6" 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-21T04:05:13.187 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-21T04:05:13.204 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-21T04:05:13.238 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-09-21T04:05:13.266 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-21T04:05:13.307 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-09-21T04:05:13.332 INFO:tasks.cephadm:Writing seed config... 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T04:05:13.333 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T04:05:13.334 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T04:05:13.335 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T04:05:13.335 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T04:05:13.335 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:05:13.336 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T04:05:13.365 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 77422328-77ce-11ef-baf5-efdc52797490 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T04:05:13.365 DEBUG:teuthology.orchestra.run.smithi110:mon.smithi110> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service 2024-09-21T04:05:13.408 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T04:05:13.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 77422328-77ce-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.110 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:05:13.565 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:05:13.566 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.110', '--skip-admin-label'] 2024-09-21T04:05:13.678 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-21T04:05:13.678 INFO:teuthology.orchestra.run.smithi110.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T04:05:13.678 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying podman|docker is present... 2024-09-21T04:05:13.699 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-21T04:05:13.699 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying lvm2 is present... 2024-09-21T04:05:13.699 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying time synchronization is in place... 2024-09-21T04:05:13.706 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T04:05:13.707 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T04:05:13.713 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T04:05:13.713 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:05:13.719 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-09-21T04:05:13.725 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-09-21T04:05:13.726 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-09-21T04:05:13.726 INFO:teuthology.orchestra.run.smithi110.stdout:Repeating the final host check... 2024-09-21T04:05:13.748 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 5.2.2 2024-09-21T04:05:13.748 INFO:teuthology.orchestra.run.smithi110.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T04:05:13.748 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl is present 2024-09-21T04:05:13.748 INFO:teuthology.orchestra.run.smithi110.stdout:lvcreate is present 2024-09-21T04:05:13.754 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T04:05:13.754 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T04:05:13.760 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T04:05:13.761 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:05:13.767 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-09-21T04:05:13.773 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-09-21T04:05:13.773 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-09-21T04:05:13.773 INFO:teuthology.orchestra.run.smithi110.stdout:Host looks OK 2024-09-21T04:05:13.773 INFO:teuthology.orchestra.run.smithi110.stdout:Cluster fsid: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:13.773 INFO:teuthology.orchestra.run.smithi110.stdout:Acquiring lock 140325152740688 on /run/cephadm/77422328-77ce-11ef-baf5-efdc52797490.lock 2024-09-21T04:05:13.774 INFO:teuthology.orchestra.run.smithi110.stdout:Lock 140325152740688 acquired on /run/cephadm/77422328-77ce-11ef-baf5-efdc52797490.lock 2024-09-21T04:05:13.774 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 3300 ... 2024-09-21T04:05:13.774 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 6789 ... 2024-09-21T04:05:13.774 INFO:teuthology.orchestra.run.smithi110.stdout:Base mon IP(s) is [172.21.15.110:3300, 172.21.15.110:6789], mon addrv is [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-09-21T04:05:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.110 metric 100 2024-09-21T04:05:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.110 metric 100 2024-09-21T04:05:13.780 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T04:05:13.780 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T04:05:13.780 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link noprefixroute 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T04:05:13.783 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-09-21T04:05:13.784 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-09-21T04:05:13.784 INFO:teuthology.orchestra.run.smithi110.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T04:05:13.784 INFO:teuthology.orchestra.run.smithi110.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T04:05:13.784 INFO:teuthology.orchestra.run.smithi110.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-21T04:05:14.296 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:b51f6d1e10fe8de9e06c08c0a23ce0010d2bef42d30a6f89f3ac07fd857715ba 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying config sha256:0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de 2024-09-21T04:05:14.297 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T04:05:15.260 INFO:teuthology.orchestra.run.smithi110.stdout:ceph: stdout ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable) 2024-09-21T04:05:15.260 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph version: ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable) 2024-09-21T04:05:15.261 INFO:teuthology.orchestra.run.smithi110.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T04:05:16.199 INFO:teuthology.orchestra.run.smithi110.stdout:stat: stdout 167 167 2024-09-21T04:05:16.199 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial keys... 2024-09-21T04:05:17.213 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQD8Re5ml4QAIBAAvhvm1NcQoNu5zCKZdwa+3w== 2024-09-21T04:05:18.227 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQD9Re5mPfzbHhAAHPS15eG7PlQtQjk8wiZMsQ== 2024-09-21T04:05:19.150 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQD+Re5mu83QIBAA73/ikihJU7lSRnpxAGwenQ== 2024-09-21T04:05:19.150 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial monmap... 2024-09-21T04:05:20.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T04:05:20.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-21T04:05:20.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool for smithi110 [v2:172.21.15.110:3300,v1:172.21.15.110:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = pacific 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: set fsid to 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:05:20.141 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mon... 2024-09-21T04:05:21.079 INFO:teuthology.orchestra.run.smithi110.stdout:create mon.smithi110 on 2024-09-21T04:05:21.277 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T04:05:21.419 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T04:05:21.597 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-77422328-77ce-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490.target. 2024-09-21T04:05:21.597 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-77422328-77ce-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490.target. 2024-09-21T04:05:21.819 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110 2024-09-21T04:05:21.819 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service: Unit ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service not loaded. 2024-09-21T04:05:21.978 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490.target.wants/ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service → /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490@.service. 2024-09-21T04:05:22.447 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:22 smithi110 podman[27337]: 2024-09-21 04:05:22.407758711 +0000 UTC m=+0.314508470 container start ad4f11e33f28498346215654954fac265871ef1b64ed802475a6d59292eb38f4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-6a1bdf8, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T04:05:22.528 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:05:22.528 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-21T04:05:22.535 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:05:22.535 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:05:22.535 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-21T04:05:22.535 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T04:05:22.536 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon to start... 2024-09-21T04:05:22.536 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon... 2024-09-21T04:05:22.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:22 smithi110 ceph-mon[27356]: mkfs 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:22.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:22 smithi110 ceph-mon[27356]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-09-21T04:05:22.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:22 smithi110 bash[27337]: ad4f11e33f28498346215654954fac265871ef1b64ed802475a6d59292eb38f4 2024-09-21T04:05:22.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:22 smithi110 systemd[1]: Started Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout id: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout services: 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi110 (age 0.565804s) 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout data: 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:mon is available 2024-09-21T04:05:23.535 INFO:teuthology.orchestra.run.smithi110.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: monmap e1: 1 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: fsmap 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: mgrmap e1: no daemons active 2024-09-21T04:05:23.647 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:23 smithi110 ceph-mon[27356]: from='client.? 172.21.15.110:0/1767041243' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T04:05:24.557 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T04:05:24.558 INFO:teuthology.orchestra.run.smithi110.stdout:Generating new minimal ceph.conf... 2024-09-21T04:05:24.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:24 smithi110 ceph-mon[27356]: from='client.? 172.21.15.110:0/1645704750' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-21T04:05:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:24 smithi110 ceph-mon[27356]: from='client.? 172.21.15.110:0/1645704750' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T04:05:25.634 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:25 smithi110 ceph-mon[27356]: from='client.? 172.21.15.110:0/2831148193' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:05:25.637 INFO:teuthology.orchestra.run.smithi110.stdout:Restarting the monitor... 2024-09-21T04:05:25.913 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:25 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:05:26.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:25 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[27352]: 2024-09-21T04:05:25.910+0000 7f7755301640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:05:26.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:25 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[27352]: 2024-09-21T04:05:25.910+0000 7f7755301640 -1 mon.smithi110@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T04:05:26.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 podman[27600]: 2024-09-21 04:05:26.156381608 +0000 UTC m=+0.355248230 container died ad4f11e33f28498346215654954fac265871ef1b64ed802475a6d59292eb38f4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-6a1bdf8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:05:26.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 podman[27600]: 2024-09-21 04:05:26.510612467 +0000 UTC m=+0.709479087 container cleanup ad4f11e33f28498346215654954fac265871ef1b64ed802475a6d59292eb38f4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-6a1bdf8) 2024-09-21T04:05:26.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 bash[27600]: ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110 2024-09-21T04:05:27.133 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 podman[27613]: 2024-09-21 04:05:26.7698624 +0000 UTC m=+0.606104313 container remove ad4f11e33f28498346215654954fac265871ef1b64ed802475a6d59292eb38f4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-6a1bdf8, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T04:05:27.134 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service: Deactivated successfully. 2024-09-21T04:05:27.134 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 systemd[1]: Stopped Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:05:27.134 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:26 smithi110 systemd[1]: Starting Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 podman[27704]: 2024-09-21 04:05:27.034603331 +0000 UTC m=+0.015447237 image pull 0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de quay.ceph.io/ceph-ci/ceph:reef 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 podman[27704]: 2024-09-21 04:05:27.137373245 +0000 UTC m=+0.118217190 container create 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=reef-6a1bdf8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 podman[27704]: 2024-09-21 04:05:27.365372442 +0000 UTC m=+0.346216365 container init 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-6a1bdf8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 podman[27704]: 2024-09-21 04:05:27.368752935 +0000 UTC m=+0.349596832 container start 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, ceph=True, RELEASE=reef-6a1bdf8, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:05:27.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable), process ceph-mon, pid 2 2024-09-21T04:05:27.462 INFO:teuthology.orchestra.run.smithi110.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: pidfile_write: ignore empty --pid-file 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: load: jerasure load: lrc 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Git sha 0 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Compile date 2024-09-20 20:34:00 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: DB SUMMARY 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: DB Session ID: K6HEX170U9OLURPZ05WY 2024-09-21T04:05:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi110/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi110/store.db: 000009.log size: 91130 ; 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:05:27.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.env: 0x55950da77bc0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.info_log: 0x55950ebd2dc0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.statistics: (nil) 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.use_fsync: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:05:27.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.db_log_dir: 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.wal_dir: 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:05:27.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:05:27.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:05:27.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:05:27.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:05:27.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.write_buffer_manager: 0x55950ebe0320 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.unordered_write: 0 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:05:27.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.row_cache: None 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.wal_filter: None 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.wal_compression: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:05:27.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:05:27.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_open_files: -1 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Compression algorithms supported: 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kZSTD supported: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:05:27.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000010 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.merge_operator: 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_filter: None 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:05:27.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55950ebd2a20) 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_type: 0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_index_type: 0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_shortening: 1 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: checksum: 4 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: no_block_cache: 0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache: 0x55950ebc71f0 2024-09-21T04:05:27.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_options: 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: capacity : 536870912 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_shard_bits : 4 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: strict_capacity_limit : 0 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_compressed: (nil) 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: persistent_cache: (nil) 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size: 4096 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size_deviation: 10 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_restart_interval: 16 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_block_restart_interval: 1 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_block_size: 4096 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: partition_filters: 0 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: use_delta_encoding: 1 2024-09-21T04:05:27.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: filter_policy: bloomfilter 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: whole_key_filtering: 1 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: verify_compression: 0 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: format_version: 5 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_index_compression: 1 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_align: 0 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_auto_readahead_size: 262144 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: prepopulate_block_cache: 0 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression: NoCompression 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:05:27.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.num_levels: 7 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:05:27.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:05:27.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:05:27.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:05:27.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.ttl: 2592000 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:05:27.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 851bb4f2-0a44-4f16-8fba-304dcb9f1d37 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891527394484, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891527395463, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87165, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 282, "table_properties": {"data_size": 85220, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11738, "raw_average_key_size": 48, "raw_value_size": 78690, "raw_average_value_size": 326, "num_data_blocks": 11, "num_entries": 241, "num_filter_entries": 241, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726891527, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "851bb4f2-0a44-4f16-8fba-304dcb9f1d37", "db_session_id": "K6HEX170U9OLURPZ05WY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891527395557, "job": 1, "event": "recovery_finished"} 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi110/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55950ebf2e00 2024-09-21T04:05:27.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: DB pointer 0x55950ece8000 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** DB Stats ** 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:05:27.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L0 2/0 87.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Sum 2/0 87.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 125.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T04:05:27.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative compaction: 0.00 GB write, 26.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval compaction: 0.00 GB write, 26.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache BinnedLRUCache@0x55950ebc71f0#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: starting mon.smithi110 rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi110 fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???) e1 preinit fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).mds e1 new map 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).mds e1 print_map 2024-09-21T04:05:27.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: e1 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: legacy client fscid: -1 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: No filesystems configured 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: monmap e1: 1 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: fsmap 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 ceph-mon[27723]: mgrmap e1: no daemons active 2024-09-21T04:05:27.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 bash[27704]: 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 2024-09-21T04:05:27.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:27 smithi110 systemd[1]: Started Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:05:28.434 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T04:05:28.455 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:05:28.455 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mgr... 2024-09-21T04:05:28.456 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T04:05:28.456 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T04:05:28.456 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-21T04:05:28.725 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77422328-77ce-11ef-baf5-efdc52797490@mgr.smithi110.axmqzb 2024-09-21T04:05:28.725 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77422328-77ce-11ef-baf5-efdc52797490@mgr.smithi110.axmqzb.service: Unit ceph-77422328-77ce-11ef-baf5-efdc52797490@mgr.smithi110.axmqzb.service not loaded. 2024-09-21T04:05:28.878 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490.target.wants/ceph-77422328-77ce-11ef-baf5-efdc52797490@mgr.smithi110.axmqzb.service → /etc/systemd/system/ceph-77422328-77ce-11ef-baf5-efdc52797490@.service. 2024-09-21T04:05:29.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:28 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/881170793' entity='client.admin' 2024-09-21T04:05:29.566 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:05:29.566 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-21T04:05:29.573 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:05:29.573 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:05:29.573 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-21T04:05:29.573 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T04:05:29.582 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:05:29.582 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr to start... 2024-09-21T04:05:29.589 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr... 2024-09-21T04:05:30.400 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:30 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/463513118' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "77422328-77ce-11ef-baf5-efdc52797490", 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:05:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.701 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:05:30.702 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:05:30.703 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:05:22.444034+0000", 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:05:30.704 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (1/15)... 2024-09-21T04:05:33.355 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:33 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2672161741' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "77422328-77ce-11ef-baf5-efdc52797490", 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:05:33.779 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:05:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:33.781 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:05:33.781 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:05:33.781 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:05:33.781 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:05:33.784 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:05:33.785 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:05:22.444034+0000", 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:05:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (2/15)... 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: mgrmap e2: smithi110.axmqzb(active, starting, since 0.00292639s) 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:05:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:34 smithi110 ceph-mon[27723]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:05:35.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:05:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:05:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: from='mgr.14100 172.21.15.110:0/3779248198' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:35 smithi110 ceph-mon[27723]: mgrmap e3: smithi110.axmqzb(active, since 1.0068s) 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "77422328-77ce-11ef-baf5-efdc52797490", 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:05:36.952 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:05:36.953 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:05:36.954 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:05:36.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:05:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:05:22.444034+0000", 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:05:36.959 INFO:teuthology.orchestra.run.smithi110.stdout:mgr is available 2024-09-21T04:05:37.544 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:37 smithi110 ceph-mon[27723]: mgrmap e4: smithi110.axmqzb(active, since 2s) 2024-09-21T04:05:37.544 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:37 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3952962097' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T04:05:38.107 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T04:05:38.108 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling cephadm module... 2024-09-21T04:05:38.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:38 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1438766818' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-21T04:05:39.519 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:39 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2817117129' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-21T04:05:40.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:40 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2817117129' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T04:05:40.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:40 smithi110 ceph-mon[27723]: mgrmap e5: smithi110.axmqzb(active, since 5s) 2024-09-21T04:05:40.997 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:05:40.997 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "smithi110.axmqzb", 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-09-21T04:05:40.998 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 5... 2024-09-21T04:05:41.248 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:41 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2690339638' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: mgrmap e6: smithi110.axmqzb(active, starting, since 0.00309224s) 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:05:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:05:44.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:44 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:05:45.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:45 smithi110 ceph-mon[27723]: Found migration_current of "None". Setting to last migration. 2024-09-21T04:05:45.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:45 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:45.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:45 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:45.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:45 smithi110 ceph-mon[27723]: mgrmap e7: smithi110.axmqzb(active, since 1.00556s) 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 5 is available 2024-09-21T04:05:45.612 INFO:teuthology.orchestra.run.smithi110.stdout:Setting orchestrator backend to cephadm... 2024-09-21T04:05:46.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:46 smithi110 ceph-mon[27723]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T04:05:46.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:46 smithi110 ceph-mon[27723]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T04:05:46.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:46 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:05:47.287 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: [21/Sep/2024:04:05:45] ENGINE Bus STARTING 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: [21/Sep/2024:04:05:45] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: [21/Sep/2024:04:05:45] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: [21/Sep/2024:04:05:45] ENGINE Bus STARTED 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: [21/Sep/2024:04:05:45] ENGINE Error in HTTPServer.serve 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:05:47.288 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:05:47.289 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: mgrmap e8: smithi110.axmqzb(active, since 2s) 2024-09-21T04:05:47.290 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:47.290 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:47.290 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:47 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:05:47.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T04:05:47.908 INFO:teuthology.orchestra.run.smithi110.stdout:Generating ssh key... 2024-09-21T04:05:48.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:48 smithi110 ceph-mon[27723]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:49 smithi110 ceph-mon[27723]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:49 smithi110 ceph-mon[27723]: Generating ssh key... 2024-09-21T04:05:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:49 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:49 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:50.380 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC768K6LansU80C45pnCANkjMbWFb/Md3+UWh+qd4cyJqEAlyyKdmrPnPSAOp7cjZAumDQqMBbZs2HMaDbROUuCIQUU1HA9Dbk7SBYWdOAJID3q1BQE+PjulI/AYddddCzCHBnHY4DkQOWRcMauy6Mx44AKh0d9caae1wskmLYJ8FkIxGkHWO4tE89CF+L5q5KXwVELK8RbQCZza3xSTVQ5yHpIYMBUlKFgDTPwsqDWRf8/P8FJXL3hKyk2S5eITjRHTLUfoB+ybaUwKoLOArCGvHeUe71s7Cr06n1bSkY1anC+E9ZpzJgY+nkyb3KpcgHK2r9MXNSPrroKh540QUDMJw5tFKJju9lldPV/v9YEssmM+1JqDpZ/YkZf7QiYT/1md82b1S7u3WIFdRChp6S6fSHoew5avG6YQ8zQs30bWtUChbRz4MGtKIk/vsJ84Bh1Up6mVsuUBc4Y/Imi+YsgNoLGwJ1zThKryzeIeQIFGnYvnSx5R7JDG1qem/KIk= ceph-77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:05:50.380 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:05:50.381 INFO:teuthology.orchestra.run.smithi110.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T04:05:50.383 INFO:teuthology.orchestra.run.smithi110.stdout:Adding host smithi110... 2024-09-21T04:05:51.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:50 smithi110 ceph-mon[27723]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:51.700 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:51 smithi110 ceph-mon[27723]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:52.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:52 smithi110 ceph-mon[27723]: Deploying cephadm binary to smithi110 2024-09-21T04:05:54.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Added host 'smithi110' with addr '172.21.15.110' 2024-09-21T04:05:54.208 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying mon service with default placement... 2024-09-21T04:05:54.757 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:54 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:54.757 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:54 smithi110 ceph-mon[27723]: Added host smithi110 2024-09-21T04:05:54.757 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:54 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:05:55.381 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T04:05:55.381 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying mgr service with default placement... 2024-09-21T04:05:56.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:55 smithi110 ceph-mon[27723]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:56.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:55 smithi110 ceph-mon[27723]: Saving service mon spec with placement count:5 2024-09-21T04:05:56.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:55 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:57.389 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:57 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:57.389 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:57 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:57.390 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:57 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:57.930 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T04:05:57.930 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying crash service with default placement... 2024-09-21T04:05:58.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:58 smithi110 ceph-mon[27723]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:58.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:58 smithi110 ceph-mon[27723]: Saving service mgr spec with placement count:2 2024-09-21T04:05:58.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:58 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:59.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-21T04:05:59.103 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying ceph-exporter service with default placement... 2024-09-21T04:05:59.571 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:59 smithi110 ceph-mon[27723]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:05:59.571 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:59 smithi110 ceph-mon[27723]: Saving service crash spec with placement * 2024-09-21T04:05:59.571 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:59 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:05:59.571 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:05:59 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:00.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-21T04:06:00.538 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying prometheus service with default placement... 2024-09-21T04:06:00.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:00 smithi110 ceph-mon[27723]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:00 smithi110 ceph-mon[27723]: Saving service ceph-exporter spec with placement * 2024-09-21T04:06:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:00 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:01.957 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-21T04:06:01.957 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying grafana service with default placement... 2024-09-21T04:06:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:02 smithi110 ceph-mon[27723]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:02 smithi110 ceph-mon[27723]: Saving service prometheus spec with placement count:1 2024-09-21T04:06:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:02 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:03.263 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-21T04:06:03.263 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying node-exporter service with default placement... 2024-09-21T04:06:03.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:03 smithi110 ceph-mon[27723]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:03 smithi110 ceph-mon[27723]: Saving service grafana spec with placement count:1 2024-09-21T04:06:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:03 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:04.377 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-21T04:06:04.377 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying alertmanager service with default placement... 2024-09-21T04:06:04.934 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:04 smithi110 ceph-mon[27723]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:04.934 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:04 smithi110 ceph-mon[27723]: Saving service node-exporter spec with placement * 2024-09-21T04:06:04.934 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:04 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:04.934 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:04 smithi110 ceph-mon[27723]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:05.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-21T04:06:06.086 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:05 smithi110 ceph-mon[27723]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:06.087 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:05 smithi110 ceph-mon[27723]: Saving service alertmanager spec with placement count:1 2024-09-21T04:06:06.087 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:05 smithi110 ceph-mon[27723]: from='mgr.14118 172.21.15.110:0/4072218531' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:07.192 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:07 smithi110 ceph-mon[27723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:07.192 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:07 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/700512715' entity='client.admin' 2024-09-21T04:06:07.780 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling the dashboard module... 2024-09-21T04:06:08.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:08 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2099172603' entity='client.admin' 2024-09-21T04:06:09.542 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:09 smithi110 ceph-mon[27723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:09.542 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:09 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/808024011' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-21T04:06:10.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:10 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/808024011' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T04:06:10.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:10 smithi110 ceph-mon[27723]: mgrmap e9: smithi110.axmqzb(active, since 25s) 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "smithi110.axmqzb", 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T04:06:11.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:06:11.104 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-09-21T04:06:11.104 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 9... 2024-09-21T04:06:11.413 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:11 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/176975379' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: mgrmap e10: smithi110.axmqzb(active, starting, since 0.00299374s) 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:06:14.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:06:14.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:06:14.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:06:14.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 9 is available 2024-09-21T04:06:15.771 INFO:teuthology.orchestra.run.smithi110.stdout:Generating a dashboard self-signed certificate... 2024-09-21T04:06:16.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:15 smithi110 ceph-mon[27723]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T04:06:16.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:15 smithi110 ceph-mon[27723]: mgrmap e11: smithi110.axmqzb(active, since 1.00556s) 2024-09-21T04:06:16.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:15 smithi110 ceph-mon[27723]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T04:06:16.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:15 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:17.103 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-21T04:06:17.103 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial admin user... 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: [21/Sep/2024:04:06:15] ENGINE Bus STARTING 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: [21/Sep/2024:04:06:15] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: [21/Sep/2024:04:06:15] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: [21/Sep/2024:04:06:15] ENGINE Bus STARTED 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: [21/Sep/2024:04:06:15] ENGINE Error in HTTPServer.serve 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:06:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:17.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:17.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:16 smithi110 ceph-mon[27723]: mgrmap e12: smithi110.axmqzb(active, since 2s) 2024-09-21T04:06:18.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:17 smithi110 ceph-mon[27723]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:17 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:19.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:18 smithi110 ceph-mon[27723]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:19.266 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ps2ljXjJYmMv4HUdX7Oj7OD.AUfhGrSSl51.StvWZQ2H19oEKG1u2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726891577, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T04:06:19.267 INFO:teuthology.orchestra.run.smithi110.stdout:Fetching dashboard port number... 2024-09-21T04:06:20.146 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:19 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1349397554' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-21T04:06:20.389 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T04:06:20.398 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:06:20.399 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-09-21T04:06:20.405 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:06:20.405 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-09-21T04:06:20.405 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-09-21T04:06:20.406 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph Dashboard is now available at: 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: URL: https://smithi110.front.sepia.ceph.com:8443/ 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: User: admin 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: Password: kizd7esor1 2024-09-21T04:06:20.407 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:20.408 INFO:teuthology.orchestra.run.smithi110.stdout:Saving cluster configuration to /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config directory 2024-09-21T04:06:20.408 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling autotune for osd_memory_target 2024-09-21T04:06:21.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:21 smithi110 ceph-mon[27723]: mgrmap e13: smithi110.axmqzb(active, since 6s) 2024-09-21T04:06:22.811 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: ceph telemetry on 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout:For more information see: 2024-09-21T04:06:22.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.813 INFO:teuthology.orchestra.run.smithi110.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T04:06:22.813 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:22.813 INFO:teuthology.orchestra.run.smithi110.stdout:Bootstrap complete. 2024-09-21T04:06:22.845 INFO:tasks.cephadm:Fetching config... 2024-09-21T04:06:22.845 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:06:22.845 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T04:06:22.861 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T04:06:22.861 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:06:22.861 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T04:06:22.917 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T04:06:22.917 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:06:22.917 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/keyring of=/dev/stdout 2024-09-21T04:06:22.982 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T04:06:22.982 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:06:22.982 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T04:06:23.037 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T04:06:23.037 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC768K6LansU80C45pnCANkjMbWFb/Md3+UWh+qd4cyJqEAlyyKdmrPnPSAOp7cjZAumDQqMBbZs2HMaDbROUuCIQUU1HA9Dbk7SBYWdOAJID3q1BQE+PjulI/AYddddCzCHBnHY4DkQOWRcMauy6Mx44AKh0d9caae1wskmLYJ8FkIxGkHWO4tE89CF+L5q5KXwVELK8RbQCZza3xSTVQ5yHpIYMBUlKFgDTPwsqDWRf8/P8FJXL3hKyk2S5eITjRHTLUfoB+ybaUwKoLOArCGvHeUe71s7Cr06n1bSkY1anC+E9ZpzJgY+nkyb3KpcgHK2r9MXNSPrroKh540QUDMJw5tFKJju9lldPV/v9YEssmM+1JqDpZ/YkZf7QiYT/1md82b1S7u3WIFdRChp6S6fSHoew5avG6YQ8zQs30bWtUChbRz4MGtKIk/vsJ84Bh1Up6mVsuUBc4Y/Imi+YsgNoLGwJ1zThKryzeIeQIFGnYvnSx5R7JDG1qem/KIk= ceph-77422328-77ce-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T04:06:23.111 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC768K6LansU80C45pnCANkjMbWFb/Md3+UWh+qd4cyJqEAlyyKdmrPnPSAOp7cjZAumDQqMBbZs2HMaDbROUuCIQUU1HA9Dbk7SBYWdOAJID3q1BQE+PjulI/AYddddCzCHBnHY4DkQOWRcMauy6Mx44AKh0d9caae1wskmLYJ8FkIxGkHWO4tE89CF+L5q5KXwVELK8RbQCZza3xSTVQ5yHpIYMBUlKFgDTPwsqDWRf8/P8FJXL3hKyk2S5eITjRHTLUfoB+ybaUwKoLOArCGvHeUe71s7Cr06n1bSkY1anC+E9ZpzJgY+nkyb3KpcgHK2r9MXNSPrroKh540QUDMJw5tFKJju9lldPV/v9YEssmM+1JqDpZ/YkZf7QiYT/1md82b1S7u3WIFdRChp6S6fSHoew5avG6YQ8zQs30bWtUChbRz4MGtKIk/vsJ84Bh1Up6mVsuUBc4Y/Imi+YsgNoLGwJ1zThKryzeIeQIFGnYvnSx5R7JDG1qem/KIk= ceph-77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:06:23.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC768K6LansU80C45pnCANkjMbWFb/Md3+UWh+qd4cyJqEAlyyKdmrPnPSAOp7cjZAumDQqMBbZs2HMaDbROUuCIQUU1HA9Dbk7SBYWdOAJID3q1BQE+PjulI/AYddddCzCHBnHY4DkQOWRcMauy6Mx44AKh0d9caae1wskmLYJ8FkIxGkHWO4tE89CF+L5q5KXwVELK8RbQCZza3xSTVQ5yHpIYMBUlKFgDTPwsqDWRf8/P8FJXL3hKyk2S5eITjRHTLUfoB+ybaUwKoLOArCGvHeUe71s7Cr06n1bSkY1anC+E9ZpzJgY+nkyb3KpcgHK2r9MXNSPrroKh540QUDMJw5tFKJju9lldPV/v9YEssmM+1JqDpZ/YkZf7QiYT/1md82b1S7u3WIFdRChp6S6fSHoew5avG6YQ8zQs30bWtUChbRz4MGtKIk/vsJ84Bh1Up6mVsuUBc4Y/Imi+YsgNoLGwJ1zThKryzeIeQIFGnYvnSx5R7JDG1qem/KIk= ceph-77422328-77ce-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T04:06:23.168 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC768K6LansU80C45pnCANkjMbWFb/Md3+UWh+qd4cyJqEAlyyKdmrPnPSAOp7cjZAumDQqMBbZs2HMaDbROUuCIQUU1HA9Dbk7SBYWdOAJID3q1BQE+PjulI/AYddddCzCHBnHY4DkQOWRcMauy6Mx44AKh0d9caae1wskmLYJ8FkIxGkHWO4tE89CF+L5q5KXwVELK8RbQCZza3xSTVQ5yHpIYMBUlKFgDTPwsqDWRf8/P8FJXL3hKyk2S5eITjRHTLUfoB+ybaUwKoLOArCGvHeUe71s7Cr06n1bSkY1anC+E9ZpzJgY+nkyb3KpcgHK2r9MXNSPrroKh540QUDMJw5tFKJju9lldPV/v9YEssmM+1JqDpZ/YkZf7QiYT/1md82b1S7u3WIFdRChp6S6fSHoew5avG6YQ8zQs30bWtUChbRz4MGtKIk/vsJ84Bh1Up6mVsuUBc4Y/Imi+YsgNoLGwJ1zThKryzeIeQIFGnYvnSx5R7JDG1qem/KIk= ceph-77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:06:23.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T04:06:23.375 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:06:23.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:23 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/306234512' entity='client.admin' 2024-09-21T04:06:26.287 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T04:06:26.287 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T04:06:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:06:26.810 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:26 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2498819002' entity='client.admin' 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:06:29.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:28 smithi110 ceph-mon[27723]: Deploying daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:06:29.374 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-09-21T04:06:29.374 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:06:29.374 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-09-21T04:06:29.393 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:06:29.393 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:06:29.451 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-09-21T04:06:29.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch host add smithi148 2024-09-21T04:06:29.771 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:06:30.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:29 smithi110 ceph-mon[27723]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:30.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:29 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:06:33.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:33 smithi110 ceph-mon[27723]: Deploying daemon crash.smithi110 on smithi110 2024-09-21T04:06:34.522 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:34 smithi110 ceph-mon[27723]: Deploying cephadm binary to smithi148 2024-09-21T04:06:34.522 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:34 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:34.523 INFO:teuthology.orchestra.run.smithi110.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-09-21T04:06:35.227 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T04:06:35.439 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:35.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:35 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:36.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:36 smithi110 ceph-mon[27723]: Added host smithi148 2024-09-21T04:06:36.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:36 smithi110 ceph-mon[27723]: Deploying daemon node-exporter.smithi110 on smithi110 2024-09-21T04:06:37.134 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:06:37.134 INFO:teuthology.orchestra.run.smithi110.stdout:[{"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-09-21T04:06:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:37 smithi110 ceph-mon[27723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:37.862 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T04:06:37.862 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T04:06:38.066 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:06:38.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:38 smithi110 ceph-mon[27723]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:06:39.524 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:39 smithi110 ceph-mon[27723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:39.524 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:39 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:41.625 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:41.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:41 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2812359487' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-21T04:06:42.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:42 smithi110 ceph-mon[27723]: Deploying daemon alertmanager.smithi110 on smithi110 2024-09-21T04:06:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:42 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2812359487' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T04:06:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:42 smithi110 ceph-mon[27723]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T04:06:42.858 INFO:tasks.cephadm:Adding mon.smithi110 on smithi110 2024-09-21T04:06:42.858 INFO:tasks.cephadm:Adding mon.smithi148 on smithi148 2024-09-21T04:06:42.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch apply mon '2;smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148' 2024-09-21T04:06:43.056 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:43.099 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:43 smithi110 ceph-mon[27723]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:44.860 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-09-21T04:06:45.341 DEBUG:teuthology.orchestra.run.smithi148:mon.smithi148> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service 2024-09-21T04:06:45.343 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:06:45.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:06:45.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:45 smithi110 ceph-mon[27723]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:45 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:45 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:46.410 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:46 smithi110 ceph-mon[27723]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:06:46.410 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:46 smithi110 ceph-mon[27723]: Saving service mon spec with placement smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148;count:2 2024-09-21T04:06:47.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:06:47.303 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:06:47.303 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:06:47.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:47 smithi110 ceph-mon[27723]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:48.404 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:48 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/1200251940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:06:48.829 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:06:48.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:06:49.026 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:49.071 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:49.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:49 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:06:50.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:50 smithi110 ceph-mon[27723]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T04:06:50.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:50 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:06:50.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:50 smithi110 ceph-mon[27723]: Deploying daemon grafana.smithi110 on smithi110 2024-09-21T04:06:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:06:50.565 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:06:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:06:51.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:51 smithi110 ceph-mon[27723]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:51.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:51 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3522218897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:06:52.165 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:06:52.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:06:52.363 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:52.408 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:53.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:53 smithi110 ceph-mon[27723]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:54.109 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:06:54.110 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:06:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:06:55.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:55 smithi110 ceph-mon[27723]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:55.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:55 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/1220622296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:06:55.710 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:06:55.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:06:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:57 smithi110 ceph-mon[27723]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:06:57.584 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:06:57.584 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:06:57.585 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:06:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:58 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3213284056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:06:59.171 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:06:59.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:06:59.364 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:59.408 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:06:59.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:06:59 smithi110 ceph-mon[27723]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:01.095 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:01.095 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:01.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:01 smithi110 ceph-mon[27723]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:02.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/1847558956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:02 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:02.709 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:02.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:02.903 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:02.947 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:03.401 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:03 smithi110 ceph-mon[27723]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:03.401 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:03 smithi110 ceph-mon[27723]: Deploying daemon prometheus.smithi110 on smithi110 2024-09-21T04:07:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:04.557 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:04.557 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:05 smithi110 ceph-mon[27723]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:05 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:05 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/508977669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:06.289 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:06.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:06.489 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:06.533 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:07.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:07 smithi110 ceph-mon[27723]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:08.123 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:08.124 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:09 smithi110 ceph-mon[27723]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:09 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2127453922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:09.655 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:09.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:09.852 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:09.896 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:11.262 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:11 smithi110 ceph-mon[27723]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:11.550 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:11.550 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:11.550 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:12.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:12 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/4047921717' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:13.150 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:13.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:13.389 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:13 smithi110 ceph-mon[27723]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:13 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:13 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:13 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:13 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-21T04:07:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:14 smithi110 ceph-mon[27723]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T04:07:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:14 smithi110 ceph-mon[27723]: mgrmap e14: smithi110.axmqzb(active, since 59s) 2024-09-21T04:07:15.090 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:15.090 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:15.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:15 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/431522872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:16.611 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:16.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:16.807 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:16.851 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: mgrmap e15: smithi110.axmqzb(active, starting, since 0.00312811s) 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:07:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:18.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:07:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:18.425 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:18.426 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:19 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:07:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:19 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3849381736' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:19 smithi110 ceph-mon[27723]: mgrmap e16: smithi110.axmqzb(active, since 1.00576s) 2024-09-21T04:07:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:19 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:20.066 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:20.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:20.709 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:20.709 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:21.222 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:21.641 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: [21/Sep/2024:04:07:19] ENGINE Bus STARTING 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: [21/Sep/2024:04:07:19] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: [21/Sep/2024:04:07:19] ENGINE Error in HTTPServer.serve 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:07:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: [21/Sep/2024:04:07:19] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: [21/Sep/2024:04:07:19] ENGINE Bus STARTED 2024-09-21T04:07:21.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:21 smithi110 ceph-mon[27723]: mgrmap e17: smithi110.axmqzb(active, since 2s) 2024-09-21T04:07:23.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:23.453 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:23.453 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:23.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:23.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:23.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:23.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:23.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:07:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2427943640' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:07:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:07:25.302 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:25.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:25.820 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:25.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:25.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:07:25.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:07:25.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:25.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:25 smithi110 ceph-mon[27723]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:07:27.869 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:27.869 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:27.869 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:28.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:27 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2805816058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:29.639 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:29.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:29 smithi110 ceph-mon[27723]: Deploying daemon crash.smithi148 on smithi148 2024-09-21T04:07:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:31.665 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:31.665 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:31.683 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:32.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:31 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/1283958724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:33.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:33.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:33.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:33.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:33.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:32 smithi110 ceph-mon[27723]: Deploying daemon node-exporter.smithi148 on smithi148 2024-09-21T04:07:33.453 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:33.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:33.652 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:36.783 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:36.783 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:07:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:37 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2724464634' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:38.786 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:38.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:39.231 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:39 smithi110 ceph-mon[27723]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:40.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:40 smithi110 ceph-mon[27723]: Deploying daemon mon.smithi148 on smithi148 2024-09-21T04:07:41.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:41.099 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:05:19.513898Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:07:41.100 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-21T04:07:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:41 smithi110 ceph-mon[27723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:41 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/44712780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.197 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:07:43.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:07:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 systemd[1]: Starting Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:07:43.568 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:43 smithi110 ceph-mon[27723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.568 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 podman[34926]: 2024-09-21 04:07:43.578556911 +0000 UTC m=+0.097986239 container create 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=reef-6a1bdf8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 podman[34926]: 2024-09-21 04:07:43.497361838 +0000 UTC m=+0.016791193 image pull 0adab276c97e6f3143ffbca5eeff37fcd7dc3b5c0f339a9f747381d9993602de quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 podman[34926]: 2024-09-21 04:07:43.764182141 +0000 UTC m=+0.283611485 container init 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-6a1bdf8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 podman[34926]: 2024-09-21 04:07:43.767507721 +0000 UTC m=+0.286937049 container start 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-6a1bdf8, CEPH_POINT_RELEASE=) 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable), process ceph-mon, pid 2 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pidfile_write: ignore empty --pid-file 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: load: jerasure load: lrc 2024-09-21T04:07:43.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Git sha 0 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Compile date 2024-09-20 20:34:00 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: DB SUMMARY 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: DB Session ID: Z8GESIQYUNDV4ID5OR0U 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi148/store.db dir, Total Num: 0, files: 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi148/store.db: 000004.log size: 511 ; 2024-09-21T04:07:43.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.env: 0x55c9e4412bc0 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.info_log: 0x55c9e69a8dc0 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:07:43.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.statistics: (nil) 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.use_fsync: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:07:43.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.db_log_dir: 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.wal_dir: 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:07:43.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.write_buffer_manager: 0x55c9e69b6320 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.unordered_write: 0 2024-09-21T04:07:43.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.row_cache: None 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.wal_filter: None 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.wal_compression: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:07:43.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:07:43.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_open_files: -1 2024-09-21T04:07:43.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:07:43.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:07:43.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Compression algorithms supported: 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kZSTD supported: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:07:43.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000005 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.merge_operator: 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_filter: None 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:07:43.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c9e69a8a20) 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_type: 0 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_index_type: 0 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_shortening: 1 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: checksum: 4 2024-09-21T04:07:43.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: no_block_cache: 0 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache: 0x55c9e699d350 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_options: 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: capacity : 536870912 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_shard_bits : 4 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: strict_capacity_limit : 0 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_compressed: (nil) 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: persistent_cache: (nil) 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size: 4096 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size_deviation: 10 2024-09-21T04:07:43.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_restart_interval: 16 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_block_restart_interval: 1 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_block_size: 4096 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: partition_filters: 0 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: use_delta_encoding: 1 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: filter_policy: bloomfilter 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: whole_key_filtering: 1 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: verify_compression: 0 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: format_version: 5 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_index_compression: 1 2024-09-21T04:07:43.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_align: 0 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_auto_readahead_size: 262144 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: prepopulate_block_cache: 0 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression: NoCompression 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.num_levels: 7 2024-09-21T04:07:43.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:07:43.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:07:43.860 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:07:43.861 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:07:43.862 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:07:43.863 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.ttl: 2592000 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:07:43.864 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d8f8c1a-7a78-495f-8fbb-f74794b1d5ac 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891663794580, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891663795009, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726891663, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d8f8c1a-7a78-495f-8fbb-f74794b1d5ac", "db_session_id": "Z8GESIQYUNDV4ID5OR0U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726891663795106, "job": 1, "event": "recovery_finished"} 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T04:07:43.865 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi148/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9e69c8e00 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: DB pointer 0x55c9e6abe000 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** DB Stats ** 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:07:43.866 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T04:07:43.867 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T04:07:43.868 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache BinnedLRUCache@0x55c9e699d350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T04:07:43.869 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148 does not exist in monmap, will attempt to join an existing cluster 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: starting mon.smithi148 rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi148 fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(???) e0 preinit fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).mds e1 new map 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).mds e1 print_map 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e1 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: -1 2024-09-21T04:07:43.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: No filesystems configured 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-21T04:07:43.871 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:07:43.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon crash.smithi110 on smithi110 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying cephadm binary to smithi148 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Added host smithi148 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon node-exporter.smithi110 on smithi110 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.874 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2812359487' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon alertmanager.smithi110 on smithi110 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2812359487' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.875 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Saving service mon spec with placement smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148;count:2 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/1200251940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.876 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T04:07:43.877 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon grafana.smithi110 on smithi110 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3522218897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/1220622296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.878 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3213284056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/1847558956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.879 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon prometheus.smithi110 on smithi110 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/508977669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2127453922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/4047921717' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.880 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14162 172.21.15.110:0/1444412835' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mgrmap e14: smithi110.axmqzb(active, since 59s) 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/431522872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mgrmap e15: smithi110.axmqzb(active, starting, since 0.00312811s) 2024-09-21T04:07:43.881 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:07:43.882 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3849381736' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mgrmap e16: smithi110.axmqzb(active, since 1.00576s) 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: [21/Sep/2024:04:07:19] ENGINE Bus STARTING 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: [21/Sep/2024:04:07:19] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: [21/Sep/2024:04:07:19] ENGINE Error in HTTPServer.serve 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:07:43.883 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-09-21T04:07:43.884 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: [21/Sep/2024:04:07:19] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: [21/Sep/2024:04:07:19] ENGINE Bus STARTED 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mgrmap e17: smithi110.axmqzb(active, since 2s) 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.885 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2427943640' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:07:43.886 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.887 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2805816058' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon crash.smithi148 on smithi148 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/1283958724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon node-exporter.smithi148 on smithi148 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.889 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2724464634' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.890 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: Deploying daemon mon.smithi148 on smithi148 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/44712780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 ceph-mon[34957]: mon.smithi148@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 bash[34926]: 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 2024-09-21T04:07:43.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:43 smithi148 systemd[1]: Started Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:07:43.915 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:07:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:07:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: mon.smithi110 calling monitor election 2024-09-21T04:07:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:07:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: mon.smithi148 calling monitor election 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: monmap e2: 2 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: fsmap 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: mgrmap e17: smithi110.axmqzb(active, since 30s) 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: overall HEALTH_OK 2024-09-21T04:07:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: mon.smithi110 calling monitor election 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: mon.smithi148 calling monitor election 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: monmap e2: 2 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: fsmap 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: mgrmap e17: smithi110.axmqzb(active, since 30s) 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: overall HEALTH_OK 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/2347653176' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:49.334 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:07:49.334 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","modified":"2024-09-21T04:07:43.822032Z","created":"2024-09-21T04:05:19.513898Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi148","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T04:07:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-09-21T04:07:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:49 smithi110 ceph-mon[27723]: mgrmap e18: smithi110.axmqzb(active, since 30s), standbys: smithi148.zjrahi 2024-09-21T04:07:50.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:07:50.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:07:50.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:49 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/559599629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:50.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:50.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:49 smithi148 ceph-mon[34957]: mgrmap e18: smithi110.axmqzb(active, since 30s), standbys: smithi148.zjrahi 2024-09-21T04:07:50.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:07:50.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:07:50.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:49 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/559599629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:07:50.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:07:50.494 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T04:07:50.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T04:07:50.705 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:51.129 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:50 smithi110 ceph-mon[27723]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:50 smithi148 ceph-mon[34957]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:07:52.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:07:52.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:52 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:52.509 INFO:teuthology.orchestra.run.smithi110.stdout:# minimal ceph.conf for 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:07:52.509 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-21T04:07:52.509 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:07:52.509 INFO:teuthology.orchestra.run.smithi110.stdout: mon_host = [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:07:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:07:52.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:52 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:53.115 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T04:07:53.115 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:07:53.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T04:07:53.143 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:07:53.143 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:53.208 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:07:53.208 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T04:07:53.242 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:07:53.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:07:53.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:53 smithi110 ceph-mon[27723]: Reconfiguring mon.smithi110 (unknown last config time)... 2024-09-21T04:07:53.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:53 smithi110 ceph-mon[27723]: Reconfiguring daemon mon.smithi110 on smithi110 2024-09-21T04:07:53.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:53 smithi110 ceph-mon[27723]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:53.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:53 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3780163568' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:53.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:53.311 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T04:07:53.311 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T04:07:53.314 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T04:07:53.314 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T04:07:53.356 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T04:07:53.356 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T04:07:53.399 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T04:07:53.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T04:07:53.442 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T04:07:53.442 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T04:07:53.485 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T04:07:53.486 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T04:07:53.528 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T04:07:53.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T04:07:53.571 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T04:07:53.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T04:07:53.593 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:53 smithi148 ceph-mon[34957]: Reconfiguring mon.smithi110 (unknown last config time)... 2024-09-21T04:07:53.593 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:53 smithi148 ceph-mon[34957]: Reconfiguring daemon mon.smithi110 on smithi110 2024-09-21T04:07:53.593 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:53 smithi148 ceph-mon[34957]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:53.594 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:53 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3780163568' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:53.594 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:53.614 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T04:07:53.614 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T04:07:53.658 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T04:07:53.658 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T04:07:53.702 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T04:07:53.703 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T04:07:53.748 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T04:07:53.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T04:07:53.793 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T04:07:53.793 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T04:07:53.836 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T04:07:53.836 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T04:07:53.879 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T04:07:53.879 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T04:07:53.922 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T04:07:53.922 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T04:07:53.966 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-21T04:07:53.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-21T04:07:54.009 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T04:07:54.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T04:07:54.052 DEBUG:tasks.cephadm:set 18 configs 2024-09-21T04:07:54.052 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi110') 2024-09-21T04:07:54.401 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.402 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.408 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.538 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.538 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:54.872 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: Reconfiguring mgr.smithi110.axmqzb (unknown last config time)... 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:55 smithi110 ceph-mon[27723]: Reconfiguring daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: Reconfiguring mgr.smithi110.axmqzb (unknown last config time)... 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:07:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:55 smithi148 ceph-mon[34957]: Reconfiguring daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:07:57.590 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:57 smithi110 ceph-mon[27723]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:57 smithi148 ceph-mon[34957]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:58.344 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:58.344 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.019 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.019 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.020 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.020 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:07:59.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:07:59 smithi110 ceph-mon[27723]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:07:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:07:59 smithi148 ceph-mon[34957]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:01.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 -- 172.21.15.110:0/3487043877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0ffc30 msgr2=0x7f269c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 --2- 172.21.15.110:0/3487043877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0ffc30 0x7f269c1000b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2690009880 tx=0x7f269002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 -- 172.21.15.110:0/3487043877 shutdown_connections 2024-09-21T04:08:01.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 --2- 172.21.15.110:0/3487043877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0ffc30 0x7f269c1000b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 --2- 172.21.15.110:0/3487043877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f269c0fea30 0x7f269c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.048+0000 7f26a4680640 1 -- 172.21.15.110:0/3487043877 >> 172.21.15.110:0/3487043877 conn(0x7f269c0fa1c0 msgr2=0x7f269c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 -- 172.21.15.110:0/3487043877 shutdown_connections 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 -- 172.21.15.110:0/3487043877 wait complete. 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 Processor -- start 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 -- start start 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f269c0ffc30 0x7f269c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c199e90 con 0x7f269c0fea30 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a4680640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c19a000 con 0x7f269c0ffc30 2024-09-21T04:08:01.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a241c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a241c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36352/0 (socket says 172.21.15.110:36352) 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.049+0000 7f26a241c640 1 -- 172.21.15.110:0/4127435337 learned_addr learned my addr 172.21.15.110:0/4127435337 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a241c640 1 -- 172.21.15.110:0/4127435337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f269c0ffc30 msgr2=0x7f269c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a241c640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f269c0ffc30 0x7f269c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a241c640 1 -- 172.21.15.110:0/4127435337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009560 con 0x7f269c0fea30 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a241c640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f269400b3d0 tx=0x7f269400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2694011800 con 0x7f269c0fea30 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(23 keys) v1 ==== 991+0+0 (secure 0 0 0) 0x7f2694011da0 con 0x7f269c0fea30 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c19a2e0 con 0x7f269c0fea30 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2694010da0 con 0x7f269c0fea30 2024-09-21T04:08:01.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.050+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c19a8b0 con 0x7f269c0fea30 2024-09-21T04:08:01.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.051+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c0feeb0 con 0x7f269c0fea30 2024-09-21T04:08:01.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.051+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f2694011960 con 0x7f269c0fea30 2024-09-21T04:08:01.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.052+0000 7f269b7fe640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 0x7f268406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.052+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f269408ae40 con 0x7f269c0fea30 2024-09-21T04:08:01.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.052+0000 7f26a1c1b640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 0x7f268406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.052+0000 7f26a1c1b640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 0x7f268406f090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2690004ec0 tx=0x7f2690000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.056+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f26940562e0 con 0x7f269c0fea30 2024-09-21T04:08:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.225+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) v1 -- 0x7f269c106a10 con 0x7f269c0fea30 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.229+0000 7f269b7fe640 1 -- 172.21.15.110:0/4127435337 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v12)=0 v12) v1 ==== 125+0+0 (secure 0 0 0) 0x7f26940150c0 con 0x7f269c0fea30 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.231+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 msgr2=0x7f268406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.231+0000 7f26a4680640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 0x7f268406f090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2690004ec0 tx=0x7f2690000bc0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.231+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 msgr2=0x7f269c0686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.231+0000 7f26a4680640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f269400b3d0 tx=0x7f269400b890 comp rx=0 tx=0).stop 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 shutdown_connections 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f269c0ffc30 0x7f269c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f268406cbd0 0x7f268406f090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 --2- 172.21.15.110:0/4127435337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f269c0fea30 0x7f269c0686f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 >> 172.21.15.110:0/4127435337 conn(0x7f269c0fa1c0 msgr2=0x7f269c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 shutdown_connections 2024-09-21T04:08:01.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.232+0000 7f26a4680640 1 -- 172.21.15.110:0/4127435337 wait complete. 2024-09-21T04:08:01.514 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:01 smithi110 ceph-mon[27723]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:01.514 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:01 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/110193960' entity='client.admin' 2024-09-21T04:08:01.514 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:01 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4127435337' entity='client.admin' 2024-09-21T04:08:01.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:01 smithi148 ceph-mon[34957]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:01.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:01 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/110193960' entity='client.admin' 2024-09-21T04:08:01.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:01 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4127435337' entity='client.admin' 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.375+0000 7f12357fa640 1 -- 172.21.15.110:0/110193960 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v11)=0 v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7f122c08a400 con 0x7f12380ffca0 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f122006cbd0 msgr2=0x7f122006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 --2- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f122006cbd0 0x7f122006f090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1228006010 tx=0x7f1228004a80 comp rx=0 tx=0).stop 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12380ffca0 msgr2=0x7f1238195020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 --2- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12380ffca0 0x7f1238195020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b510 tx=0x7f122c00b9d0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 shutdown_connections 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 --2- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f122006cbd0 0x7f122006f090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 --2- 172.21.15.110:0/110193960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12380ffca0 0x7f1238195020 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 --2- 172.21.15.110:0/110193960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12380feaa0 0x7f1238194ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 >> 172.21.15.110:0/110193960 conn(0x7f12380fa230 msgr2=0x7f1238068400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 shutdown_connections 2024-09-21T04:08:02.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:00.377+0000 7f123e75e640 1 -- 172.21.15.110:0/110193960 wait complete. 2024-09-21T04:08:03.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:02 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1588666088' entity='client.admin' 2024-09-21T04:08:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:02 smithi110 ceph-mon[27723]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:02 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1588666088' entity='client.admin' 2024-09-21T04:08:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:02 smithi148 ceph-mon[34957]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- 172.21.15.110:0/2194860451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80fea30 msgr2=0x7f9de80fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 --2- 172.21.15.110:0/2194860451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80fea30 0x7f9de80fee30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009880 tx=0x7f9de402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- 172.21.15.110:0/2194860451 shutdown_connections 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 --2- 172.21.15.110:0/2194860451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80ffc30 0x7f9de81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 --2- 172.21.15.110:0/2194860451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80fea30 0x7f9de80fee30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- 172.21.15.110:0/2194860451 >> 172.21.15.110:0/2194860451 conn(0x7f9de80fa1c0 msgr2=0x7f9de80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- 172.21.15.110:0/2194860451 shutdown_connections 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- 172.21.15.110:0/2194860451 wait complete. 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 Processor -- start 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 -- start start 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.788+0000 7f9defee6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80fea30 0x7f9de8194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9defee6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:04.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9defee6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de81956b0 con 0x7f9de80ffc30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9defee6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8195820 con 0x7f9de80fea30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36372/0 (socket says 172.21.15.110:36372) 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 -- 172.21.15.110:0/1588666088 learned_addr learned my addr 172.21.15.110:0/1588666088 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9dedc82640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80fea30 0x7f9de8194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 -- 172.21.15.110:0/1588666088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80fea30 msgr2=0x7f9de8194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80fea30 0x7f9de8194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 -- 172.21.15.110:0/1588666088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009560 con 0x7f9de80ffc30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ded481640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800b450 tx=0x7f9dd800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd8011860 con 0x7f9de80ffc30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(23 keys) v1 ==== 991+0+0 (secure 0 0 0) 0x7f9dd8011e00 con 0x7f9de80ffc30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de819a2c0 con 0x7f9de80ffc30 2024-09-21T04:08:04.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd8010de0 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.789+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de819a890 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.790+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de80feeb0 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.791+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f9dd80119c0 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.791+0000 7f9ddeffd640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 0x7f9dd406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.791+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9dd808afd0 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.791+0000 7f9dedc82640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 0x7f9dd406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.792+0000 7f9dedc82640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 0x7f9dd406f090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9de4000c00 tx=0x7f9de4004e60 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.796+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9dd8056480 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.963+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f9de81069f0 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.967+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(24 keys) v1 ==== 1022+0+0 (secure 0 0 0) 0x7f9dd805ee90 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.967+0000 7f9ddeffd640 1 -- 172.21.15.110:0/1588666088 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9dd808a790 con 0x7f9de80ffc30 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 msgr2=0x7f9dd406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 0x7f9dd406f090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9de4000c00 tx=0x7f9de4004e60 comp rx=0 tx=0).stop 2024-09-21T04:08:04.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 msgr2=0x7f9de81950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800b450 tx=0x7f9dd800b910 comp rx=0 tx=0).stop 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 shutdown_connections 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9dd406cbd0 0x7f9dd406f090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de80ffc30 0x7f9de81950e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 --2- 172.21.15.110:0/1588666088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de80fea30 0x7f9de8194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 >> 172.21.15.110:0/1588666088 conn(0x7f9de80fa1c0 msgr2=0x7f9de8067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 shutdown_connections 2024-09-21T04:08:04.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:01.969+0000 7f9defee6640 1 -- 172.21.15.110:0/1588666088 wait complete. 2024-09-21T04:08:04.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:05.135 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:04 smithi110 ceph-mon[27723]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:05.319 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:04 smithi148 ceph-mon[34957]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:06.521 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:06.521 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:07.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:07 smithi110 ceph-mon[27723]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:07 smithi148 ceph-mon[34957]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:09 smithi110 ceph-mon[27723]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:09 smithi148 ceph-mon[34957]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:11.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 -- 172.21.15.110:0/4289281734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c100690 msgr2=0x7fa01c102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 --2- 172.21.15.110:0/4289281734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c100690 0x7fa01c102a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009880 tx=0x7fa00c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 -- 172.21.15.110:0/4289281734 shutdown_connections 2024-09-21T04:08:11.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 --2- 172.21.15.110:0/4289281734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c102fc0 0x7fa01c1053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 --2- 172.21.15.110:0/4289281734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c100690 0x7fa01c102a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.221+0000 7fa020f1a640 1 -- 172.21.15.110:0/4289281734 >> 172.21.15.110:0/4289281734 conn(0x7fa01c0fa230 msgr2=0x7fa01c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:11.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.222+0000 7fa020f1a640 1 -- 172.21.15.110:0/4289281734 shutdown_connections 2024-09-21T04:08:11.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.222+0000 7fa020f1a640 1 -- 172.21.15.110:0/4289281734 wait complete. 2024-09-21T04:08:11.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.222+0000 7fa020f1a640 1 Processor -- start 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa020f1a640 1 -- start start 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa020f1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c100690 0x7fa01c0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa020f1a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa020f1a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0feb80 con 0x7fa01c102fc0 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa020f1a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0fecf0 con 0x7fa01c100690 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60542/0 (socket says 172.21.15.110:60542) 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 -- 172.21.15.110:0/1153363962 learned_addr learned my addr 172.21.15.110:0/1153363962 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa01a59c640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c100690 0x7fa01c0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 -- 172.21.15.110:0/1153363962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c100690 msgr2=0x7fa01c0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c100690 0x7fa01c0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 -- 172.21.15.110:0/1153363962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c009560 con 0x7fa01c102fc0 2024-09-21T04:08:11.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.223+0000 7fa013fff640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d6f0 tx=0x7fa00400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.224+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa004009a30 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.224+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(24 keys) v1 ==== 1022+0+0 (secure 0 0 0) 0x7fa004010460 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.224+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00400f5f0 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.224+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c0fefd0 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.224+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c19d3d0 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.225+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c100a70 con 0x7fa01c102fc0 2024-09-21T04:08:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.226+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fa004009b90 con 0x7fa01c102fc0 2024-09-21T04:08:11.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.226+0000 7fa0137fe640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 0x7fa00806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.226+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa00408ad80 con 0x7fa01c102fc0 2024-09-21T04:08:11.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.226+0000 7fa01a59c640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 0x7fa00806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.227+0000 7fa01a59c640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 0x7fa00806f090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa00c004ef0 tx=0x7fa00c005df0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:11.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.231+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa0040562b0 con 0x7fa01c102fc0 2024-09-21T04:08:11.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.399+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fa01c108430 con 0x7fa01c102fc0 2024-09-21T04:08:11.402 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:11 smithi110 ceph-mon[27723]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:11.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.403+0000 7fa0137fe640 1 -- 172.21.15.110:0/1153363962 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v14)=0 v14) v1 ==== 117+0+0 (secure 0 0 0) 0x7fa00408a540 con 0x7fa01c102fc0 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 msgr2=0x7fa00806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 0x7fa00806f090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa00c004ef0 tx=0x7fa00c005df0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 msgr2=0x7fa01c0fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d6f0 tx=0x7fa00400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 shutdown_connections 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa00806cbd0 0x7fa00806f090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c102fc0 0x7fa01c0fe640 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 --2- 172.21.15.110:0/1153363962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c100690 0x7fa01c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 >> 172.21.15.110:0/1153363962 conn(0x7fa01c0fa230 msgr2=0x7fa01c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 shutdown_connections 2024-09-21T04:08:11.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7fa020f1a640 1 -- 172.21.15.110:0/1153363962 wait complete. 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 -- 172.21.15.110:0/193903858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0ffc30 msgr2=0x7f54fc1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 --2- 172.21.15.110:0/193903858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1000b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009880 tx=0x7f54f002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 -- 172.21.15.110:0/193903858 shutdown_connections 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 --2- 172.21.15.110:0/193903858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1000b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 --2- 172.21.15.110:0/193903858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0fea30 0x7f54fc0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 -- 172.21.15.110:0/193903858 >> 172.21.15.110:0/193903858 conn(0x7f54fc0fa1c0 msgr2=0x7f54fc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 -- 172.21.15.110:0/193903858 shutdown_connections 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.405+0000 7f5500ee9640 1 -- 172.21.15.110:0/193903858 wait complete. 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 Processor -- start 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 -- start start 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc195ea0 con 0x7f54fc0fea30 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f5500ee9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc196010 con 0x7f54fc0ffc30 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60548/0 (socket says 172.21.15.110:60548) 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f54fa59c640 1 -- 172.21.15.110:0/737327032 learned_addr learned my addr 172.21.15.110:0/737327032 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.406+0000 7f54f9d9b640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54fa59c640 1 -- 172.21.15.110:0/737327032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0ffc30 msgr2=0x7f54fc1958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54fa59c640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54fa59c640 1 -- 172.21.15.110:0/737327032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0009560 con 0x7f54fc0fea30 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54fa59c640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d900 tx=0x7f54ec00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54ec009c00 con 0x7f54fc0fea30 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc19aab0 con 0x7f54fc0fea30 2024-09-21T04:08:11.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(24 keys) v1 ==== 1022+0+0 (secure 0 0 0) 0x7f54ec009d60 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54ec00f5f0 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.407+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc19afd0 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.408+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc0fee30 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.409+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f54ec010460 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.409+0000 7f54eb7fe640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 0x7f54e406f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.409+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f54ec08b3e0 con 0x7f54fc0fea30 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.409+0000 7f54f9d9b640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 0x7f54e406f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.410+0000 7f54f9d9b640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 0x7f54e406f0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f0005bd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:11.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.414+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f54ec056890 con 0x7f54fc0fea30 2024-09-21T04:08:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.579+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f54fc19b280 con 0x7f54fc0fea30 2024-09-21T04:08:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.583+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(25 keys) v1 ==== 1053+0+0 (secure 0 0 0) 0x7f54ec056890 con 0x7f54fc0fea30 2024-09-21T04:08:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.583+0000 7f54eb7fe640 1 -- 172.21.15.110:0/737327032 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) v1 ==== 123+0+0 (secure 0 0 0) 0x7f54ec08aba0 con 0x7f54fc0fea30 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 msgr2=0x7f54e406f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 0x7f54e406f0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f0005bd0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 msgr2=0x7f54fc195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d900 tx=0x7f54ec00ddc0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 shutdown_connections 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54fc0ffc30 0x7f54fc1958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f54e406cbe0 0x7f54e406f0a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 --2- 172.21.15.110:0/737327032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54fc0fea30 0x7f54fc195390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 >> 172.21.15.110:0/737327032 conn(0x7f54fc0fa1c0 msgr2=0x7f54fc067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 shutdown_connections 2024-09-21T04:08:11.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.585+0000 7f5500ee9640 1 -- 172.21.15.110:0/737327032 wait complete. 2024-09-21T04:08:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:11 smithi148 ceph-mon[34957]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:12.208 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:12.209 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:12.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:12 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1153363962' entity='client.admin' 2024-09-21T04:08:12.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:12 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/737327032' entity='client.admin' 2024-09-21T04:08:12.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:12 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1006177586' entity='client.admin' 2024-09-21T04:08:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:12 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1153363962' entity='client.admin' 2024-09-21T04:08:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:12 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/737327032' entity='client.admin' 2024-09-21T04:08:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:12 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1006177586' entity='client.admin' 2024-09-21T04:08:13.574 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:13 smithi110 ceph-mon[27723]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:13 smithi148 ceph-mon[34957]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:14.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 -- 172.21.15.110:0/710125910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80fea80 msgr2=0x7f30f80fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:14.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 --2- 172.21.15.110:0/710125910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80fea80 0x7f30f80fee80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009c20 tx=0x7f30e802a970 comp rx=0 tx=0).stop 2024-09-21T04:08:14.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 -- 172.21.15.110:0/710125910 shutdown_connections 2024-09-21T04:08:14.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 --2- 172.21.15.110:0/710125910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80ffc80 0x7f30f8100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 --2- 172.21.15.110:0/710125910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80fea80 0x7f30f80fee80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 -- 172.21.15.110:0/710125910 >> 172.21.15.110:0/710125910 conn(0x7f30f80fa230 msgr2=0x7f30f80fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 -- 172.21.15.110:0/710125910 shutdown_connections 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.874+0000 7f30fcf54640 1 -- 172.21.15.110:0/710125910 wait complete. 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 Processor -- start 2024-09-21T04:08:14.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 -- start start 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80fea80 0x7f30f8068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819a8a0 con 0x7f30f80ffc80 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30fcf54640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819a9e0 con 0x7f30f80fea80 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30f5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:14.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30f5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60566/0 (socket says 172.21.15.110:60566) 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30f5d9b640 1 -- 172.21.15.110:0/1006177586 learned_addr learned my addr 172.21.15.110:0/1006177586 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.875+0000 7f30f659c640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80fea80 0x7f30f8068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30f5d9b640 1 -- 172.21.15.110:0/1006177586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80fea80 msgr2=0x7f30f8068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30f5d9b640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80fea80 0x7f30f8068f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30f5d9b640 1 -- 172.21.15.110:0/1006177586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e8009900 con 0x7f30f80ffc80 2024-09-21T04:08:14.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30f5d9b640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30e000b420 tx=0x7f30e000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:14.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30e0011800 con 0x7f30f80ffc80 2024-09-21T04:08:14.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f819ac90 con 0x7f30f80ffc80 2024-09-21T04:08:14.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(25 keys) v1 ==== 1053+0+0 (secure 0 0 0) 0x7f30e0011de0 con 0x7f30f80ffc80 2024-09-21T04:08:14.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30e000f2e0 con 0x7f30f80ffc80 2024-09-21T04:08:14.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.876+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f819b260 con 0x7f30f80ffc80 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.877+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f81074e0 con 0x7f30f80ffc80 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.878+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f30e0010380 con 0x7f30f80ffc80 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.878+0000 7f30ef7fe640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 0x7f30e406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.878+0000 7f30f659c640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 0x7f30e406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.878+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f30e008a9d0 con 0x7f30f80ffc80 2024-09-21T04:08:14.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.879+0000 7f30f659c640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 0x7f30e406f090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000f80 tx=0x7f30e803d040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:11.882+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f30e0055e80 con 0x7f30f80ffc80 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.057+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f30f81076c0 con 0x7f30f80ffc80 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.061+0000 7f30ef7fe640 1 -- 172.21.15.110:0/1006177586 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) v1 ==== 108+0+0 (secure 0 0 0) 0x7f30e008a190 con 0x7f30f80ffc80 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 msgr2=0x7f30e406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 0x7f30e406f090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000f80 tx=0x7f30e803d040 comp rx=0 tx=0).stop 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 msgr2=0x7f30f819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:14.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30e000b420 tx=0x7f30e000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 shutdown_connections 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f30e406cbd0 0x7f30e406f090 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30f80ffc80 0x7f30f819a2d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 --2- 172.21.15.110:0/1006177586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f80fea80 0x7f30f8068f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 >> 172.21.15.110:0/1006177586 conn(0x7f30f80fa230 msgr2=0x7f30f8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:14.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 shutdown_connections 2024-09-21T04:08:14.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:12.063+0000 7f30fcf54640 1 -- 172.21.15.110:0/1006177586 wait complete. 2024-09-21T04:08:15.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 -- 172.21.15.110:0/2594694143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980ffc30 msgr2=0x7f42981000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:15.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 --2- 172.21.15.110:0/2594694143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980ffc30 0x7f42981000b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4288009880 tx=0x7f428802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 -- 172.21.15.110:0/2594694143 shutdown_connections 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 --2- 172.21.15.110:0/2594694143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980ffc30 0x7f42981000b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 --2- 172.21.15.110:0/2594694143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980fea30 0x7f42980fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 -- 172.21.15.110:0/2594694143 >> 172.21.15.110:0/2594694143 conn(0x7f42980fa1c0 msgr2=0x7f42980fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 -- 172.21.15.110:0/2594694143 shutdown_connections 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.315+0000 7f429f5bb640 1 -- 172.21.15.110:0/2594694143 wait complete. 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 Processor -- start 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 -- start start 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 0x7f42981959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298195fa0 con 0x7f42980fea30 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429f5bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298196110 con 0x7f42980ffc30 2024-09-21T04:08:15.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57482/0 (socket says 172.21.15.110:57482) 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 -- 172.21.15.110:0/3403763274 learned_addr learned my addr 172.21.15.110:0/3403763274 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429cb56640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 0x7f42981959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 -- 172.21.15.110:0/3403763274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 msgr2=0x7f42981959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 0x7f42981959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 -- 172.21.15.110:0/3403763274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009560 con 0x7f42980fea30 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429cb56640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 0x7f42981959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.316+0000 7f429d357640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f429400d900 tx=0x7f429400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.317+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4294009c00 con 0x7f42980fea30 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.317+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(25 keys) v1 ==== 1053+0+0 (secure 0 0 0) 0x7f4294009d60 con 0x7f42980fea30 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.317+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429819abb0 con 0x7f42980fea30 2024-09-21T04:08:15.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.317+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f429400f5f0 con 0x7f42980fea30 2024-09-21T04:08:15.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.317+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429819b100 con 0x7f42980fea30 2024-09-21T04:08:15.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.318+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4298107430 con 0x7f42980fea30 2024-09-21T04:08:15.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.318+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f429400f750 con 0x7f42980fea30 2024-09-21T04:08:15.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.319+0000 7f428e7fc640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 0x7f428406f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:15.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.319+0000 7f429cb56640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 0x7f428406f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:15.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.319+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f429408af30 con 0x7f42980fea30 2024-09-21T04:08:15.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.319+0000 7f429cb56640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 0x7f428406f0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4288006010 tx=0x7f4288000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:15.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.323+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f42940564d0 con 0x7f42980fea30 2024-09-21T04:08:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.498+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f429819b430 con 0x7f42980fea30 2024-09-21T04:08:15.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.502+0000 7f428e7fc640 1 -- 172.21.15.110:0/3403763274 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7f429819b430 con 0x7f42980fea30 2024-09-21T04:08:15.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 msgr2=0x7f428406f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:15.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 0x7f428406f0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4288006010 tx=0x7f4288000bc0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 msgr2=0x7f4298195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f429400d900 tx=0x7f429400ddc0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 shutdown_connections 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42980ffc30 0x7f42981959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cbe0 0x7f428406f0a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 --2- 172.21.15.110:0/3403763274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42980fea30 0x7f4298195490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:15.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 >> 172.21.15.110:0/3403763274 conn(0x7f42980fa1c0 msgr2=0x7f4298067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:15.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 shutdown_connections 2024-09-21T04:08:15.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:15.504+0000 7f429f5bb640 1 -- 172.21.15.110:0/3403763274 wait complete. 2024-09-21T04:08:15.661 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:15 smithi110 ceph-mon[27723]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:15 smithi148 ceph-mon[34957]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- 172.21.15.110:0/3413738111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fe430 msgr2=0x7f95200fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 --2- 172.21.15.110:0/3413738111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fe430 0x7f95200fe830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f951c009880 tx=0x7f951c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- 172.21.15.110:0/3413738111 shutdown_connections 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 --2- 172.21.15.110:0/3413738111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fed70 0x7f95200ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 --2- 172.21.15.110:0/3413738111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fe430 0x7f95200fe830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- 172.21.15.110:0/3413738111 >> 172.21.15.110:0/3413738111 conn(0x7f95200fa1c0 msgr2=0x7f95200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- 172.21.15.110:0/3413738111 shutdown_connections 2024-09-21T04:08:16.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- 172.21.15.110:0/3413738111 wait complete. 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 Processor -- start 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- start start 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fe430 0x7f9520197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9520198130 con 0x7f95200fed70 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.204+0000 7f95279cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201982a0 con 0x7f95200fe430 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f952576b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fe430 0x7f9520197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57496/0 (socket says 172.21.15.110:57496) 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 -- 172.21.15.110:0/2690582781 learned_addr learned my addr 172.21.15.110:0/2690582781 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 -- 172.21.15.110:0/2690582781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fe430 msgr2=0x7f9520197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fe430 0x7f9520197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 -- 172.21.15.110:0/2690582781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009560 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f9524f6a640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f951000d720 tx=0x7f951000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95100051f0 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952019cd40 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(25 keys) v1 ==== 1053+0+0 (secure 0 0 0) 0x7f9510009820 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9510004d00 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.205+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952019d310 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.206+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952010a710 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.207+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f9510009990 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.207+0000 7f95167fc640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 0x7f950c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.207+0000 7f952576b640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 0x7f950c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.207+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f951008abe0 con 0x7f95200fed70 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.208+0000 7f952576b640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 0x7f950c06f090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c0058d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.211+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9510056140 con 0x7f95200fed70 2024-09-21T04:08:16.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.385+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) v1 -- 0x7f952010a8a0 con 0x7f95200fed70 2024-09-21T04:08:16.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.389+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f9510056140 con 0x7f95200fed70 2024-09-21T04:08:16.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.390+0000 7f95167fc640 1 -- 172.21.15.110:0/2690582781 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v18)=0 v18) v1 ==== 123+0+0 (secure 0 0 0) 0x7f95100170c0 con 0x7f95200fed70 2024-09-21T04:08:16.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.391+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 msgr2=0x7f950c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.391+0000 7f95279cf640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 0x7f950c06f090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.391+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 msgr2=0x7f9520197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.391+0000 7f95279cf640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f951000d720 tx=0x7f951000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 shutdown_connections 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f950c06cbd0 0x7f950c06f090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95200fed70 0x7f9520197b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 --2- 172.21.15.110:0/2690582781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95200fe430 0x7f9520197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 >> 172.21.15.110:0/2690582781 conn(0x7f95200fa1c0 msgr2=0x7f9520106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 shutdown_connections 2024-09-21T04:08:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:16.392+0000 7f95279cf640 1 -- 172.21.15.110:0/2690582781 wait complete. 2024-09-21T04:08:16.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:16 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3403763274' entity='client.admin' 2024-09-21T04:08:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:16 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2690582781' entity='client.admin' 2024-09-21T04:08:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:16 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3403763274' entity='client.admin' 2024-09-21T04:08:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:16 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2690582781' entity='client.admin' 2024-09-21T04:08:17.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- 172.21.15.110:0/2420355155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40ffca0 msgr2=0x7f16c4100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:17.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/2420355155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40ffca0 0x7f16c4100120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009880 tx=0x7f16c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- 172.21.15.110:0/2420355155 shutdown_connections 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/2420355155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40ffca0 0x7f16c4100120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/2420355155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40feaa0 0x7f16c40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- 172.21.15.110:0/2420355155 >> 172.21.15.110:0/2420355155 conn(0x7f16c40fa230 msgr2=0x7f16c40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- 172.21.15.110:0/2420355155 shutdown_connections 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- 172.21.15.110:0/2420355155 wait complete. 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 Processor -- start 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.597+0000 7f16cbcb4640 1 -- start start 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40ffca0 0x7f16c4194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4194f30 con 0x7f16c40feaa0 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41950a0 con 0x7f16c40ffca0 2024-09-21T04:08:17.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57518/0 (socket says 172.21.15.110:57518) 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 -- 172.21.15.110:0/304320082 learned_addr learned my addr 172.21.15.110:0/304320082 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c924f640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40ffca0 0x7f16c4194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 -- 172.21.15.110:0/304320082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40ffca0 msgr2=0x7f16c4194960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40ffca0 0x7f16c4194960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 -- 172.21.15.110:0/304320082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009560 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16c9a50640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009880 tx=0x7f16b8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8012070 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f16b8007d90 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c4068460 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8018ea0 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.598+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c40689b0 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.599+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c40feea0 con 0x7f16c40feaa0 2024-09-21T04:08:17.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.600+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f16b8016070 con 0x7f16c40feaa0 2024-09-21T04:08:17.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.600+0000 7f16b6ffd640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 0x7f16b006f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:17.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.600+0000 7f16c924f640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 0x7f16b006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:17.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.600+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f16b808c4b0 con 0x7f16c40feaa0 2024-09-21T04:08:17.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.601+0000 7f16c924f640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 0x7f16b006f090 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f16c002a9c0 tx=0x7f16c0000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:17.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.604+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f16b8057a10 con 0x7f16c40feaa0 2024-09-21T04:08:17.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:17 smithi110 ceph-mon[27723]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:17.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.774+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f16c41075e0 con 0x7f16c40feaa0 2024-09-21T04:08:17.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.778+0000 7f16b6ffd640 1 -- 172.21.15.110:0/304320082 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) v1 ==== 106+0+0 (secure 0 0 0) 0x7f16b808bc70 con 0x7f16c40feaa0 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.780+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 msgr2=0x7f16b006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.780+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 0x7f16b006f090 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f16c002a9c0 tx=0x7f16c0000bc0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.780+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 msgr2=0x7f16c4194420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.780+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009880 tx=0x7f16b8009d40 comp rx=0 tx=0).stop 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 shutdown_connections 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c40ffca0 0x7f16c4194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b006cbd0 0x7f16b006f090 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 --2- 172.21.15.110:0/304320082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c40feaa0 0x7f16c4194420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 >> 172.21.15.110:0/304320082 conn(0x7f16c40fa230 msgr2=0x7f16c41032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:17.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 shutdown_connections 2024-09-21T04:08:17.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:17.781+0000 7f16cbcb4640 1 -- 172.21.15.110:0/304320082 wait complete. 2024-09-21T04:08:17.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:17 smithi148 ceph-mon[34957]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:18.895 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:18 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/304320082' entity='client.admin' 2024-09-21T04:08:18.895 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:18 smithi110 ceph-mon[27723]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:18.895 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:19.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:18 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/304320082' entity='client.admin' 2024-09-21T04:08:19.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:18 smithi148 ceph-mon[34957]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:19.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:19.974 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi110') 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: Reconfiguring ceph-exporter.smithi110 (monmap changed)... 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: Reconfiguring daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:08:20.607 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:20 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1047358656' entity='client.admin' 2024-09-21T04:08:20.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:20.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:20.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: Reconfiguring ceph-exporter.smithi110 (monmap changed)... 2024-09-21T04:08:20.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:08:20.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:20.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: Reconfiguring daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:08:20.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:20 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1047358656' entity='client.admin' 2024-09-21T04:08:20.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.983+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3963747938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 msgr2=0x7f7df40ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:20.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.983+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3963747938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df40ff1d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009880 tx=0x7f7de402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3963747938 shutdown_connections 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3963747938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df40ff1d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3963747938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 0x7f7df4100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3963747938 >> 172.21.15.110:0/3963747938 conn(0x7f7df40f99b0 msgr2=0x7f7df40fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3963747938 shutdown_connections 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3963747938 wait complete. 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 Processor -- start 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- start start 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:20.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 0x7f7df4101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df4103030 con 0x7f7df40fed50 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfc2ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41016b0 con 0x7f7df4100010 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfa048640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfa048640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57558/0 (socket says 172.21.15.110:57558) 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.984+0000 7f7dfa048640 1 -- 172.21.15.110:0/3782699874 learned_addr learned my addr 172.21.15.110:0/3782699874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7df9847640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 0x7f7df4101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfa048640 1 -- 172.21.15.110:0/3782699874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 msgr2=0x7f7df4101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfa048640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 0x7f7df4101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfa048640 1 -- 172.21.15.110:0/3782699874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4009560 con 0x7f7df40fed50 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfa048640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c8f0 tx=0x7f7df000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df00041f0 con 0x7f7df40fed50 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f7df0007df0 con 0x7f7df40fed50 2024-09-21T04:08:20.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df4101990 con 0x7f7df40fed50 2024-09-21T04:08:20.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df4101ee0 con 0x7f7df40fed50 2024-09-21T04:08:20.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.985+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df000f450 con 0x7f7df40fed50 2024-09-21T04:08:20.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.986+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df4107590 con 0x7f7df40fed50 2024-09-21T04:08:20.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.986+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f7df0007950 con 0x7f7df40fed50 2024-09-21T04:08:20.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.987+0000 7f7deb7fe640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 0x7f7de006f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:20.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.987+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7df0013070 con 0x7f7df40fed50 2024-09-21T04:08:20.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.987+0000 7f7df9847640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 0x7f7de006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:20.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.987+0000 7f7df9847640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 0x7f7de006f090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7df4102870 tx=0x7f7de4005b50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:20.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.991+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7df0056630 con 0x7f7df40fed50 2024-09-21T04:08:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.154+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f7df4107790 con 0x7f7df40fed50 2024-09-21T04:08:21.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.157+0000 7f7deb7fe640 1 -- 172.21.15.110:0/3782699874 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7df008a940 con 0x7f7df40fed50 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 msgr2=0x7f7de006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 0x7f7de006f090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7df4102870 tx=0x7f7de4005b50 comp rx=0 tx=0).stop 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 msgr2=0x7f7df4102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c8f0 tx=0x7f7df000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 shutdown_connections 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df4100010 0x7f7df4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f7de006cbd0 0x7f7de006f090 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 --2- 172.21.15.110:0/3782699874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df40fed50 0x7f7df4102af0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 >> 172.21.15.110:0/3782699874 conn(0x7f7df40f99b0 msgr2=0x7f7df40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 shutdown_connections 2024-09-21T04:08:21.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.160+0000 7f7dfc2ac640 1 -- 172.21.15.110:0/3782699874 wait complete. 2024-09-21T04:08:21.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:21 smithi110 ceph-mon[27723]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:21 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3782699874' entity='client.admin' 2024-09-21T04:08:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 -- 172.21.15.110:0/1182651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740fea30 msgr2=0x7fd7740fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/1182651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740fea30 0x7fd7740fee30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009c20 tx=0x7fd76c02a970 comp rx=0 tx=0).stop 2024-09-21T04:08:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 -- 172.21.15.110:0/1182651638 shutdown_connections 2024-09-21T04:08:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/1182651638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740ffc30 0x7fd7741000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/1182651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740fea30 0x7fd7740fee30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 -- 172.21.15.110:0/1182651638 >> 172.21.15.110:0/1182651638 conn(0x7fd7740fa1c0 msgr2=0x7fd7740fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 -- 172.21.15.110:0/1182651638 shutdown_connections 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.444+0000 7fd77bd2e640 1 -- 172.21.15.110:0/1182651638 wait complete. 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 Processor -- start 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 -- start start 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740fea30 0x7fd774068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77419a9b0 con 0x7fd7740ffc30 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.445+0000 7fd77bd2e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77419aaf0 con 0x7fd7740fea30 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57584/0 (socket says 172.21.15.110:57584) 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 -- 172.21.15.110:0/900698850 learned_addr learned my addr 172.21.15.110:0/900698850 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd779aca640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740fea30 0x7fd774068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 -- 172.21.15.110:0/900698850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740fea30 msgr2=0x7fd774068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740fea30 0x7fd774068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 -- 172.21.15.110:0/900698850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c009900 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd7792c9640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b420 tx=0x7fd76400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd764011800 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7fd764011e00 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77419ada0 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd76400f2e0 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.446+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77419b340 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.447+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740feeb0 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.447+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fd764010380 con 0x7fd7740ffc30 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.448+0000 7fd76affd640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 0x7fd76006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.448+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd76408ab00 con 0x7fd7740ffc30 2024-09-21T04:08:21.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.448+0000 7fd779aca640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 0x7fd76006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:21.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.448+0000 7fd779aca640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 0x7fd76006f0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c009860 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:21.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.452+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd764055fb0 con 0x7fd7740ffc30 2024-09-21T04:08:21.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.625+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fd77419b5c0 con 0x7fd7740ffc30 2024-09-21T04:08:21.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.629+0000 7fd76affd640 1 -- 172.21.15.110:0/900698850 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd76408a2c0 con 0x7fd7740ffc30 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 msgr2=0x7fd76006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 0x7fd76006f0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c009860 comp rx=0 tx=0).stop 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 msgr2=0x7fd77419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b420 tx=0x7fd76400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 shutdown_connections 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd76006cbe0 0x7fd76006f0a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7740ffc30 0x7fd77419a3e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 --2- 172.21.15.110:0/900698850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7740fea30 0x7fd774068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 >> 172.21.15.110:0/900698850 conn(0x7fd7740fa1c0 msgr2=0x7fd774102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 shutdown_connections 2024-09-21T04:08:21.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:21.631+0000 7fd77bd2e640 1 -- 172.21.15.110:0/900698850 wait complete. 2024-09-21T04:08:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:21 smithi148 ceph-mon[34957]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:21 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3782699874' entity='client.admin' 2024-09-21T04:08:22.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:22 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/900698850' entity='client.admin' 2024-09-21T04:08:22.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:22 smithi110 ceph-mon[27723]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:22.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:22 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/900698850' entity='client.admin' 2024-09-21T04:08:22.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:22 smithi148 ceph-mon[34957]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:24.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 -- 172.21.15.110:0/3379765735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fe440 msgr2=0x7f47dc0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3379765735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fe440 0x7f47dc0fe840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 -- 172.21.15.110:0/3379765735 shutdown_connections 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3379765735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fed80 0x7f47dc0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3379765735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fe440 0x7f47dc0fe840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 -- 172.21.15.110:0/3379765735 >> 172.21.15.110:0/3379765735 conn(0x7f47dc0fa1f0 msgr2=0x7f47dc0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 -- 172.21.15.110:0/3379765735 shutdown_connections 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.423+0000 7f47e1d29640 1 -- 172.21.15.110:0/3379765735 wait complete. 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 Processor -- start 2024-09-21T04:08:24.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 -- start start 2024-09-21T04:08:24.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fe440 0x7f47dc197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:24.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc198210 con 0x7f47dc0fed80 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47e1d29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc198380 con 0x7f47dc0fe440 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47daffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42628/0 (socket says 172.21.15.110:42628) 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47daffd640 1 -- 172.21.15.110:0/3922915218 learned_addr learned my addr 172.21.15.110:0/3922915218 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.424+0000 7f47db7fe640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fe440 0x7f47dc197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47daffd640 1 -- 172.21.15.110:0/3922915218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fe440 msgr2=0x7f47dc197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47daffd640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fe440 0x7f47dc197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47daffd640 1 -- 172.21.15.110:0/3922915218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009560 con 0x7f47dc0fed80 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47daffd640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d720 tx=0x7f47d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d00051f0 con 0x7f47dc0fed80 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f47d0009820 con 0x7f47dc0fed80 2024-09-21T04:08:24.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc19ce20 con 0x7f47dc0fed80 2024-09-21T04:08:24.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d0004e10 con 0x7f47dc0fed80 2024-09-21T04:08:24.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.425+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc19d340 con 0x7f47dc0fed80 2024-09-21T04:08:24.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.427+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc0fe840 con 0x7f47dc0fed80 2024-09-21T04:08:24.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.427+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f47d0009990 con 0x7f47dc0fed80 2024-09-21T04:08:24.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.427+0000 7f47d8ff9640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 0x7f47c406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:24.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.428+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f47d008ac30 con 0x7f47dc0fed80 2024-09-21T04:08:24.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.428+0000 7f47db7fe640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 0x7f47c406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:24.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.428+0000 7f47db7fe640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 0x7f47c406f020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc031430 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:24.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.432+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f47d0056190 con 0x7f47dc0fed80 2024-09-21T04:08:24.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.597+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f47dc10a880 con 0x7f47dc0fed80 2024-09-21T04:08:24.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.601+0000 7f47d8ff9640 1 -- 172.21.15.110:0/3922915218 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f47d00170c0 con 0x7f47dc0fed80 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 msgr2=0x7f47c406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 0x7f47c406f020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc031430 comp rx=0 tx=0).stop 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 msgr2=0x7f47dc197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d720 tx=0x7f47d000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 shutdown_connections 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f47c406cb60 0x7f47c406f020 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47dc0fed80 0x7f47dc197c40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 --2- 172.21.15.110:0/3922915218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47dc0fe440 0x7f47dc197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 >> 172.21.15.110:0/3922915218 conn(0x7f47dc0fa1f0 msgr2=0x7f47dc106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:24.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 shutdown_connections 2024-09-21T04:08:24.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:24.603+0000 7f47e1d29640 1 -- 172.21.15.110:0/3922915218 wait complete. 2024-09-21T04:08:25.460 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:25 smithi110 ceph-mon[27723]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:25.460 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:25 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3922915218' entity='client.admin' 2024-09-21T04:08:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:25 smithi148 ceph-mon[34957]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:25 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3922915218' entity='client.admin' 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 -- 172.21.15.110:0/2619612935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0ffca0 msgr2=0x7f85ac100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 --2- 172.21.15.110:0/2619612935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0ffca0 0x7f85ac100120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8598009880 tx=0x7f859802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 -- 172.21.15.110:0/2619612935 shutdown_connections 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 --2- 172.21.15.110:0/2619612935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0ffca0 0x7f85ac100120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 --2- 172.21.15.110:0/2619612935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0feaa0 0x7f85ac0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 -- 172.21.15.110:0/2619612935 >> 172.21.15.110:0/2619612935 conn(0x7f85ac0fa230 msgr2=0x7f85ac0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 -- 172.21.15.110:0/2619612935 shutdown_connections 2024-09-21T04:08:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.128+0000 7f85b2feb640 1 -- 172.21.15.110:0/2619612935 wait complete. 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 Processor -- start 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 -- start start 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 0x7f85ac195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac196060 con 0x7f85ac0feaa0 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b2feb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1961d0 con 0x7f85ac0ffca0 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57532/0 (socket says 172.21.15.110:57532) 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 -- 172.21.15.110:0/1047358656 learned_addr learned my addr 172.21.15.110:0/1047358656 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85abfff640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 0x7f85ac195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 -- 172.21.15.110:0/1047358656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 msgr2=0x7f85ac195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 0x7f85ac195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 -- 172.21.15.110:0/1047358656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009560 con 0x7f85ac0feaa0 2024-09-21T04:08:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85abfff640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 0x7f85ac195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.129+0000 7f85b0d87640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c920 tx=0x7f85a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.130+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a00041f0 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.130+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac19ac70 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.130+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f85a0007e50 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.130+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a00049d0 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.130+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac19b1c0 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.131+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac1075e0 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.131+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f85a0004350 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.132+0000 7f85a9ffb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 0x7f859c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.132+0000 7f85abfff640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 0x7f859c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.132+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f85a008af60 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.132+0000 7f85abfff640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 0x7f859c06f090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85ac196a70 tx=0x7f8598031430 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.136+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f85a0056410 con 0x7f85ac0feaa0 2024-09-21T04:08:25.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.304+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f85ac19b3b0 con 0x7f85ac0feaa0 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.308+0000 7f85a9ffb640 1 -- 172.21.15.110:0/1047358656 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v20)=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f85a008a720 con 0x7f85ac0feaa0 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 msgr2=0x7f859c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 0x7f859c06f090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85ac196a70 tx=0x7f8598031430 comp rx=0 tx=0).stop 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 msgr2=0x7f85ac195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c920 tx=0x7f85a000cde0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 shutdown_connections 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85ac0ffca0 0x7f85ac195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f859c06cbd0 0x7f859c06f090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 --2- 172.21.15.110:0/1047358656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85ac0feaa0 0x7f85ac195550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 >> 172.21.15.110:0/1047358656 conn(0x7f85ac0fa230 msgr2=0x7f85ac0684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 shutdown_connections 2024-09-21T04:08:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:20.310+0000 7f85b2feb640 1 -- 172.21.15.110:0/1047358656 wait complete. 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 -- 172.21.15.110:0/617747551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 msgr2=0x7f29201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 --2- 172.21.15.110:0/617747551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29201029b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2910009c50 tx=0x7f291002a9a0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 -- 172.21.15.110:0/617747551 shutdown_connections 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 --2- 172.21.15.110:0/617747551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2920102ef0 0x7f29201052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 --2- 172.21.15.110:0/617747551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29201029b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 -- 172.21.15.110:0/617747551 >> 172.21.15.110:0/617747551 conn(0x7f29200fa1c0 msgr2=0x7f29200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 -- 172.21.15.110:0/617747551 shutdown_connections 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.963+0000 7f2927aa2640 1 -- 172.21.15.110:0/617747551 wait complete. 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 Processor -- start 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 -- start start 2024-09-21T04:08:26.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2920102ef0 0x7f29200fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200feab0 con 0x7f29201005c0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200fec20 con 0x7f2920102ef0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42648/0 (socket says 172.21.15.110:42648) 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 -- 172.21.15.110:0/1809585022 learned_addr learned my addr 172.21.15.110:0/1809585022 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 -- 172.21.15.110:0/1809585022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2920102ef0 msgr2=0x7f29200fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2920102ef0 0x7f29200fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 -- 172.21.15.110:0/1809585022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910009900 con 0x7f29201005c0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f292583e640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2910009c20 tx=0x7f2910030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2910003970 con 0x7f29201005c0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f2910004050 con 0x7f29201005c0 2024-09-21T04:08:26.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.964+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200feea0 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.965+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291003a620 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.965+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292019d6e0 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.965+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920100a20 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.966+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f291000a040 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.966+0000 7f291effd640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 0x7f290806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.967+0000 7f292503d640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 0x7f290806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.967+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f29100ac920 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.967+0000 7f292503d640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 0x7f290806f090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29200ffca0 tx=0x7f2914006c90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:25.971+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2910077e50 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.138+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f29201086f0 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.142+0000 7f291effd640 1 -- 172.21.15.110:0/1809585022 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) v1 ==== 120+0+0 (secure 0 0 0) 0x7f29100ac0e0 con 0x7f29201005c0 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 msgr2=0x7f290806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:26.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 0x7f290806f090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29200ffca0 tx=0x7f2914006c90 comp rx=0 tx=0).stop 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 msgr2=0x7f29200fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2910009c20 tx=0x7f2910030890 comp rx=0 tx=0).stop 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 shutdown_connections 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2920102ef0 0x7f29200fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f290806cbd0 0x7f290806f090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 --2- 172.21.15.110:0/1809585022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29201005c0 0x7f29200fff20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 >> 172.21.15.110:0/1809585022 conn(0x7f29200fa1c0 msgr2=0x7f29200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 shutdown_connections 2024-09-21T04:08:26.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:26.144+0000 7f2927aa2640 1 -- 172.21.15.110:0/1809585022 wait complete. 2024-09-21T04:08:27.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:27 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1809585022' entity='client.admin' 2024-09-21T04:08:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:27 smithi110 ceph-mon[27723]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:27 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1809585022' entity='client.admin' 2024-09-21T04:08:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:27 smithi148 ceph-mon[34957]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:27.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1133102576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74100680 msgr2=0x7eff74102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:27.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1133102576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74100680 0x7eff74102a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff70009880 tx=0x7eff7002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1133102576 shutdown_connections 2024-09-21T04:08:27.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1133102576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74102fb0 0x7eff741053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1133102576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74100680 0x7eff74102a70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1133102576 >> 172.21.15.110:0/1133102576 conn(0x7eff740fa1c0 msgr2=0x7eff740fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1133102576 shutdown_connections 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.447+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1133102576 wait complete. 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 Processor -- start 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 -- start start 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74100680 0x7eff740fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff740feb70 con 0x7eff74102fb0 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7b4d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff740fece0 con 0x7eff74100680 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff78a6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff78a6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42664/0 (socket says 172.21.15.110:42664) 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff78a6e640 1 -- 172.21.15.110:0/1041250809 learned_addr learned my addr 172.21.15.110:0/1041250809 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff7926f640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74100680 0x7eff740fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:27.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff78a6e640 1 -- 172.21.15.110:0/1041250809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74100680 msgr2=0x7eff740fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.448+0000 7eff78a6e640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74100680 0x7eff740fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff78a6e640 1 -- 172.21.15.110:0/1041250809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009560 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff78a6e640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff6400b3d0 tx=0x7eff6400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff64011800 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7eff64011e00 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff740fefc0 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6400f2e0 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.449+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7419d690 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.450+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff74100ae0 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.450+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7eff64010380 con 0x7eff74102fb0 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.451+0000 7eff6a7fc640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 0x7eff6006f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:27.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.451+0000 7eff7926f640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 0x7eff6006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:27.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.451+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7eff6408ae60 con 0x7eff74102fb0 2024-09-21T04:08:27.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.451+0000 7eff7926f640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 0x7eff6006f090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff70000c00 tx=0x7eff700043a0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:27.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.455+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7eff64056310 con 0x7eff74102fb0 2024-09-21T04:08:27.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.622+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7eff741086f0 con 0x7eff74102fb0 2024-09-21T04:08:27.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.626+0000 7eff6a7fc640 1 -- 172.21.15.110:0/1041250809 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v25)=0 v25) v1 ==== 119+0+0 (secure 0 0 0) 0x7eff6408a620 con 0x7eff74102fb0 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 msgr2=0x7eff6006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 0x7eff6006f090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff70000c00 tx=0x7eff700043a0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 msgr2=0x7eff740fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff6400b3d0 tx=0x7eff6400b890 comp rx=0 tx=0).stop 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 shutdown_connections 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7eff6006cbd0 0x7eff6006f090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff74102fb0 0x7eff740fe630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 --2- 172.21.15.110:0/1041250809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff74100680 0x7eff740fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:27.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 >> 172.21.15.110:0/1041250809 conn(0x7eff740fa1c0 msgr2=0x7eff740fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:27.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.628+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 shutdown_connections 2024-09-21T04:08:27.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:27.629+0000 7eff7b4d3640 1 -- 172.21.15.110:0/1041250809 wait complete. 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- 172.21.15.110:0/285896757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 msgr2=0x7f3c540fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/285896757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c540fff90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009880 tx=0x7f3c4802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- 172.21.15.110:0/285896757 shutdown_connections 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/285896757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 0x7f3c54101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/285896757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c540fff90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- 172.21.15.110:0/285896757 >> 172.21.15.110:0/285896757 conn(0x7f3c540fb320 msgr2=0x7f3c540fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- 172.21.15.110:0/285896757 shutdown_connections 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- 172.21.15.110:0/285896757 wait complete. 2024-09-21T04:08:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 Processor -- start 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- start start 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 0x7f3c54195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541960a0 con 0x7f3c540ffb90 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.086+0000 7f3c5ad68640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54196210 con 0x7f3c54100d90 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42684/0 (socket says 172.21.15.110:42684) 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 -- 172.21.15.110:0/1404171251 learned_addr learned my addr 172.21.15.110:0/1404171251 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c53fff640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 0x7f3c54195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 -- 172.21.15.110:0/1404171251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 msgr2=0x7f3c54195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 0x7f3c54195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 -- 172.21.15.110:0/1404171251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48009560 con 0x7f3c540ffb90 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c58b04640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004cd0 tx=0x7f3c48004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:28.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c480054e0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1084+0+0 (secure 0 0 0) 0x7f3c48005640 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5419ac50 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c48031c30 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.087+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5419b140 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.088+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54108730 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.089+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f3c480035f0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.089+0000 7f3c51ffb640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 0x7f3c4406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.089+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3c480ac2b0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.089+0000 7f3c53fff640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 0x7f3c4406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.090+0000 7f3c53fff640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 0x7f3c4406f090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c54196ab0 tx=0x7f3c40009110 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.093+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3c480777e0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.265+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f3c5419b3b0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.269+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c480777e0 con 0x7f3c540ffb90 2024-09-21T04:08:28.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.269+0000 7f3c51ffb640 1 -- 172.21.15.110:0/1404171251 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v26)=0 v26) v1 ==== 114+0+0 (secure 0 0 0) 0x7f3c4804a0b0 con 0x7f3c540ffb90 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 msgr2=0x7f3c4406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 0x7f3c4406f090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c54196ab0 tx=0x7f3c40009110 comp rx=0 tx=0).stop 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 msgr2=0x7f3c54195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004cd0 tx=0x7f3c48004d00 comp rx=0 tx=0).stop 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 shutdown_connections 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c54100d90 0x7f3c54195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3c4406cbd0 0x7f3c4406f090 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 --2- 172.21.15.110:0/1404171251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c540ffb90 0x7f3c54195590 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 >> 172.21.15.110:0/1404171251 conn(0x7f3c540fb320 msgr2=0x7f3c540fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 shutdown_connections 2024-09-21T04:08:28.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:28.271+0000 7f3c5ad68640 1 -- 172.21.15.110:0/1404171251 wait complete. 2024-09-21T04:08:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:28 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1041250809' entity='client.admin' 2024-09-21T04:08:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:28 smithi110 ceph-mon[27723]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:28 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1404171251' entity='client.admin' 2024-09-21T04:08:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:28 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1041250809' entity='client.admin' 2024-09-21T04:08:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:28 smithi148 ceph-mon[34957]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:28 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1404171251' entity='client.admin' 2024-09-21T04:08:29.369 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi110') 2024-09-21T04:08:29.370 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi110') 2024-09-21T04:08:29.370 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi110') 2024-09-21T04:08:29.370 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi110') 2024-09-21T04:08:29.370 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi110') 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- 172.21.15.110:0/1115164840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 msgr2=0x7f10541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 --2- 172.21.15.110:0/1115164840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f10541029b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1044009c50 tx=0x7f104402a9a0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- 172.21.15.110:0/1115164840 shutdown_connections 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 --2- 172.21.15.110:0/1115164840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 0x7f10541052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 --2- 172.21.15.110:0/1115164840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f10541029b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- 172.21.15.110:0/1115164840 >> 172.21.15.110:0/1115164840 conn(0x7f10540fa1c0 msgr2=0x7f10540fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- 172.21.15.110:0/1115164840 shutdown_connections 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- 172.21.15.110:0/1115164840 wait complete. 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 Processor -- start 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- start start 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 0x7f1054195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.279+0000 7f105b72f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054196260 con 0x7f10541005c0 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f105b72f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541963d0 con 0x7f1054102ef0 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42708/0 (socket says 172.21.15.110:42708) 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 -- 172.21.15.110:0/3115453465 learned_addr learned my addr 172.21.15.110:0/3115453465 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:30.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f1058cca640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 0x7f1054195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 -- 172.21.15.110:0/3115453465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 msgr2=0x7f1054195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 0x7f1054195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 -- 172.21.15.110:0/3115453465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044009900 con 0x7f10541005c0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10594cb640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10440098d0 tx=0x7f1044030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1044003b40 con 0x7f10541005c0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1044030cc0 con 0x7f10541005c0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105419ae10 con 0x7f10541005c0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1044044790 con 0x7f10541005c0 2024-09-21T04:08:30.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.280+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105419b2f0 con 0x7f10541005c0 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.281+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10541087c0 con 0x7f10541005c0 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.282+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f104400a040 con 0x7f10541005c0 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.282+0000 7f10527fc640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 0x7f103c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.282+0000 7f1058cca640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 0x7f103c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.282+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f10440ac710 con 0x7f10541005c0 2024-09-21T04:08:30.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.283+0000 7f1058cca640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 0x7f103c06f090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1054196c70 tx=0x7f1048008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:30.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.287+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1044077bc0 con 0x7f10541005c0 2024-09-21T04:08:30.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.454+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f10541089a0 con 0x7f10541005c0 2024-09-21T04:08:30.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.458+0000 7f10527fc640 1 -- 172.21.15.110:0/3115453465 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v27)=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7f10440abed0 con 0x7f10541005c0 2024-09-21T04:08:30.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 msgr2=0x7f103c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 0x7f103c06f090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1054196c70 tx=0x7f1048008040 comp rx=0 tx=0).stop 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 msgr2=0x7f1054195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10440098d0 tx=0x7f1044030890 comp rx=0 tx=0).stop 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 shutdown_connections 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1054102ef0 0x7f1054195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f103c06cbd0 0x7f103c06f090 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 --2- 172.21.15.110:0/3115453465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10541005c0 0x7f1054195750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.460+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 >> 172.21.15.110:0/3115453465 conn(0x7f10540fa1c0 msgr2=0x7f10540fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.461+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 shutdown_connections 2024-09-21T04:08:30.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:30.461+0000 7f105b72f640 1 -- 172.21.15.110:0/3115453465 wait complete. 2024-09-21T04:08:31.291 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi110') 2024-09-21T04:08:31.291 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi110') 2024-09-21T04:08:31.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:31 smithi110 ceph-mon[27723]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:31.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:31 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3115453465' entity='client.admin' 2024-09-21T04:08:31.787 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi110') 2024-09-21T04:08:31.787 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi110') 2024-09-21T04:08:31.788 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi110') 2024-09-21T04:08:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:31 smithi148 ceph-mon[34957]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:31 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3115453465' entity='client.admin' 2024-09-21T04:08:32.119 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi110') 2024-09-21T04:08:32.119 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi110') 2024-09-21T04:08:32.119 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi110') 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 -- 172.21.15.110:0/3611013783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0fea30 msgr2=0x7f756c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 --2- 172.21.15.110:0/3611013783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0fea30 0x7f756c0fee30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f755c009880 tx=0x7f755c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 -- 172.21.15.110:0/3611013783 shutdown_connections 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 --2- 172.21.15.110:0/3611013783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0ffc30 0x7f756c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 --2- 172.21.15.110:0/3611013783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0fea30 0x7f756c0fee30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 -- 172.21.15.110:0/3611013783 >> 172.21.15.110:0/3611013783 conn(0x7f756c0fa1c0 msgr2=0x7f756c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 -- 172.21.15.110:0/3611013783 shutdown_connections 2024-09-21T04:08:32.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.611+0000 7f75722cb640 1 -- 172.21.15.110:0/3611013783 wait complete. 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 Processor -- start 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 -- start start 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0fea30 0x7f756c195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c196210 con 0x7f756c0ffc30 2024-09-21T04:08:32.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f75722cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c196380 con 0x7f756c0fea30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60172/0 (socket says 172.21.15.110:60172) 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 -- 172.21.15.110:0/1905155012 learned_addr learned my addr 172.21.15.110:0/1905155012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756bfff640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0fea30 0x7f756c195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 -- 172.21.15.110:0/1905155012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0fea30 msgr2=0x7f756c195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0fea30 0x7f756c195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.612+0000 7f756b7fe640 1 -- 172.21.15.110:0/1905155012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009560 con 0x7f756c0ffc30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f756b7fe640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f756000b450 tx=0x7f756000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7560011860 con 0x7f756c0ffc30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7560011e80 con 0x7f756c0ffc30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c19ae20 con 0x7f756c0ffc30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f756000f570 con 0x7f756c0ffc30 2024-09-21T04:08:32.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.613+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c19b3c0 con 0x7f756c0ffc30 2024-09-21T04:08:32.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.614+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c0feeb0 con 0x7f756c0ffc30 2024-09-21T04:08:32.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.614+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f75600103e0 con 0x7f756c0ffc30 2024-09-21T04:08:32.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.615+0000 7f75697fa640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 0x7f755406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:32.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.615+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f756008af80 con 0x7f756c0ffc30 2024-09-21T04:08:32.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.615+0000 7f756bfff640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 0x7f755406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:32.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.615+0000 7f756bfff640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 0x7f755406f090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f755c031540 tx=0x7f755c000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:32.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.619+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f75600bee20 con 0x7f756c0ffc30 2024-09-21T04:08:32.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.784+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f756c19b690 con 0x7f756c0ffc30 2024-09-21T04:08:32.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.788+0000 7f75697fa640 1 -- 172.21.15.110:0/1905155012 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v28)=0 v28) v1 ==== 120+0+0 (secure 0 0 0) 0x7f75600564b0 con 0x7f756c0ffc30 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 msgr2=0x7f755406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 0x7f755406f090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f755c031540 tx=0x7f755c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 msgr2=0x7f756c195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f756000b450 tx=0x7f756000b910 comp rx=0 tx=0).stop 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 shutdown_connections 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f755406cbd0 0x7f755406f090 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f756c0ffc30 0x7f756c195c40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 --2- 172.21.15.110:0/1905155012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f756c0fea30 0x7f756c195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 >> 172.21.15.110:0/1905155012 conn(0x7f756c0fa1c0 msgr2=0x7f756c067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:32.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 shutdown_connections 2024-09-21T04:08:32.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:32.790+0000 7f75722cb640 1 -- 172.21.15.110:0/1905155012 wait complete. 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1905155012' entity='client.admin' 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:33.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:33.631 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi110') 2024-09-21T04:08:33.632 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi110') 2024-09-21T04:08:33.632 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '77422328-77ce-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi110') 2024-09-21T04:08:33.632 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T04:08:33.632 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:08:33.632 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T04:08:33.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T04:08:33.649 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1905155012' entity='client.admin' 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:33.703 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 04:07:11.751045115 +0000 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 04:01:13.056344292 +0000 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 04:01:13.056344292 +0000 2024-09-21T04:08:33.704 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 04:01:13.056344292 +0000 2024-09-21T04:08:33.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T04:08:33.769 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T04:08:33.769 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T04:08:33.769 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000110259 s, 4.6 MB/s 2024-09-21T04:08:33.770 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T04:08:33.826 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 04:07:11.751045115 +0000 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 04:01:13.367338972 +0000 2024-09-21T04:08:33.882 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 04:01:13.367338972 +0000 2024-09-21T04:08:33.883 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 04:01:13.367338972 +0000 2024-09-21T04:08:33.883 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T04:08:33.946 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T04:08:33.947 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T04:08:33.947 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000108726 s, 4.7 MB/s 2024-09-21T04:08:33.948 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T04:08:34.004 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-21T04:08:34.060 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T04:08:34.060 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.060 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 04:07:11.751045115 +0000 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 04:01:13.679333633 +0000 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 04:01:13.679333633 +0000 2024-09-21T04:08:34.061 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 04:01:13.679333633 +0000 2024-09-21T04:08:34.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T04:08:34.124 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T04:08:34.124 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T04:08:34.124 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.3756e-05 s, 5.5 MB/s 2024-09-21T04:08:34.125 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T04:08:34.182 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-21T04:08:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T04:08:34.237 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 04:07:11.752045099 +0000 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 04:01:14.011327953 +0000 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 04:01:14.011327953 +0000 2024-09-21T04:08:34.238 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 04:01:14.011327953 +0000 2024-09-21T04:08:34.238 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T04:08:34.300 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T04:08:34.301 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T04:08:34.301 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000128982 s, 4.0 MB/s 2024-09-21T04:08:34.302 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T04:08:34.358 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:08:34.358 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T04:08:34.379 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T04:08:34.379 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.437 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 04:07:43.303676412 +0000 2024-09-21T04:08:34.438 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 04:01:12.094053227 +0000 2024-09-21T04:08:34.438 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 04:01:12.094053227 +0000 2024-09-21T04:08:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 04:01:12.094053227 +0000 2024-09-21T04:08:34.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T04:08:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T04:08:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T04:08:34.504 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000251843 s, 2.0 MB/s 2024-09-21T04:08:34.506 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T04:08:34.563 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-21T04:08:34.620 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T04:08:34.620 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.620 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 04:07:43.304676389 +0000 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 04:01:12.426045633 +0000 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 04:01:12.426045633 +0000 2024-09-21T04:08:34.621 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 04:01:12.426045633 +0000 2024-09-21T04:08:34.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T04:08:34.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:34 smithi110 ceph-mon[27723]: Reconfiguring crash.smithi110 (monmap changed)... 2024-09-21T04:08:34.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:34 smithi110 ceph-mon[27723]: Reconfiguring daemon crash.smithi110 on smithi110 2024-09-21T04:08:34.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:34 smithi148 ceph-mon[34957]: Reconfiguring crash.smithi110 (monmap changed)... 2024-09-21T04:08:34.682 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:34 smithi148 ceph-mon[34957]: Reconfiguring daemon crash.smithi110 on smithi110 2024-09-21T04:08:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T04:08:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T04:08:34.684 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000247453 s, 2.1 MB/s 2024-09-21T04:08:34.686 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T04:08:34.742 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 04:07:43.304676389 +0000 2024-09-21T04:08:34.797 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 04:01:12.751038200 +0000 2024-09-21T04:08:34.798 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 04:01:12.751038200 +0000 2024-09-21T04:08:34.798 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 04:01:12.751038200 +0000 2024-09-21T04:08:34.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T04:08:34.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T04:08:34.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T04:08:34.861 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000226391 s, 2.3 MB/s 2024-09-21T04:08:34.862 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T04:08:34.919 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-21T04:08:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T04:08:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:08:34.975 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 04:07:43.304676389 +0000 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 04:01:13.082030632 +0000 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 04:01:13.082030632 +0000 2024-09-21T04:08:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 04:01:13.082030632 +0000 2024-09-21T04:08:34.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T04:08:35.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T04:08:35.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T04:08:35.038 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000222759 s, 2.3 MB/s 2024-09-21T04:08:35.040 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T04:08:35.096 INFO:tasks.cephadm:Deploying osd.0 on smithi110 with /dev/vg_nvme/lv_4... 2024-09-21T04:08:35.096 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T04:08:35.883 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:35.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:35 smithi110 ceph-mon[27723]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:35.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:35 smithi148 ceph-mon[34957]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:35.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:35.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:36.606 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:36 smithi110 ceph-mon[27723]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-09-21T04:08:36.606 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:36 smithi110 ceph-mon[27723]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-09-21T04:08:36.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:36 smithi148 ceph-mon[34957]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-09-21T04:08:36.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:36 smithi148 ceph-mon[34957]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-09-21T04:08:37.626 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:37 smithi110 ceph-mon[27723]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:37.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:37 smithi148 ceph-mon[34957]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:39.548 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:39 smithi110 ceph-mon[27723]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:39 smithi148 ceph-mon[34957]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:41 smithi110 ceph-mon[27723]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:41 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:41 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:41 smithi148 ceph-mon[34957]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:41 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:41 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:08:42.357 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-09-21T04:08:42.658 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:42 smithi110 ceph-mon[27723]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-09-21T04:08:42.658 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:42 smithi110 ceph-mon[27723]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-09-21T04:08:42.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:42 smithi148 ceph-mon[34957]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-09-21T04:08:42.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:42 smithi148 ceph-mon[34957]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-09-21T04:08:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:43 smithi148 ceph-mon[34957]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:43.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:43 smithi110 ceph-mon[27723]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:44.661 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:08:45.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:45 smithi110 ceph-mon[27723]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:45.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:45 smithi148 ceph-mon[34957]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:46.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 -- 172.21.15.110:0/4084323247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 msgr2=0x7f2b84102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:46.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4084323247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84102a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009c50 tx=0x7f2b7402a9a0 comp rx=0 tx=0).stop 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 -- 172.21.15.110:0/4084323247 shutdown_connections 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4084323247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 0x7f2b84105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4084323247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84102a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 -- 172.21.15.110:0/4084323247 >> 172.21.15.110:0/4084323247 conn(0x7f2b840fa230 msgr2=0x7f2b840fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 -- 172.21.15.110:0/4084323247 shutdown_connections 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.909+0000 7f2b8a361640 1 -- 172.21.15.110:0/4084323247 wait complete. 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 Processor -- start 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 -- start start 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 0x7f2b84195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84196150 con 0x7f2b84100610 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b8a361640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841962c0 con 0x7f2b84102f40 2024-09-21T04:08:46.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55010/0 (socket says 172.21.15.110:55010) 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.910+0000 7f2b83fff640 1 -- 172.21.15.110:0/4242736184 learned_addr learned my addr 172.21.15.110:0/4242736184 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b837fe640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 0x7f2b84195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b83fff640 1 -- 172.21.15.110:0/4242736184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 msgr2=0x7f2b84195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b83fff640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 0x7f2b84195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b83fff640 1 -- 172.21.15.110:0/4242736184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74009900 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b83fff640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006010 tx=0x7f2b74030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b74003b40 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b74030cc0 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b74044700 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8419ad00 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.911+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8419b1a0 con 0x7f2b84100610 2024-09-21T04:08:46.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.912+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b841087c0 con 0x7f2b84100610 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.912+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f2b7400a040 con 0x7f2b84100610 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.913+0000 7f2b817fa640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 0x7f2b6c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.913+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2b740ac5f0 con 0x7f2b84100610 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.913+0000 7f2b837fe640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 0x7f2b6c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.913+0000 7f2b837fe640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 0x7f2b6c06f090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005fd0 tx=0x7f2b78005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:46.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:46.917+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2b74077aa0 con 0x7f2b84100610 2024-09-21T04:08:47.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:08:47.106+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b8419b410 con 0x7f2b6c06cbd0 2024-09-21T04:08:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:47 smithi110 ceph-mon[27723]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:47 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:08:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:47 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:08:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:47 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:47 smithi148 ceph-mon[34957]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:47 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:08:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:47 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:08:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:47 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:48 smithi148 ceph-mon[34957]: from='client.14320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:08:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:48.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:48 smithi110 ceph-mon[27723]: from='client.14320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:08:48.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:48.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:48.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:08:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:49 smithi110 ceph-mon[27723]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:08:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:49 smithi110 ceph-mon[27723]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:08:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:49 smithi110 ceph-mon[27723]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:49 smithi148 ceph-mon[34957]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:08:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:49 smithi148 ceph-mon[34957]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:08:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:49 smithi148 ceph-mon[34957]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:51.810 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:51 smithi110 ceph-mon[27723]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:51.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:51 smithi148 ceph-mon[34957]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:53.599 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:53 smithi148 ceph-mon[34957]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:53.599 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:53.599 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:53.599 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:08:53.599 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:53 smithi110 ceph-mon[27723]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:08:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: Reconfiguring ceph-exporter.smithi148 (monmap changed)... 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: Reconfiguring daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2336627724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "373c3716-ec2e-4d0a-9eb8-c192e38b47ae"}]: dispatch 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2336627724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "373c3716-ec2e-4d0a-9eb8-c192e38b47ae"}]': finished 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T04:08:54.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:08:54.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:54.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:54.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:08:54.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: Reconfiguring ceph-exporter.smithi148 (monmap changed)... 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: Reconfiguring daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2336627724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "373c3716-ec2e-4d0a-9eb8-c192e38b47ae"}]: dispatch 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2336627724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "373c3716-ec2e-4d0a-9eb8-c192e38b47ae"}]': finished 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:54.760 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:54.761 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:08:54.761 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:55.897 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:55 smithi148 ceph-mon[34957]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:55.897 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:55 smithi148 ceph-mon[34957]: Reconfiguring crash.smithi148 (monmap changed)... 2024-09-21T04:08:55.897 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:55 smithi148 ceph-mon[34957]: Reconfiguring daemon crash.smithi148 on smithi148 2024-09-21T04:08:55.898 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:55 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1382154042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:08:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:55 smithi110 ceph-mon[27723]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:55 smithi110 ceph-mon[27723]: Reconfiguring crash.smithi148 (monmap changed)... 2024-09-21T04:08:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:55 smithi110 ceph-mon[27723]: Reconfiguring daemon crash.smithi148 on smithi148 2024-09-21T04:08:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:55 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1382154042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: Reconfiguring mgr.smithi148.zjrahi (monmap changed)... 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: Reconfiguring daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:08:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:56 smithi110 ceph-mon[27723]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: Reconfiguring mgr.smithi148.zjrahi (monmap changed)... 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: Reconfiguring daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:08:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:56 smithi148 ceph-mon[34957]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: Reconfiguring mon.smithi148 (monmap changed)... 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:58.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:57 smithi110 ceph-mon[27723]: Reconfiguring daemon mon.smithi148 on smithi148 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: Reconfiguring mon.smithi148 (monmap changed)... 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:08:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:57 smithi148 ceph-mon[34957]: Reconfiguring daemon mon.smithi148 on smithi148 2024-09-21T04:08:59.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:08:59.267 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:08:59.268 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:08:58 smithi148 ceph-mon[34957]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:08:59.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:08:59.351 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:08:58 smithi110 ceph-mon[27723]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:01.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:01.095 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:01.095 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:00 smithi110 ceph-mon[27723]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:00 smithi148 ceph-mon[34957]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:03.264 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:03 smithi110 ceph-mon[27723]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:03.264 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-21T04:09:03.264 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:03.264 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:03 smithi110 ceph-mon[27723]: Deploying daemon osd.0 on smithi110 2024-09-21T04:09:03.264 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:03 smithi148 ceph-mon[34957]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-21T04:09:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:03 smithi148 ceph-mon[34957]: Deploying daemon osd.0 on smithi110 2024-09-21T04:09:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:05.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:05 smithi110 ceph-mon[27723]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:05.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:05 smithi148 ceph-mon[34957]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:06.636 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:06 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:06.636 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:06 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:06.637 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:06 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:06.637 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:06 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:06.637 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:06 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:06 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:06 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:06 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:06 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:06 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:07.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:07 smithi110 ceph-mon[27723]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:07 smithi148 ceph-mon[34957]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:09.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:09 smithi110 ceph-mon[27723]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:09.620 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:09.620 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:09.620 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:09 smithi148 ceph-mon[34957]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:11.716 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:11 smithi110 ceph-mon[27723]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:11.716 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:11.716 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:11.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.744+0000 7f2b817fa640 1 -- 172.21.15.110:0/4242736184 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b8419b410 con 0x7f2b6c06cbd0 2024-09-21T04:09:11.746 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 0 on host 'smithi110' 2024-09-21T04:09:11.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 msgr2=0x7f2b6c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 0x7f2b6c06f090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b78005fd0 tx=0x7f2b78005850 comp rx=0 tx=0).stop 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 msgr2=0x7f2b84195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006010 tx=0x7f2b74030890 comp rx=0 tx=0).stop 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 shutdown_connections 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b84102f40 0x7f2b84195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f2b6c06cbd0 0x7f2b6c06f090 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 --2- 172.21.15.110:0/4242736184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b84100610 0x7f2b84195640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 >> 172.21.15.110:0/4242736184 conn(0x7f2b840fa230 msgr2=0x7f2b840fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 shutdown_connections 2024-09-21T04:09:11.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:11.747+0000 7f2b8a361640 1 -- 172.21.15.110:0/4242736184 wait complete. 2024-09-21T04:09:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:11 smithi148 ceph-mon[34957]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:12.722 DEBUG:teuthology.orchestra.run.smithi110:osd.0> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T04:09:12.724 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_3... 2024-09-21T04:09:12.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T04:09:13.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:13.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:13.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:12 smithi110 ceph-mon[27723]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:13.052 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:09:12 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:09:12.819+0000 7f881a048640 -1 osd.0 0 log_to_monitors true 2024-09-21T04:09:13.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:13.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:13.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:12 smithi148 ceph-mon[34957]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:14.068 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:13 smithi110 ceph-mon[27723]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-21T04:09:14.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:13 smithi148 ceph-mon[34957]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-21T04:09:14.213 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:09:14.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:14 smithi110 ceph-mon[27723]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:09:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:14 smithi110 ceph-mon[27723]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T04:09:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:14 smithi110 ceph-mon[27723]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:09:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:14 smithi110 ceph-mon[27723]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:14 smithi148 ceph-mon[34957]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:09:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:14 smithi148 ceph-mon[34957]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T04:09:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:14 smithi148 ceph-mon[34957]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:09:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:14 smithi148 ceph-mon[34957]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:15.705 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:09:15.490+0000 7f8816fe1640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T04:09:15.705 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:09:15.494+0000 7f880ddd9640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:09:16.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-mon[27723]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:09:16.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-mon[27723]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T04:09:16.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:16.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:16.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:15 smithi110 ceph-mon[27723]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' 2024-09-21T04:09:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:15 smithi148 ceph-mon[34957]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:09:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:15 smithi148 ceph-mon[34957]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T04:09:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:15 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:15 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:15 smithi148 ceph-mon[34957]: from='osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904]' entity='osd.0' 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] boot 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T04:09:16.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: osd.0 [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] boot 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T04:09:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:09:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:18 smithi110 ceph-mon[27723]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T04:09:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:09:18.135 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-09-21T04:09:18.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:18.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:18.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:18 smithi148 ceph-mon[34957]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T04:09:18.675 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:09:19.068 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:19 smithi110 ceph-mon[27723]: pgmap v58: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:19.068 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:19 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:19 smithi148 ceph-mon[34957]: pgmap v58: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:19 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 -- 172.21.15.110:0/682692623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 msgr2=0x7f6230100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 --2- 172.21.15.110:0/682692623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230100490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f622c009880 tx=0x7f622c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 -- 172.21.15.110:0/682692623 shutdown_connections 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 --2- 172.21.15.110:0/682692623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f62300ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 --2- 172.21.15.110:0/682692623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230100490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 -- 172.21.15.110:0/682692623 >> 172.21.15.110:0/682692623 conn(0x7f62300f99b0 msgr2=0x7f62300fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 -- 172.21.15.110:0/682692623 shutdown_connections 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.406+0000 7f62374ff640 1 -- 172.21.15.110:0/682692623 wait complete. 2024-09-21T04:09:20.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 Processor -- start 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 -- start start 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f6230195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230196270 con 0x7f6230100090 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f62374ff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62301963e0 con 0x7f62300fecf0 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55860/0 (socket says 172.21.15.110:55860) 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 -- 172.21.15.110:0/1302410250 learned_addr learned my addr 172.21.15.110:0/1302410250 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f623529b640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f6230195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 -- 172.21.15.110:0/1302410250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 msgr2=0x7f6230195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f6230195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f6234a9a640 1 -- 172.21.15.110:0/1302410250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009560 con 0x7f6230100090 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.407+0000 7f623529b640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f6230195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f6234a9a640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f622000b400 tx=0x7f622000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:20.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6220011800 con 0x7f6230100090 2024-09-21T04:09:20.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6220011e20 con 0x7f6230100090 2024-09-21T04:09:20.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623019ae80 con 0x7f6230100090 2024-09-21T04:09:20.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f622000f2e0 con 0x7f6230100090 2024-09-21T04:09:20.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.408+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623019b3c0 con 0x7f6230100090 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.409+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62300ff1f0 con 0x7f6230100090 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.409+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7f6220010380 con 0x7f6230100090 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.410+0000 7f62267fc640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 0x7f621c06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.410+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f622008b060 con 0x7f6230100090 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.410+0000 7f623529b640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 0x7f621c06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:20.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.410+0000 7f623529b640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 0x7f621c06f090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f622c009230 tx=0x7f622c0314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:20.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.414+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f62200562b0 con 0x7f6230100090 2024-09-21T04:09:20.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:20.588+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f62300fbbc0 con 0x7f621c06cbd0 2024-09-21T04:09:21.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:21 smithi110 ceph-mon[27723]: pgmap v59: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:09:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:09:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:21 smithi148 ceph-mon[34957]: pgmap v59: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:09:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:09:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:22.409 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:22 smithi110 ceph-mon[27723]: from='client.14338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:09:22.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:22 smithi148 ceph-mon[34957]: from='client.14338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: pgmap v60: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.590 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: pgmap v60: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:23.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi110 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi110 to 11840M 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2647981184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a901efc6-b81c-4487-8056-0842cbb0dea5"}]: dispatch 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2647981184' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a901efc6-b81c-4487-8056-0842cbb0dea5"}]': finished 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T04:09:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:24.544 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi110 2024-09-21T04:09:24.544 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi110 to 11840M 2024-09-21T04:09:24.544 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:24.544 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2647981184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a901efc6-b81c-4487-8056-0842cbb0dea5"}]: dispatch 2024-09-21T04:09:24.545 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2647981184' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a901efc6-b81c-4487-8056-0842cbb0dea5"}]': finished 2024-09-21T04:09:24.545 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T04:09:24.545 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:24 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:25.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:25 smithi110 ceph-mon[27723]: pgmap v62: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:25 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/533508505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:09:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:25 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:25 smithi148 ceph-mon[34957]: pgmap v62: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:25 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/533508505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:09:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:25 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:25 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:27.530 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:27 smithi110 ceph-mon[27723]: pgmap v63: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:27 smithi148 ceph-mon[34957]: pgmap v63: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:28.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:28 smithi110 ceph-mon[27723]: pgmap v64: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:29.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:29.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:28 smithi148 ceph-mon[34957]: pgmap v64: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:31.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:31 smithi110 ceph-mon[27723]: pgmap v65: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:31.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:31 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-21T04:09:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:31 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:31 smithi148 ceph-mon[34957]: pgmap v65: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:31 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-21T04:09:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:31 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:32.506 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:32 smithi110 ceph-mon[27723]: Deploying daemon osd.1 on smithi110 2024-09-21T04:09:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:32 smithi148 ceph-mon[34957]: Deploying daemon osd.1 on smithi110 2024-09-21T04:09:33.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:33 smithi110 ceph-mon[27723]: pgmap v66: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:33.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:33 smithi148 ceph-mon[34957]: pgmap v66: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:35 smithi110 ceph-mon[27723]: pgmap v67: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:35 smithi148 ceph-mon[34957]: pgmap v67: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:37.358 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:37 smithi110 ceph-mon[27723]: pgmap v68: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:37.359 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:37.359 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:37.359 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:37 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:37.472 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:37 smithi148 ceph-mon[34957]: pgmap v68: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:37.473 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:37 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:37.473 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:37 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:37.473 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:37 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:39.243 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:38 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:39.243 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:38 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:39.243 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:38 smithi110 ceph-mon[27723]: pgmap v69: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:39.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.268+0000 7f62267fc640 1 -- 172.21.15.110:0/1302410250 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f62300fbbc0 con 0x7f621c06cbd0 2024-09-21T04:09:39.272 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 msgr2=0x7f621c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 0x7f621c06f090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f622c009230 tx=0x7f622c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 msgr2=0x7f6230195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f622000b400 tx=0x7f622000b8c0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 shutdown_connections 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f621c06cbd0 0x7f621c06f090 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6230100090 0x7f6230195ca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 --2- 172.21.15.110:0/1302410250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62300fecf0 0x7f6230195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 >> 172.21.15.110:0/1302410250 conn(0x7f62300f99b0 msgr2=0x7f62300fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 shutdown_connections 2024-09-21T04:09:39.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:39.270+0000 7f62374ff640 1 -- 172.21.15.110:0/1302410250 wait complete. 2024-09-21T04:09:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:38 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:38 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:38 smithi148 ceph-mon[34957]: pgmap v69: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:40.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:40.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:40 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:40.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:40 smithi110 ceph-mon[27723]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-21T04:09:40.303 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T04:09:40.305 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/vg_nvme/lv_2... 2024-09-21T04:09:40.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T04:09:40.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:40 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:40.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:40 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:40.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:40 smithi148 ceph-mon[34957]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-21T04:09:41.078 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:09:41.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:41 smithi110 ceph-mon[27723]: pgmap v70: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:41.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:41 smithi110 ceph-mon[27723]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:09:41.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:41 smithi110 ceph-mon[27723]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T04:09:41.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:41 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:41.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:41 smithi110 ceph-mon[27723]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:09:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:41 smithi148 ceph-mon[34957]: pgmap v70: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:41 smithi148 ceph-mon[34957]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:09:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:41 smithi148 ceph-mon[34957]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T04:09:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:41 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:41 smithi148 ceph-mon[34957]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:09:42.280 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[56910]: 2024-09-21T04:09:41.947+0000 7f3f2c6b8640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T04:09:42.281 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[56910]: 2024-09-21T04:09:42.027+0000 7f3f24ceb640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:09:42.644 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-mon[27723]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:09:42.644 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-mon[27723]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T04:09:42.644 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:42.644 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:42.645 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:42 smithi110 ceph-mon[27723]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' 2024-09-21T04:09:42.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:42 smithi148 ceph-mon[34957]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:09:42.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:42 smithi148 ceph-mon[34957]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T04:09:42.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:42 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:42.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:42 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:42.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:42 smithi148 ceph-mon[34957]: from='osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912]' entity='osd.1' 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: pgmap v73: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] boot 2024-09-21T04:09:43.642 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T04:09:43.643 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: pgmap v73: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: osd.1 [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] boot 2024-09-21T04:09:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T04:09:43.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:09:44.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-09-21T04:09:45.297 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:09:45.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:44 smithi110 ceph-mon[27723]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T04:09:45.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:44 smithi110 ceph-mon[27723]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:44 smithi148 ceph-mon[34957]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T04:09:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:44 smithi148 ceph-mon[34957]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:46.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:46 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:46.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:46 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:46 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:46 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:47.407 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:47 smithi110 ceph-mon[27723]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:47 smithi148 ceph-mon[34957]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 -- 172.21.15.110:0/157786829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200ffc50 msgr2=0x7fd7201000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 --2- 172.21.15.110:0/157786829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200ffc50 0x7fd7201000d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd714009880 tx=0x7fd71402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 -- 172.21.15.110:0/157786829 shutdown_connections 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 --2- 172.21.15.110:0/157786829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200ffc50 0x7fd7201000d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 --2- 172.21.15.110:0/157786829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7200fea50 0x7fd7200fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 -- 172.21.15.110:0/157786829 >> 172.21.15.110:0/157786829 conn(0x7fd7200fa1c0 msgr2=0x7fd7200fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:48.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 -- 172.21.15.110:0/157786829 shutdown_connections 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.032+0000 7fd726dec640 1 -- 172.21.15.110:0/157786829 wait complete. 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 Processor -- start 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 -- start start 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7200ffc50 0x7fd720194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201951a0 con 0x7fd7200fea50 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd726dec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720195310 con 0x7fd7200ffc50 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52340/0 (socket says 172.21.15.110:52340) 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 -- 172.21.15.110:0/737788609 learned_addr learned my addr 172.21.15.110:0/737788609 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 -- 172.21.15.110:0/737788609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7200ffc50 msgr2=0x7fd720194bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7200ffc50 0x7fd720194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:48.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 -- 172.21.15.110:0/737788609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714009560 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.033+0000 7fd724b88640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b400 tx=0x7fd71000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.034+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd710011800 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.034+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd710011e20 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.034+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd720199d60 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.034+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71000f2e0 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.034+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72019a330 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.035+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7201076e0 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.035+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7fd710010380 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.036+0000 7fd71dffb640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 0x7fd70806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.036+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fd71008b160 con 0x7fd7200fea50 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.036+0000 7fd71ffff640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 0x7fd70806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.036+0000 7fd71ffff640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 0x7fd70806f090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd71402a9c0 tx=0x7fd714000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:48.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.040+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd710056280 con 0x7fd7200fea50 2024-09-21T04:09:48.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:09:48.217+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd72019a5d0 con 0x7fd70806cbd0 2024-09-21T04:09:48.515 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:48.515 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:09:48.515 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:09:48.515 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:09:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:09:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:09:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:49 smithi110 ceph-mon[27723]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:49 smithi110 ceph-mon[27723]: from='client.14356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:09:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:49 smithi148 ceph-mon[34957]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:49 smithi148 ceph-mon[34957]: from='client.14356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi110 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi110 to 5920M 2024-09-21T04:09:51.124 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.125 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:51.125 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:51.125 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.125 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:51.125 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:50 smithi110 ceph-mon[27723]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi110 2024-09-21T04:09:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi110 to 5920M 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:09:51.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:50 smithi148 ceph-mon[34957]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:52.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:51 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/648362863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ef1cc98-8f37-4031-ac9e-9bf871256b1a"}]: dispatch 2024-09-21T04:09:52.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:51 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/648362863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ef1cc98-8f37-4031-ac9e-9bf871256b1a"}]': finished 2024-09-21T04:09:52.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:51 smithi110 ceph-mon[27723]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T04:09:52.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:51 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:09:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:51 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/648362863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ef1cc98-8f37-4031-ac9e-9bf871256b1a"}]: dispatch 2024-09-21T04:09:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:51 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/648362863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ef1cc98-8f37-4031-ac9e-9bf871256b1a"}]': finished 2024-09-21T04:09:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:51 smithi148 ceph-mon[34957]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T04:09:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:51 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:09:52.983 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:52 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2086537889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:09:53.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:52 smithi110 ceph-mon[27723]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:52 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2086537889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:09:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:52 smithi148 ceph-mon[34957]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:55.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:09:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:09:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:57 smithi110 ceph-mon[27723]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:57 smithi148 ceph-mon[34957]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:59 smithi110 ceph-mon[27723]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-21T04:09:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:09:59 smithi110 ceph-mon[27723]: Deploying daemon osd.2 on smithi110 2024-09-21T04:09:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:59 smithi148 ceph-mon[34957]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:09:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-21T04:09:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:09:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:09:59 smithi148 ceph-mon[34957]: Deploying daemon osd.2 on smithi110 2024-09-21T04:10:00.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:00 smithi110 ceph-mon[27723]: overall HEALTH_OK 2024-09-21T04:10:00.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:00 smithi148 ceph-mon[34957]: overall HEALTH_OK 2024-09-21T04:10:01.519 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:01 smithi110 ceph-mon[27723]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:01.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:01 smithi148 ceph-mon[34957]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:03.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:03 smithi110 ceph-mon[27723]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:03.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:03 smithi148 ceph-mon[34957]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:04.441 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:04 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:04.442 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:04 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:04.442 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:04 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:04.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:04 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:04.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:04 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:04.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:04 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:05.361 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:05 smithi110 ceph-mon[27723]: pgmap v87: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:05.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:05 smithi148 ceph-mon[34957]: pgmap v87: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:06.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.922+0000 7fd71dffb640 1 -- 172.21.15.110:0/737788609 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd72019a5d0 con 0x7fd70806cbd0 2024-09-21T04:10:06.926 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-09-21T04:10:06.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 msgr2=0x7fd70806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:06.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 0x7fd70806f090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd71402a9c0 tx=0x7fd714000bc0 comp rx=0 tx=0).stop 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 msgr2=0x7fd720194690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b400 tx=0x7fd71000b8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 shutdown_connections 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7200ffc50 0x7fd720194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd70806cbd0 0x7fd70806f090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 --2- 172.21.15.110:0/737788609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7200fea50 0x7fd720194690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 >> 172.21.15.110:0/737788609 conn(0x7fd7200fa1c0 msgr2=0x7fd720102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 shutdown_connections 2024-09-21T04:10:06.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:10:06.925+0000 7fd726dec640 1 -- 172.21.15.110:0/737788609 wait complete. 2024-09-21T04:10:07.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:07 smithi110 ceph-mon[27723]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:07.532 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:07 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:07.533 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:07 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:07.553 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T04:10:07.555 INFO:tasks.cephadm:Deploying osd.3 on smithi148 with /dev/vg_nvme/lv_4... 2024-09-21T04:10:07.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T04:10:07.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:07 smithi148 ceph-mon[34957]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:07.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:07 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:07.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:07 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:07.764 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:08.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:08 smithi148 ceph-mon[34957]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-21T04:10:08.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:08 smithi110 ceph-mon[27723]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-21T04:10:09.333 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:09 smithi148 ceph-mon[34957]: pgmap v89: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:09.333 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:09 smithi148 ceph-mon[34957]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:10:09.334 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:09 smithi148 ceph-mon[34957]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T04:10:09.334 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:09.334 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:09 smithi148 ceph-mon[34957]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:10:09.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-mon[27723]: pgmap v89: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:09.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-mon[27723]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:10:09.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-mon[27723]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T04:10:09.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:09.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-mon[27723]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-09-21T04:10:09.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:10:10.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-09-21T04:10:10.238 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[64226]: 2024-09-21T04:10:09.945+0000 7f83f3416640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T04:10:10.239 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:09 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[64226]: 2024-09-21T04:10:09.949+0000 7f83eba49640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:10:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:10.494 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:10 smithi148 ceph-mon[34957]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:10:10.495 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:10 smithi148 ceph-mon[34957]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T04:10:10.495 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:10 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:10.495 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:10 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:10.495 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:10 smithi148 ceph-mon[34957]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' 2024-09-21T04:10:10.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:10 smithi110 ceph-mon[27723]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T04:10:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:10 smithi110 ceph-mon[27723]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T04:10:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:10 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:10 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:10 smithi110 ceph-mon[27723]: from='osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533]' entity='osd.2' 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: pgmap v92: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] boot 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T04:10:11.405 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:11 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: pgmap v92: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:11.490 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: osd.2 [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] boot 2024-09-21T04:10:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T04:10:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:11 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 -- 172.21.15.148:0/4233457734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80ffca0 msgr2=0x7fecd8100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 --2- 172.21.15.148:0/4233457734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80ffca0 0x7fecd8100120 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009880 tx=0x7fecc402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 -- 172.21.15.148:0/4233457734 shutdown_connections 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 --2- 172.21.15.148:0/4233457734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80ffca0 0x7fecd8100120 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 --2- 172.21.15.148:0/4233457734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80feaa0 0x7fecd80feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 -- 172.21.15.148:0/4233457734 >> 172.21.15.148:0/4233457734 conn(0x7fecd80fa230 msgr2=0x7fecd80fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 -- 172.21.15.148:0/4233457734 shutdown_connections 2024-09-21T04:10:11.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.639+0000 7fecdec70640 1 -- 172.21.15.148:0/4233457734 wait complete. 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 Processor -- start 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 -- start start 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80ffca0 0x7fecd819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819a8d0 con 0x7fecd80ffca0 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdec70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819aa40 con 0x7fecd80feaa0 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42628/0 (socket says 172.21.15.148:42628) 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 -- 172.21.15.148:0/1964928802 learned_addr learned my addr 172.21.15.148:0/1964928802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T04:10:11.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecd7fff640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80ffca0 0x7fecd819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 -- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80ffca0 msgr2=0x7fecd819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80ffca0 0x7fecd819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 -- 172.21.15.148:0/1964928802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4009560 con 0x7fecd80feaa0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecdca0c640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feccc00d8d0 tx=0x7feccc00dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.640+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feccc009a30 con 0x7fecd80feaa0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.641+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd819ad20 con 0x7fecd80feaa0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.641+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feccc009b90 con 0x7fecd80feaa0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.641+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feccc00f680 con 0x7fecd80feaa0 2024-09-21T04:10:11.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.641+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd819b260 con 0x7fecd80feaa0 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.642+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd818e640 con 0x7fecd80feaa0 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.642+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 90796+0+0 (secure 0 0 0) 0x7feccc009d00 con 0x7fecd80feaa0 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.643+0000 7fecd5ffb640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 0x7fecc806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.643+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2846+0+0 (secure 0 0 0) 0x7feccc08b4a0 con 0x7fecd80feaa0 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.643+0000 7fecd7fff640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 0x7fecc806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:11.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.643+0000 7fecd7fff640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 0x7fecc806f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:11.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.647+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feccc056440 con 0x7fecd80feaa0 2024-09-21T04:10:11.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:11.821+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fecd802c560 con 0x7fecc806cbd0 2024-09-21T04:10:12.572 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:10:12.572 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:10:12.572 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:12.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:10:12.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:10:12.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: pgmap v94: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T04:10:13.643 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:13 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: pgmap v94: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T04:10:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:13 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:10:14.805 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:10:14.805 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:14.805 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68509]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:14.805 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68509]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:14.805 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2074087782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]': finished 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:14.806 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:14.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:14.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:14.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:14.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2074087782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]: dispatch 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]: dispatch 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80377dc2-d4fc-4bb6-8ae8-f295b4d9b774"}]': finished 2024-09-21T04:10:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:14.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:15.095 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:10:15.095 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:15.095 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68512]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:15.095 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:10:14 smithi110 sudo[68512]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:15.353 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68515]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:10:15.353 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68515]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:15.354 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68515]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:15.354 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68515]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68518]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68518]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68518]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68518]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68521]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68521]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:15.613 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68521]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:15.614 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68521]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi110 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi110 to 3946M 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2463567862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T04:10:15.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:15 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:15.916 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68524]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:10:15.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68524]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:15.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68524]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:15.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:10:15 smithi110 sudo[68524]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi110 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi110 to 3946M 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2463567862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T04:10:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:15 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:16.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:10:16.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:16.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68527]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:16.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68527]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:16.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:10:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68530]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 sudo[68530]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:16.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39715]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:10:16.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:16.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39715]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:16.872 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39715]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:16.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:16.879+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7feccc056810 con 0x7fecd80feaa0 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:10:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:16 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39719]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 sudo[39719]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:10:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:10:17.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:10:17.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:10:17.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:10:17.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:16 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:10:17.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:17 smithi148 ceph-mon[34957]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:17.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:17 smithi148 ceph-mon[34957]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:10:17.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:17 smithi148 ceph-mon[34957]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:10:17.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:17 smithi148 ceph-mon[34957]: mgrmap e19: smithi110.axmqzb(active, since 2m), standbys: smithi148.zjrahi 2024-09-21T04:10:17.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:17 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:17 smithi110 ceph-mon[27723]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:17 smithi110 ceph-mon[27723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:10:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:17 smithi110 ceph-mon[27723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:10:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:17 smithi110 ceph-mon[27723]: mgrmap e19: smithi110.axmqzb(active, since 2m), standbys: smithi148.zjrahi 2024-09-21T04:10:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:17 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:19.094 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:18 smithi148 ceph-mon[34957]: pgmap v101: 1 pgs: 1 creating+peering; 0 B data, 114 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:19.094 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:18 smithi110 ceph-mon[27723]: pgmap v101: 1 pgs: 1 creating+peering; 0 B data, 114 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:21.398 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:21 smithi148 ceph-mon[34957]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:21.398 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-21T04:10:21.398 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:21 smithi110 ceph-mon[27723]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-21T04:10:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:22.333 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:22 smithi148 ceph-mon[34957]: Deploying daemon osd.3 on smithi148 2024-09-21T04:10:22.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:22 smithi110 ceph-mon[27723]: Deploying daemon osd.3 on smithi148 2024-09-21T04:10:23.318 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:23 smithi110 ceph-mon[27723]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:23 smithi148 ceph-mon[34957]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:25 smithi110 ceph-mon[27723]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:25.439 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:25 smithi148 ceph-mon[34957]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:26.331 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:26 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:26.331 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:26 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:26.331 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:26 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:26.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:26 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:26.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:26 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:26.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:26 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:27.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:27 smithi148 ceph-mon[34957]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:27 smithi110 ceph-mon[27723]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:28.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.672+0000 7fecd5ffb640 1 -- 172.21.15.148:0/1964928802 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fecd802c560 con 0x7fecc806cbd0 2024-09-21T04:10:28.675 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 3 on host 'smithi148' 2024-09-21T04:10:28.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.674+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 msgr2=0x7fecc806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:28.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.674+0000 7fecdec70640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 0x7fecc806f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40314e0 comp rx=0 tx=0).stop 2024-09-21T04:10:28.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.674+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 msgr2=0x7fecd8068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:28.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.674+0000 7fecdec70640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feccc00d8d0 tx=0x7feccc00dd90 comp rx=0 tx=0).stop 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 shutdown_connections 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fecc806cbd0 0x7fecc806f090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecd80ffca0 0x7fecd819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 --2- 172.21.15.148:0/1964928802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd80feaa0 0x7fecd8068fa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 >> 172.21.15.148:0/1964928802 conn(0x7fecd80fa230 msgr2=0x7fecd8103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 shutdown_connections 2024-09-21T04:10:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:28.675+0000 7fecdec70640 1 -- 172.21.15.148:0/1964928802 wait complete. 2024-09-21T04:10:29.499 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:29 smithi148 ceph-mon[34957]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:29.499 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:29 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:29.499 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:29 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:29.522 DEBUG:teuthology.orchestra.run.smithi148:osd.3> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T04:10:29.524 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-09-21T04:10:29.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T04:10:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:29 smithi110 ceph-mon[27723]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:29.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:29 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:29.894 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:10:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:10:29.594+0000 7efccd09e640 -1 osd.3 0 log_to_monitors true 2024-09-21T04:10:30.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:30 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:30.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:30 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:30.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:30 smithi148 ceph-mon[34957]: from='osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:10:30.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:30 smithi148 ceph-mon[34957]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:10:30.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:30 smithi148 ceph-mon[34957]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:30 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:30 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:30 smithi110 ceph-mon[27723]: from='osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:10:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:30 smithi110 ceph-mon[27723]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:10:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:30 smithi110 ceph-mon[27723]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:31.780 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:31 smithi148 ceph-mon[34957]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:10:31.781 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:31 smithi148 ceph-mon[34957]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T04:10:31.781 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:31 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:31.781 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:31 smithi148 ceph-mon[34957]: from='osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:31.781 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:31 smithi148 ceph-mon[34957]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:31.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:31 smithi110 ceph-mon[27723]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:10:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:31 smithi110 ceph-mon[27723]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T04:10:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:31 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:31 smithi110 ceph-mon[27723]: from='osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:31 smithi110 ceph-mon[27723]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:32.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:10:32.434+0000 7efcc87fc640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T04:10:32.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:10:32.443+0000 7efcc0e2f640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:10:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:10:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: from='osd.3 ' entity='osd.3' 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] boot 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T04:10:32.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:32 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:32.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:10:32.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: from='osd.3 ' entity='osd.3' 2024-09-21T04:10:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: osd.3 [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] boot 2024-09-21T04:10:32.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T04:10:32.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:32 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:10:33.258 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:33.659 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:33.660 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:33 smithi148 ceph-mon[34957]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T04:10:33.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:33.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:33 smithi110 ceph-mon[27723]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 -- 172.21.15.148:0/708165913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 msgr2=0x7ffb700feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 --2- 172.21.15.148:0/708165913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb700feea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009880 tx=0x7ffb6002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 -- 172.21.15.148:0/708165913 shutdown_connections 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 --2- 172.21.15.148:0/708165913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 0x7ffb70100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 --2- 172.21.15.148:0/708165913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb700feea0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 -- 172.21.15.148:0/708165913 >> 172.21.15.148:0/708165913 conn(0x7ffb700fa230 msgr2=0x7ffb700fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 -- 172.21.15.148:0/708165913 shutdown_connections 2024-09-21T04:10:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.433+0000 7ffb76555640 1 -- 172.21.15.148:0/708165913 wait complete. 2024-09-21T04:10:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 Processor -- start 2024-09-21T04:10:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 -- start start 2024-09-21T04:10:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 0x7ffb70195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb70196140 con 0x7ffb700ffca0 2024-09-21T04:10:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb76555640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb701962b0 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54480/0 (socket says 172.21.15.148:54480) 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 -- 172.21.15.148:0/2377195718 learned_addr learned my addr 172.21.15.148:0/2377195718 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6f7fe640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 0x7ffb70195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 -- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 msgr2=0x7ffb70195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 0x7ffb70195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.434+0000 7ffb6ffff640 1 -- 172.21.15.148:0/2377195718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009560 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb6ffff640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a9c0 tx=0x7ffb600054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb60004aa0 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7019acf0 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb60004c00 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb60003a50 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.435+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7019b210 con 0x7ffb700feaa0 2024-09-21T04:10:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.436+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb70107690 con 0x7ffb700feaa0 2024-09-21T04:10:35.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.436+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ffb60003cd0 con 0x7ffb700feaa0 2024-09-21T04:10:35.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.437+0000 7ffb6d7fa640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 0x7ffb5806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:35.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.437+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3803+0+0 (secure 0 0 0) 0x7ffb600acd60 con 0x7ffb700feaa0 2024-09-21T04:10:35.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.437+0000 7ffb6f7fe640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 0x7ffb5806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:35.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.437+0000 7ffb6f7fe640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 0x7ffb5806f110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb64005fd0 tx=0x7ffb64005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:35.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.441+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffb600777b0 con 0x7ffb700feaa0 2024-09-21T04:10:35.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:35.611+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb7019b400 con 0x7ffb5806cc50 2024-09-21T04:10:35.614 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 152 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:35.615 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:35 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 152 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:35.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:35.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:35 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:36.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi148 2024-09-21T04:10:36.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi148 to 14912M 2024-09-21T04:10:36.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: osdmap e29: 4 total, 4 up, 4 in 2024-09-21T04:10:36.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:10:36.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:10:36.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:36 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi148 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi148 to 14912M 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: osdmap e29: 4 total, 4 up, 4 in 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:10:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:36 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:37 smithi110 ceph-mon[27723]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:10:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:37 smithi110 ceph-mon[27723]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 152 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:37 smithi110 ceph-mon[27723]: osdmap e30: 4 total, 4 up, 4 in 2024-09-21T04:10:37.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:37 smithi148 ceph-mon[34957]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:10:37.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:37 smithi148 ceph-mon[34957]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 152 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:37.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:37 smithi148 ceph-mon[34957]: osdmap e30: 4 total, 4 up, 4 in 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2228540920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]: dispatch 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]: dispatch 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]': finished 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:38 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3711182853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:10:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2228540920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]: dispatch 2024-09-21T04:10:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]: dispatch 2024-09-21T04:10:38.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfbc2e07-ddea-44e4-8d0f-df489240ee36"}]': finished 2024-09-21T04:10:38.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T04:10:38.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:38.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:38 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3711182853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:10:39.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:39 smithi110 ceph-mon[27723]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:39 smithi148 ceph-mon[34957]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:41 smithi110 ceph-mon[27723]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T04:10:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:41 smithi148 ceph-mon[34957]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T04:10:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:43 smithi110 ceph-mon[27723]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T04:10:43.744 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:43 smithi148 ceph-mon[34957]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T04:10:45.659 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:45 smithi148 ceph-mon[34957]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T04:10:45.659 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-21T04:10:45.659 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:45.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:45 smithi110 ceph-mon[27723]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T04:10:45.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-21T04:10:45.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:46 smithi110 ceph-mon[27723]: Deploying daemon osd.4 on smithi148 2024-09-21T04:10:46.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:46 smithi148 ceph-mon[34957]: Deploying daemon osd.4 on smithi148 2024-09-21T04:10:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:47 smithi148 ceph-mon[34957]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T04:10:47.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:47 smithi110 ceph-mon[27723]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T04:10:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:48.783 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:10:49.815 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:49 smithi148 ceph-mon[34957]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T04:10:49.815 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:49.815 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:49.815 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:49.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:49 smithi110 ceph-mon[27723]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T04:10:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:49.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:51.779 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:51 smithi148 ceph-mon[34957]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:10:51.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:51 smithi110 ceph-mon[27723]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:10:52.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.496+0000 7ffb6d7fa640 1 -- 172.21.15.148:0/2377195718 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb7019b400 con 0x7ffb5806cc50 2024-09-21T04:10:52.500 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 msgr2=0x7ffb5806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 0x7ffb5806f110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffb64005fd0 tx=0x7ffb64005850 comp rx=0 tx=0).stop 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 msgr2=0x7ffb70195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a9c0 tx=0x7ffb600054f0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 shutdown_connections 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ffb5806cc50 0x7ffb5806f110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb700ffca0 0x7ffb70195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 --2- 172.21.15.148:0/2377195718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb700feaa0 0x7ffb70195630 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 >> 172.21.15.148:0/2377195718 conn(0x7ffb700fa230 msgr2=0x7ffb700684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 shutdown_connections 2024-09-21T04:10:52.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:52.499+0000 7ffb76555640 1 -- 172.21.15.148:0/2377195718 wait complete. 2024-09-21T04:10:53.413 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T04:10:53.415 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_2... 2024-09-21T04:10:53.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T04:10:53.662 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:53 smithi110 ceph-mon[27723]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:53 smithi110 ceph-mon[27723]: from='osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:10:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:53 smithi110 ceph-mon[27723]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:10:53.669 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:53 smithi148 ceph-mon[34957]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:53.669 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:53.669 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:53.669 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:53 smithi148 ceph-mon[34957]: from='osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:10:53.669 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:53 smithi148 ceph-mon[34957]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:10:54.616 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: osdmap e32: 5 total, 4 up, 5 in 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:54.617 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:54 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: osdmap e32: 5 total, 4 up, 5 in 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:54.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:54 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:55.587 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:55.588 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:10:55.588 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: osdmap e33: 5 total, 4 up, 5 in 2024-09-21T04:10:55.588 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:55.588 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:55.588 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-mon[34957]: from='osd.4 ' entity='osd.4' 2024-09-21T04:10:55.588 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[47367]: 2024-09-21T04:10:55.318+0000 7fb3d6a08640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T04:10:55.588 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:10:55 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[47367]: 2024-09-21T04:10:55.327+0000 7fb3cf03b640 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: osdmap e33: 5 total, 4 up, 5 in 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:55 smithi110 ceph-mon[27723]: from='osd.4 ' entity='osd.4' 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] boot 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: osdmap e34: 5 total, 5 up, 5 in 2024-09-21T04:10:56.627 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:56.897 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T04:10:56.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-09-21T04:10:56.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:10:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:10:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: osd.4 [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] boot 2024-09-21T04:10:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: osdmap e34: 5 total, 5 up, 5 in 2024-09-21T04:10:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:10:57.482 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:10:57.817 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:57 smithi148 ceph-mon[34957]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:57.817 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:57 smithi148 ceph-mon[34957]: osdmap e35: 5 total, 5 up, 5 in 2024-09-21T04:10:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:57 smithi110 ceph-mon[27723]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 148 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:10:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:57 smithi110 ceph-mon[27723]: osdmap e35: 5 total, 5 up, 5 in 2024-09-21T04:10:59.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.526+0000 7fd237090640 1 -- 172.21.15.148:0/937952897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 msgr2=0x7fd2301012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.526+0000 7fd237090640 1 --2- 172.21.15.148:0/937952897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd2301012a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097e0 tx=0x7fd22002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- 172.21.15.148:0/937952897 shutdown_connections 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 --2- 172.21.15.148:0/937952897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd2301012a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 --2- 172.21.15.148:0/937952897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 0x7fd2300fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- 172.21.15.148:0/937952897 >> 172.21.15.148:0/937952897 conn(0x7fd2300f99b0 msgr2=0x7fd2300fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- 172.21.15.148:0/937952897 shutdown_connections 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- 172.21.15.148:0/937952897 wait complete. 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 Processor -- start 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- start start 2024-09-21T04:10:59.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 0x7fd230195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd230196140 con 0x7fd2300fde40 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.527+0000 7fd237090640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2301962b0 con 0x7fd230100e20 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59440/0 (socket says 172.21.15.148:59440) 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 -- 172.21.15.148:0/399098018 learned_addr learned my addr 172.21.15.148:0/399098018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd234e2c640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 0x7fd230195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 -- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 msgr2=0x7fd230195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 0x7fd230195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 -- 172.21.15.148:0/399098018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009490 con 0x7fd230100e20 2024-09-21T04:10:59.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd227fff640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd220030c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2200037c0 con 0x7fd230100e20 2024-09-21T04:10:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2200044f0 con 0x7fd230100e20 2024-09-21T04:10:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23019acf0 con 0x7fd230100e20 2024-09-21T04:10:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.528+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd220031d20 con 0x7fd230100e20 2024-09-21T04:10:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.529+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23019b1e0 con 0x7fd230100e20 2024-09-21T04:10:59.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.530+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23018f710 con 0x7fd230100e20 2024-09-21T04:10:59.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.530+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd220003920 con 0x7fd230100e20 2024-09-21T04:10:59.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.530+0000 7fd225ffb640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 0x7fd21806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:59.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.531+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fd2200ad3d0 con 0x7fd230100e20 2024-09-21T04:10:59.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.531+0000 7fd234e2c640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 0x7fd21806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:59.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.531+0000 7fd234e2c640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 0x7fd21806f120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd230068200 tx=0x7fd22c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:59.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.535+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd220077d10 con 0x7fd230100e20 2024-09-21T04:10:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:10:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:59.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:10:59 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:10:59.710+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd23002c4f0 con 0x7fd21806cc60 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:10:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:10:59.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:10:59.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:10:59.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:10:59 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:00.633 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:00 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi148 2024-09-21T04:11:00.633 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:00 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi148 to 7456M 2024-09-21T04:11:00.633 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:11:00.633 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:11:00.633 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:00 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi148 2024-09-21T04:11:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:00 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi148 to 7456M 2024-09-21T04:11:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:11:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:11:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:01.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:01 smithi110 ceph-mon[27723]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:11:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:01 smithi110 ceph-mon[27723]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:01.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:01 smithi148 ceph-mon[34957]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:11:01.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:01 smithi148 ceph-mon[34957]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3510571655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]: dispatch 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]: dispatch 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]': finished 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:02.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:02 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/2552644204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3510571655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]: dispatch 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]: dispatch 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5dd029b-19d1-4ce9-9377-cce1d2b70348"}]': finished 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:02 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/2552644204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:11:03.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:03 smithi110 ceph-mon[27723]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:03 smithi148 ceph-mon[34957]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:04.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:04 smithi110 ceph-mon[27723]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:04.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:04 smithi148 ceph-mon[34957]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:07.505 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:07 smithi148 ceph-mon[34957]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:07.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:07 smithi110 ceph-mon[27723]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:09.362 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:09 smithi148 ceph-mon[34957]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:09.362 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-21T04:11:09.362 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:09.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:09 smithi110 ceph-mon[27723]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:09.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-21T04:11:09.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:10.285 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:10 smithi148 ceph-mon[34957]: Deploying daemon osd.5 on smithi148 2024-09-21T04:11:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:10 smithi110 ceph-mon[27723]: Deploying daemon osd.5 on smithi148 2024-09-21T04:11:11.417 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:11 smithi148 ceph-mon[34957]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:11.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:11 smithi110 ceph-mon[27723]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:13.349 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:13 smithi148 ceph-mon[34957]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:13.650 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:13 smithi110 ceph-mon[27723]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:14.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:11:14.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:14.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:14 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:14.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:11:14.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:14.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:14 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:15 smithi110 ceph-mon[27723]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:15.678 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:15 smithi148 ceph-mon[34957]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:16.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.995+0000 7fd225ffb640 1 -- 172.21.15.148:0/399098018 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd23002c4f0 con 0x7fd21806cc60 2024-09-21T04:11:16.999 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 msgr2=0x7fd21806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 0x7fd21806f120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd230068200 tx=0x7fd22c009260 comp rx=0 tx=0).stop 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 msgr2=0x7fd230195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd220030c00 comp rx=0 tx=0).stop 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 shutdown_connections 2024-09-21T04:11:17.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd230100e20 0x7fd230195b70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:17.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd21806cc60 0x7fd21806f120 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:17.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 --2- 172.21.15.148:0/399098018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2300fde40 0x7fd230195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:17.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.997+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 >> 172.21.15.148:0/399098018 conn(0x7fd2300f99b0 msgr2=0x7fd230104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:17.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.998+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 shutdown_connections 2024-09-21T04:11:17.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:16.998+0000 7fd237090640 1 -- 172.21.15.148:0/399098018 wait complete. 2024-09-21T04:11:17.542 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T04:11:17.544 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-21T04:11:17.544 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:11:17.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:17 smithi148 ceph-mon[34957]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:17.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:17 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:17.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:17 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:17.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:17 smithi148 ceph-mon[34957]: from='osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:11:17.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:17 smithi148 ceph-mon[34957]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:11:17.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:17 smithi110 ceph-mon[27723]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:17 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:17 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:17 smithi110 ceph-mon[27723]: from='osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:11:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:17 smithi110 ceph-mon[27723]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:11:17.762 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:11:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T04:11:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:18.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:11:18.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:11:18.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:18.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:19.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:19.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:19.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- 172.21.15.110:0/2851625154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9190101280 msgr2=0x7f9190101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 --2- 172.21.15.110:0/2851625154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9190101280 0x7f9190101680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9184009880 tx=0x7f918402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- 172.21.15.110:0/2851625154 shutdown_connections 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 --2- 172.21.15.110:0/2851625154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91900ff7c0 0x7f91900ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 --2- 172.21.15.110:0/2851625154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9190101280 0x7f9190101680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- 172.21.15.110:0/2851625154 >> 172.21.15.110:0/2851625154 conn(0x7f91900fb330 msgr2=0x7f91900fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- 172.21.15.110:0/2851625154 shutdown_connections 2024-09-21T04:11:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- 172.21.15.110:0/2851625154 wait complete. 2024-09-21T04:11:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 Processor -- start 2024-09-21T04:11:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.513+0000 7f9197854640 1 -- start start 2024-09-21T04:11:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190101280 0x7f9190195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901962a0 con 0x7f91900ff7c0 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9190196410 con 0x7f9190101280 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49788/0 (socket says 172.21.15.110:49788) 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 -- 172.21.15.110:0/2149234458 learned_addr learned my addr 172.21.15.110:0/2149234458 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9194def640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190101280 0x7f9190195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 -- 172.21.15.110:0/2149234458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190101280 msgr2=0x7f9190195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190101280 0x7f9190195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 -- 172.21.15.110:0/2149234458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184009560 con 0x7f91900ff7c0 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91955f0640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f918402a9c0 tx=0x7f91840055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9184004aa0 con 0x7f91900ff7c0 2024-09-21T04:11:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919019ae50 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9184004c00 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91840039b0 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.514+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919019b340 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.515+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91900ffc40 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.516+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9184031c00 con 0x7f91900ff7c0 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.516+0000 7f91827fc640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 0x7f917c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.516+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f91840aceb0 con 0x7f91900ff7c0 2024-09-21T04:11:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.516+0000 7f9194def640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 0x7f917c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.517+0000 7f9194def640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 0x7f917c06f110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9190196cb0 tx=0x7f918c00b390 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.521+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9184077780 con 0x7f91900ff7c0 2024-09-21T04:11:19.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.685+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f919002c4a0 con 0x7f91900ff7c0 2024-09-21T04:11:19.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.686+0000 7f91827fc640 1 -- 172.21.15.110:0/2149234458 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f918404a0a0 con 0x7f91900ff7c0 2024-09-21T04:11:19.689 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:19.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 msgr2=0x7f917c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:19.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 0x7f917c06f110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9190196cb0 tx=0x7f918c00b390 comp rx=0 tx=0).stop 2024-09-21T04:11:19.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 msgr2=0x7f9190195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:19.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f918402a9c0 tx=0x7f91840055a0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 shutdown_connections 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9190101280 0x7f9190195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f917c06cc50 0x7f917c06f110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 --2- 172.21.15.110:0/2149234458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91900ff7c0 0x7f9190195790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 >> 172.21.15.110:0/2149234458 conn(0x7f91900fb330 msgr2=0x7f9190104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:19.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 shutdown_connections 2024-09-21T04:11:19.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:19.688+0000 7f9197854640 1 -- 172.21.15.110:0/2149234458 wait complete. 2024-09-21T04:11:20.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:11:20.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: osdmap e38: 6 total, 5 up, 6 in 2024-09-21T04:11:20.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:20.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:20.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2149234458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:11:20.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:20 smithi148 ceph-mon[34957]: from='osd.5 ' entity='osd.5' 2024-09-21T04:11:20.191 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:11:19 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[54279]: 2024-09-21T04:11:19.848+0000 7f4fea904640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T04:11:20.191 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:11:19 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[54279]: 2024-09-21T04:11:19.858+0000 7f4fe2f37640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:11:20.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-21T04:11:20.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: osdmap e38: 6 total, 5 up, 6 in 2024-09-21T04:11:20.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:20.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:20.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2149234458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:11:20.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:20 smithi110 ceph-mon[27723]: from='osd.5 ' entity='osd.5' 2024-09-21T04:11:20.400 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1726891855,"num_in_osds":6,"osd_in_since":1726891861,"num_remapped_pgs":0} 2024-09-21T04:11:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: purged_snaps scrub starts 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: purged_snaps scrub ok 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] boot 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:21.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:11:21.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: purged_snaps scrub starts 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: purged_snaps scrub ok 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: osd.5 [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] boot 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:11:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:11:21.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:21.603 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: Detected new or changed devices on smithi148 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi148 to 4970M 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:11:22.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:22 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: Detected new or changed devices on smithi148 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi148 to 4970M 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:11:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:22 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 -- 172.21.15.110:0/3126439061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 msgr2=0x7f0d480feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 --2- 172.21.15.110:0/3126439061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d480feea0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009880 tx=0x7f0d3802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 -- 172.21.15.110:0/3126439061 shutdown_connections 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 --2- 172.21.15.110:0/3126439061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d48100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 --2- 172.21.15.110:0/3126439061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d480feea0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 -- 172.21.15.110:0/3126439061 >> 172.21.15.110:0/3126439061 conn(0x7f0d480fa230 msgr2=0x7f0d480fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 -- 172.21.15.110:0/3126439061 shutdown_connections 2024-09-21T04:11:23.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 -- 172.21.15.110:0/3126439061 wait complete. 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.186+0000 7f0d4d583640 1 Processor -- start 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d4d583640 1 -- start start 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d4d583640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d4d583640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d4819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d4d583640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4819aa40 con 0x7f0d480feaa0 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d4d583640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4819abb0 con 0x7f0d480ffca0 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58414/0 (socket says 172.21.15.110:58414) 2024-09-21T04:11:23.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 -- 172.21.15.110:0/4172084587 learned_addr learned my addr 172.21.15.110:0/4172084587 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 -- 172.21.15.110:0/4172084587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 msgr2=0x7f0d4819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d467fc640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d4819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d4819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d46ffd640 1 -- 172.21.15.110:0/4172084587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38009560 con 0x7f0d480feaa0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.187+0000 7f0d467fc640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d4819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d46ffd640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d3802a9c0 tx=0x7f0d38005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d38004b20 con 0x7f0d480feaa0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4819ae30 con 0x7f0d480feaa0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d38004c80 con 0x7f0d480feaa0 2024-09-21T04:11:23.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d38004b20 con 0x7f0d480feaa0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.188+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4819b320 con 0x7f0d480feaa0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.189+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4818e6a0 con 0x7f0d480feaa0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.189+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f0d38003a50 con 0x7f0d480feaa0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.190+0000 7f0d2ffff640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 0x7f0d3006f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.190+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0d380acee0 con 0x7f0d480feaa0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.190+0000 7f0d467fc640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 0x7f0d3006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:23.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.190+0000 7f0d467fc640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 0x7f0d3006f180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c006fd0 tx=0x7f0d3c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:23.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.194+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0d380775f0 con 0x7f0d480feaa0 2024-09-21T04:11:23.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.355+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0d4802c480 con 0x7f0d480feaa0 2024-09-21T04:11:23.358 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:23 smithi110 ceph-mon[27723]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T04:11:23.358 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:23 smithi110 ceph-mon[27723]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:23.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.356+0000 7f0d2ffff640 1 -- 172.21.15.110:0/4172084587 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f0d3807ac10 con 0x7f0d480feaa0 2024-09-21T04:11:23.359 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:23.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 msgr2=0x7f0d3006f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:23.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 0x7f0d3006f180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c006fd0 tx=0x7f0d3c008040 comp rx=0 tx=0).stop 2024-09-21T04:11:23.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 msgr2=0x7f0d48068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:23.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d3802a9c0 tx=0x7f0d38005610 comp rx=0 tx=0).stop 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 shutdown_connections 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d480ffca0 0x7f0d4819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0d3006ccc0 0x7f0d3006f180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 --2- 172.21.15.110:0/4172084587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d480feaa0 0x7f0d48068f10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.357+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 >> 172.21.15.110:0/4172084587 conn(0x7f0d480fa230 msgr2=0x7f0d481033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.358+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 shutdown_connections 2024-09-21T04:11:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:23.358+0000 7f0d4d583640 1 -- 172.21.15.110:0/4172084587 wait complete. 2024-09-21T04:11:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:23 smithi148 ceph-mon[34957]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T04:11:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:23 smithi148 ceph-mon[34957]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:24.091 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1726891880,"num_in_osds":6,"osd_in_since":1726891861,"num_remapped_pgs":0} 2024-09-21T04:11:24.091 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:11:24.330 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:24.354 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:24 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4172084587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:11:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4172084587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:11:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:25 smithi110 ceph-mon[27723]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:25.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:25 smithi148 ceph-mon[34957]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2771192930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 msgr2=0x7fc9c00fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2771192930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c00fffa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009880 tx=0x7fc9b002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2771192930 shutdown_connections 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2771192930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c00fffa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2771192930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.950+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2771192930 >> 172.21.15.110:0/2771192930 conn(0x7fc9c00fb840 msgr2=0x7fc9c00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2771192930 shutdown_connections 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2771192930 wait complete. 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 Processor -- start 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 -- start start 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c019c420 con 0x7fc9c00ffb40 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c76fb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0195a10 con 0x7fc9c0105bf0 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c5497640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.951+0000 7fc9c5497640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58436/0 (socket says 172.21.15.110:58436) 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c5497640 1 -- 172.21.15.110:0/2985210458 learned_addr learned my addr 172.21.15.110:0/2985210458 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c4c96640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c5497640 1 -- 172.21.15.110:0/2985210458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 msgr2=0x7fc9c019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c5497640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c5497640 1 -- 172.21.15.110:0/2985210458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0009560 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c4c96640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c5497640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b420 tx=0x7fc9bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9bc011800 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9bc011e20 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c0195cf0 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9bc00f2e0 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.952+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c0196210 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.953+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c01011f0 con 0x7fc9c00ffb40 2024-09-21T04:11:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.954+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc9bc010380 con 0x7fc9c00ffb40 2024-09-21T04:11:25.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.954+0000 7fc9b67fc640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 0x7fc9ac06f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:25.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.954+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc9bc08b6e0 con 0x7fc9c00ffb40 2024-09-21T04:11:25.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.954+0000 7fc9c4c96640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 0x7fc9ac06f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:25.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.955+0000 7fc9c4c96640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 0x7fc9ac06f170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000c00 tx=0x7fc9b0005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:25.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:25.958+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc9bc055f10 con 0x7fc9c00ffb40 2024-09-21T04:11:26.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.119+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9c0100020 con 0x7fc9c00ffb40 2024-09-21T04:11:26.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.120+0000 7fc9b67fc640 1 -- 172.21.15.110:0/2985210458 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11324 (secure 0 0 0) 0x7fc9bc059530 con 0x7fc9c00ffb40 2024-09-21T04:11:26.123 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:26.124 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","created":"2024-09-21T04:05:22.443593+0000","modified":"2024-09-21T04:11:22.087199+0000","last_up_change":"2024-09-21T04:11:20.850929+0000","last_in_change":"2024-09-21T04:11:01.869822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:10:12.442999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"373c3716-ec2e-4d0a-9eb8-c192e38b47ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6803","nonce":3868599904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6805","nonce":3868599904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6809","nonce":3868599904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6807","nonce":3868599904}]},"public_addr":"172.21.15.110:6803/3868599904","cluster_addr":"172.21.15.110:6805/3868599904","heartbeat_back_addr":"172.21.15.110:6809/3868599904","heartbeat_front_addr":"172.21.15.110:6807/3868599904","state":["exists","up"]},{"osd":1,"uuid":"a901efc6-b81c-4487-8056-0842cbb0dea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6811","nonce":18252912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6813","nonce":18252912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6817","nonce":18252912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6815","nonce":18252912}]},"public_addr":"172.21.15.110:6811/18252912","cluster_addr":"172.21.15.110:6813/18252912","heartbeat_back_addr":"172.21.15.110:6817/18252912","heartbeat_front_addr":"172.21.15.110:6815/18252912","state":["exists","up"]},{"osd":2,"uuid":"4ef1cc98-8f37-4031-ac9e-9bf871256b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6819","nonce":3408208533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6821","nonce":3408208533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6825","nonce":3408208533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6823","nonce":3408208533}]},"public_addr":"172.21.15.110:6819/3408208533","cluster_addr":"172.21.15.110:6821/3408208533","heartbeat_back_addr":"172.21.15.110:6825/3408208533","heartbeat_front_addr":"172.21.15.110:6823/3408208533","state":["exists","up"]},{"osd":3,"uuid":"80377dc2-d4fc-4bb6-8ae8-f295b4d9b774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6801","nonce":76623163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6803","nonce":76623163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6807","nonce":76623163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6805","nonce":76623163}]},"public_addr":"172.21.15.148:6801/76623163","cluster_addr":"172.21.15.148:6803/76623163","heartbeat_back_addr":"172.21.15.148:6807/76623163","heartbeat_front_addr":"172.21.15.148:6805/76623163","state":["exists","up"]},{"osd":4,"uuid":"cfbc2e07-ddea-44e4-8d0f-df489240ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6809","nonce":564615118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6811","nonce":564615118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6815","nonce":564615118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6813","nonce":564615118}]},"public_addr":"172.21.15.148:6809/564615118","cluster_addr":"172.21.15.148:6811/564615118","heartbeat_back_addr":"172.21.15.148:6815/564615118","heartbeat_front_addr":"172.21.15.148:6813/564615118","state":["exists","up"]},{"osd":5,"uuid":"f5dd029b-19d1-4ce9-9377-cce1d2b70348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6817","nonce":958827228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6819","nonce":958827228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6823","nonce":958827228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6821","nonce":958827228}]},"public_addr":"172.21.15.148:6817/958827228","cluster_addr":"172.21.15.148:6819/958827228","heartbeat_back_addr":"172.21.15.148:6823/958827228","heartbeat_front_addr":"172.21.15.148:6821/958827228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:13.833321+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:41.103978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:08.627858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:30.608053+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:54.266275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:11:18.341884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6800/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:6801/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:6801/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2087016304":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/602456430":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2783964987":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/1589768070":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/598619296":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6801/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/1485849066":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/3142935389":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/2973624819":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/3973251001":"2024-09-22T04:07:18.131661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 msgr2=0x7fc9ac06f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 0x7fc9ac06f170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0000c00 tx=0x7fc9b0005a20 comp rx=0 tx=0).stop 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 msgr2=0x7fc9c019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b420 tx=0x7fc9bc00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 shutdown_connections 2024-09-21T04:11:26.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c0105bf0 0x7fc9c019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:26.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc9ac06ccb0 0x7fc9ac06f170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:26.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 --2- 172.21.15.110:0/2985210458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9c00ffb40 0x7fc9c019b850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:26.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 >> 172.21.15.110:0/2985210458 conn(0x7fc9c00fb840 msgr2=0x7fc9c00fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:26.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.122+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 shutdown_connections 2024-09-21T04:11:26.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:26.123+0000 7fc9c76fb640 1 -- 172.21.15.110:0/2985210458 wait complete. 2024-09-21T04:11:26.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:26 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2985210458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:26 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2985210458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:26.743 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T04:10:12.442999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T04:11:26.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T04:11:26.940 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:27.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:27 smithi110 ceph-mon[27723]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:27 smithi148 ceph-mon[34957]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 -- 172.21.15.110:0/1805446798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 msgr2=0x7ff02c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 --2- 172.21.15.110:0/1805446798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0fffa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff028009880 tx=0x7ff02802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 -- 172.21.15.110:0/1805446798 shutdown_connections 2024-09-21T04:11:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 --2- 172.21.15.110:0/1805446798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0fffa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 --2- 172.21.15.110:0/1805446798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 0x7ff02c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 -- 172.21.15.110:0/1805446798 >> 172.21.15.110:0/1805446798 conn(0x7ff02c0fb840 msgr2=0x7ff02c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 -- 172.21.15.110:0/1805446798 shutdown_connections 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.620+0000 7ff0345b2640 1 -- 172.21.15.110:0/1805446798 wait complete. 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 Processor -- start 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 -- start start 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 0x7ff02c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c073230 con 0x7ff02c0ffb40 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff0345b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c0733a0 con 0x7ff02c105bf0 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58450/0 (socket says 172.21.15.110:58450) 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 -- 172.21.15.110:0/2142266181 learned_addr learned my addr 172.21.15.110:0/2142266181 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff031b4d640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 0x7ff02c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 -- 172.21.15.110:0/2142266181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 msgr2=0x7ff02c072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 0x7ff02c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.621+0000 7ff03234e640 1 -- 172.21.15.110:0/2142266181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009560 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff03234e640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff02000b420 tx=0x7ff02000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff020011800 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff020011e20 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c1a1cf0 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff02000f2e0 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.622+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c1a20d0 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.623+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c1011f0 con 0x7ff02c0ffb40 2024-09-21T04:11:28.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.623+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff020010380 con 0x7ff02c0ffb40 2024-09-21T04:11:28.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.624+0000 7ff01f7fe640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 0x7ff01806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:28.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.624+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff02008b880 con 0x7ff02c0ffb40 2024-09-21T04:11:28.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.624+0000 7ff031b4d640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 0x7ff01806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:28.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.624+0000 7ff031b4d640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 0x7ff01806f110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff028000c00 tx=0x7ff028004120 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:28.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.628+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff020055f90 con 0x7ff02c0ffb40 2024-09-21T04:11:28.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.791+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff02c108010 con 0x7ff02c0ffb40 2024-09-21T04:11:28.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.792+0000 7ff01f7fe640 1 -- 172.21.15.110:0/2142266181 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0200595b0 con 0x7ff02c0ffb40 2024-09-21T04:11:28.795 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 msgr2=0x7ff01806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 0x7ff01806f110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff028000c00 tx=0x7ff028004120 comp rx=0 tx=0).stop 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 msgr2=0x7ff02c0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff02000b420 tx=0x7ff02000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 shutdown_connections 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c105bf0 0x7ff02c072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff01806cc50 0x7ff01806f110 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 --2- 172.21.15.110:0/2142266181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c0ffb40 0x7ff02c0727b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 >> 172.21.15.110:0/2142266181 conn(0x7ff02c0fb840 msgr2=0x7ff02c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 shutdown_connections 2024-09-21T04:11:28.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:28.794+0000 7ff0345b2640 1 -- 172.21.15.110:0/2142266181 wait complete. 2024-09-21T04:11:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:29 smithi148 ceph-mon[34957]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:29 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2142266181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-21T04:11:29.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:29 smithi110 ceph-mon[27723]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:29.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:29 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2142266181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-21T04:11:29.527 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T04:11:29.527 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T04:11:29.725 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:31.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.367+0000 7fa02df6a640 1 -- 172.21.15.110:0/1847366672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280fea70 msgr2=0x7fa0280fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.367+0000 7fa02df6a640 1 --2- 172.21.15.110:0/1847366672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280fea70 0x7fa0280fee70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa018009880 tx=0x7fa01802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 -- 172.21.15.110:0/1847366672 shutdown_connections 2024-09-21T04:11:31.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 --2- 172.21.15.110:0/1847366672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280ffc70 0x7fa0281000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 --2- 172.21.15.110:0/1847366672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280fea70 0x7fa0280fee70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 -- 172.21.15.110:0/1847366672 >> 172.21.15.110:0/1847366672 conn(0x7fa0280fa200 msgr2=0x7fa0280fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 -- 172.21.15.110:0/1847366672 shutdown_connections 2024-09-21T04:11:31.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 -- 172.21.15.110:0/1847366672 wait complete. 2024-09-21T04:11:31.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 Processor -- start 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.368+0000 7fa02df6a640 1 -- start start 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 0x7fa028102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028101590 con 0x7fa0280ffc70 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028101700 con 0x7fa0280fea70 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58476/0 (socket says 172.21.15.110:58476) 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 -- 172.21.15.110:0/257030206 learned_addr learned my addr 172.21.15.110:0/257030206 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa0277fe640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 0x7fa028102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 -- 172.21.15.110:0/257030206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 msgr2=0x7fa028102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 0x7fa028102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 -- 172.21.15.110:0/257030206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009560 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa0277fe640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 0x7fa028102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa026ffd640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b420 tx=0x7fa01c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c011800 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa01c011e20 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0281019e0 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c00f2e0 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.369+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02819d790 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.370+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0280feef0 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.371+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fa01c010380 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.371+0000 7fa024ff9640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 0x7fa01006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.371+0000 7fa0277fe640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 0x7fa01006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.371+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa01c08b880 con 0x7fa0280ffc70 2024-09-21T04:11:31.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.372+0000 7fa0277fe640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 0x7fa01006f110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa018009230 tx=0x7fa0180314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.376+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa01c055f90 con 0x7fa0280ffc70 2024-09-21T04:11:31.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:31 smithi110 ceph-mon[27723]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:31.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.601+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa0281076e0 con 0x7fa0280ffc70 2024-09-21T04:11:31.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.605+0000 7fa024ff9640 1 -- 172.21.15.110:0/257030206 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa01c0595b0 con 0x7fa0280ffc70 2024-09-21T04:11:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-21T04:11:31.619 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQBzR+5mHwruIxAArvN9oWSBNBSVNJVyvEfc7g== 2024-09-21T04:11:31.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 msgr2=0x7fa01006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 0x7fa01006f110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa018009230 tx=0x7fa0180314e0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 msgr2=0x7fa028101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b420 tx=0x7fa01c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 shutdown_connections 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa01006cc50 0x7fa01006f110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0280ffc70 0x7fa028101050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 --2- 172.21.15.110:0/257030206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0280fea70 0x7fa028102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 >> 172.21.15.110:0/257030206 conn(0x7fa0280fa200 msgr2=0x7fa028067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 shutdown_connections 2024-09-21T04:11:31.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:31.607+0000 7fa02df6a640 1 -- 172.21.15.110:0/257030206 wait complete. 2024-09-21T04:11:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:31 smithi148 ceph-mon[34957]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:32.326 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:11:32.326 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T04:11:32.326 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T04:11:32.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T04:11:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi148/config 2024-09-21T04:11:32.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:32 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/257030206' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:32.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:32 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/257030206' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:11:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:32 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/257030206' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:32 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/257030206' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:11:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:33 smithi148 ceph-mon[34957]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:33.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:33 smithi110 ceph-mon[27723]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:33.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:34.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 -- 172.21.15.148:0/3630774046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 msgr2=0x7fc8a4100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3630774046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4100120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc898009880 tx=0x7fc89802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 -- 172.21.15.148:0/3630774046 shutdown_connections 2024-09-21T04:11:34.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3630774046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4100120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3630774046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.085+0000 7fc8ac721640 1 -- 172.21.15.148:0/3630774046 >> 172.21.15.148:0/3630774046 conn(0x7fc8a40fa230 msgr2=0x7fc8a40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:34.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 -- 172.21.15.148:0/3630774046 shutdown_connections 2024-09-21T04:11:34.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 -- 172.21.15.148:0/3630774046 wait complete. 2024-09-21T04:11:34.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 Processor -- start 2024-09-21T04:11:34.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 -- start start 2024-09-21T04:11:34.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a4102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a41015c0 con 0x7fc8a40feaa0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.086+0000 7fc8ac721640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4101730 con 0x7fc8a40ffca0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42774/0 (socket says 172.21.15.148:42774) 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 -- 172.21.15.148:0/3972668136 learned_addr learned my addr 172.21.15.148:0/3972668136 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8aa4bd640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a4102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 -- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 msgr2=0x7fc8a4102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a4102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 -- 172.21.15.148:0/3972668136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898009560 con 0x7fc8a40ffca0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8aa4bd640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a9cbc640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc898006010 tx=0x7fc8980055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:34.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc898004b20 con 0x7fc8a40ffca0 2024-09-21T04:11:34.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc898004c80 con 0x7fc8a40ffca0 2024-09-21T04:11:34.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a41019b0 con 0x7fc8a40ffca0 2024-09-21T04:11:34.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc898003b80 con 0x7fc8a40ffca0 2024-09-21T04:11:34.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.087+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a419d7e0 con 0x7fc8a40ffca0 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.088+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a4107730 con 0x7fc8a40ffca0 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.089+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc898031640 con 0x7fc8a40ffca0 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.090+0000 7fc8a37fe640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 0x7fc88c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.090+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc8980ad130 con 0x7fc8a40ffca0 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.090+0000 7fc8aa4bd640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 0x7fc88c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:34.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.090+0000 7fc8aa4bd640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 0x7fc88c06f110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40ffb00 tx=0x7fc8940073c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:34.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.094+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc8980778c0 con 0x7fc8a40ffca0 2024-09-21T04:11:34.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.321+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc8a4107910 con 0x7fc8a40ffca0 2024-09-21T04:11:34.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.326+0000 7fc8a37fe640 1 -- 172.21.15.148:0/3972668136 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc89807aee0 con 0x7fc8a40ffca0 2024-09-21T04:11:34.329 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-09-21T04:11:34.329 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB2R+5myQlLExAA8Pg/15HjoPy6AIdFlL8qgw== 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 msgr2=0x7fc88c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 0x7fc88c06f110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40ffb00 tx=0x7fc8940073c0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 msgr2=0x7fc8a4101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc898006010 tx=0x7fc8980055f0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 shutdown_connections 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8a40ffca0 0x7fc8a4101080 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc88c06cc50 0x7fc88c06f110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 --2- 172.21.15.148:0/3972668136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8a40feaa0 0x7fc8a4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 >> 172.21.15.148:0/3972668136 conn(0x7fc8a40fa230 msgr2=0x7fc8a40682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:34.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 shutdown_connections 2024-09-21T04:11:34.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:11:34.328+0000 7fc8ac721640 1 -- 172.21.15.148:0/3972668136 wait complete. 2024-09-21T04:11:34.960 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:11:34.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-21T04:11:34.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T04:11:34.998 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T04:11:35.208 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:35.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:35 smithi110 ceph-mon[27723]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:35.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:35 smithi110 ceph-mon[27723]: from='client.? 172.21.15.148:0/3972668136' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:35.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:35 smithi110 ceph-mon[27723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:35.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:35 smithi110 ceph-mon[27723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:11:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:35 smithi148 ceph-mon[34957]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:35 smithi148 ceph-mon[34957]: from='client.? 172.21.15.148:0/3972668136' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:35 smithi148 ceph-mon[34957]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:11:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:35 smithi148 ceph-mon[34957]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:11:36.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 -- 172.21.15.110:0/3360003989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 msgr2=0x7f514c103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3360003989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c103e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f513c009880 tx=0x7f513c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 -- 172.21.15.110:0/3360003989 shutdown_connections 2024-09-21T04:11:36.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3360003989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3360003989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c103e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 -- 172.21.15.110:0/3360003989 >> 172.21.15.110:0/3360003989 conn(0x7f514c0f9ef0 msgr2=0x7f514c0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 -- 172.21.15.110:0/3360003989 shutdown_connections 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.942+0000 7f5151f3b640 1 -- 172.21.15.110:0/3360003989 wait complete. 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 Processor -- start 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 -- start start 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1989e0 con 0x7f514c103a80 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f5151f3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c19c750 con 0x7f514c0ff230 2024-09-21T04:11:36.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.943+0000 7f514affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48948/0 (socket says 172.21.15.110:48948) 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 -- 172.21.15.110:0/3473937090 learned_addr learned my addr 172.21.15.110:0/3473937090 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514b7fe640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 -- 172.21.15.110:0/3473937090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 msgr2=0x7f514c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 -- 172.21.15.110:0/3473937090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c009560 con 0x7f514c103a80 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514b7fe640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f514affd640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f514000d6f0 tx=0x7f514000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5140009a30 con 0x7f514c103a80 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51400054e0 con 0x7f514c103a80 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c19ca30 con 0x7f514c103a80 2024-09-21T04:11:36.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5140010460 con 0x7f514c103a80 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.944+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c19cf80 con 0x7f514c103a80 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.945+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c100240 con 0x7f514c103a80 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.945+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5140009b90 con 0x7f514c103a80 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.946+0000 7f5148ff9640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 0x7f513406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.946+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f514008bac0 con 0x7f514c103a80 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.946+0000 7f514b7fe640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 0x7f513406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.946+0000 7f514b7fe640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 0x7f513406f110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c005e60 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:36.950+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f51400561d0 con 0x7f514c103a80 2024-09-21T04:11:37.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.112+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f514c0ff670 con 0x7f514c103a80 2024-09-21T04:11:37.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.113+0000 7f5148ff9640 1 -- 172.21.15.110:0/3473937090 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+187 (secure 0 0 0) 0x7f51400597f0 con 0x7f514c103a80 2024-09-21T04:11:37.116 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 msgr2=0x7f513406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 0x7f513406f110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c005e60 comp rx=0 tx=0).stop 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 msgr2=0x7f514c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f514000d6f0 tx=0x7f514000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 shutdown_connections 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f513406cc50 0x7f513406f110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514c103a80 0x7f514c198350 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 --2- 172.21.15.110:0/3473937090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ff230 0x7f514c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 >> 172.21.15.110:0/3473937090 conn(0x7f514c0f9ef0 msgr2=0x7f514c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:37.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 shutdown_connections 2024-09-21T04:11:37.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:37.115+0000 7f5151f3b640 1 -- 172.21.15.110:0/3473937090 wait complete. 2024-09-21T04:11:37.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:37 smithi110 ceph-mon[27723]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:37 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3473937090' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-21T04:11:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:37 smithi148 ceph-mon[34957]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:37 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3473937090' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-21T04:11:37.968 INFO:teuthology.orchestra.run.smithi110.stdout:[{"version":43,"timestamp":"2024-09-21T04:11:20.966595+0000","name":"","changes":[{"name":"osd/host:smithi148/osd_memory_target","previous_value":"7818243072","new_value":"5212162048"}]}] 2024-09-21T04:11:37.968 INFO:tasks.ceph_manager:config epoch is 43 2024-09-21T04:11:37.968 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T04:11:37.968 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T04:11:37.969 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T04:11:38.168 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:39 smithi148 ceph-mon[34957]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:39.546 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:39 smithi110 ceph-mon[27723]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- 172.21.15.110:0/3340965137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c1049a0 msgr2=0x7f472c104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 --2- 172.21.15.110:0/3340965137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c1049a0 0x7f472c104d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f471c009880 tx=0x7f471c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- 172.21.15.110:0/3340965137 shutdown_connections 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 --2- 172.21.15.110:0/3340965137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c0fe9a0 0x7f472c0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 --2- 172.21.15.110:0/3340965137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c1049a0 0x7f472c104d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- 172.21.15.110:0/3340965137 >> 172.21.15.110:0/3340965137 conn(0x7f472c0758c0 msgr2=0x7f472c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- 172.21.15.110:0/3340965137 shutdown_connections 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- 172.21.15.110:0/3340965137 wait complete. 2024-09-21T04:11:39.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 Processor -- start 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.865+0000 7f473208a640 1 -- start start 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f473208a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f473208a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c1049a0 0x7f472c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f473208a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c19c520 con 0x7f472c0fe9a0 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f473208a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c195ab0 con 0x7f472c1049a0 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48962/0 (socket says 172.21.15.110:48962) 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 -- 172.21.15.110:0/1113346499 learned_addr learned my addr 172.21.15.110:0/1113346499 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f4722ffd640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c1049a0 0x7f472c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 -- 172.21.15.110:0/1113346499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c1049a0 msgr2=0x7f472c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c1049a0 0x7f472c19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 -- 172.21.15.110:0/1113346499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c009560 con 0x7f472c0fe9a0 2024-09-21T04:11:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.866+0000 7f472b7fe640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f471c02a9c0 tx=0x7f471c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.867+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471c004aa0 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.867+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f471c004c00 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.867+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c195d30 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.867+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471c003a20 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.867+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c196220 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.868+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c1000e0 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.868+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f471c031c90 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.869+0000 7f47297fa640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 0x7f471806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.869+0000 7f4722ffd640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 0x7f471806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.869+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f471c0acf40 con 0x7f472c0fe9a0 2024-09-21T04:11:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.869+0000 7f4722ffd640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 0x7f471806f110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f472c196f20 tx=0x7f4714009110 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:39.873+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f471c0e0e20 con 0x7f472c0fe9a0 2024-09-21T04:11:40.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.079+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f472c0fee80 con 0x7f472c0fe9a0 2024-09-21T04:11:40.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.082+0000 7f47297fa640 1 -- 172.21.15.110:0/1113346499 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174020 (secure 0 0 0) 0x7f471c0de7e0 con 0x7f472c0fe9a0 2024-09-21T04:11:40.087 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:40.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 msgr2=0x7f471806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 0x7f471806f110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f472c196f20 tx=0x7f4714009110 comp rx=0 tx=0).stop 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 msgr2=0x7f472c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f471c02a9c0 tx=0x7f471c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 shutdown_connections 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f472c1049a0 0x7f472c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f471806cc50 0x7f471806f110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 --2- 172.21.15.110:0/1113346499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472c0fe9a0 0x7f472c19b9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 >> 172.21.15.110:0/1113346499 conn(0x7f472c0758c0 msgr2=0x7f472c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 shutdown_connections 2024-09-21T04:11:40.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:40.085+0000 7f473208a640 1 -- 172.21.15.110:0/1113346499 wait complete. 2024-09-21T04:11:40.443 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:40 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1113346499' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-21T04:11:40.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:40 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1113346499' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-21T04:11:40.765 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"flags":0,"active_gid":14213,"active_name":"smithi110.axmqzb","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1362067387},{"type":"v1","addr":"172.21.15.110:6801","nonce":1362067387}]},"active_addr":"172.21.15.110:6801/1362067387","active_change":"2024-09-21T04:07:18.131768+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14234,"name":"smithi148.zjrahi","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.110:8443/","prometheus":"http://172.21.15.110:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2497023553}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":521148859}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1940237520}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":278566480}]}]} 2024-09-21T04:11:40.770 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T04:11:40.770 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T04:11:40.770 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:11:40.969 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:41 smithi110 ceph-mon[27723]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:41 smithi148 ceph-mon[34957]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 -- 172.21.15.110:0/2155610903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 msgr2=0x7f51a8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 --2- 172.21.15.110:0/2155610903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a8105fd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5198009880 tx=0x7f519802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 -- 172.21.15.110:0/2155610903 shutdown_connections 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 --2- 172.21.15.110:0/2155610903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 0x7f51a80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 --2- 172.21.15.110:0/2155610903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a8105fd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 -- 172.21.15.110:0/2155610903 >> 172.21.15.110:0/2155610903 conn(0x7f51a80fb840 msgr2=0x7f51a80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 -- 172.21.15.110:0/2155610903 shutdown_connections 2024-09-21T04:11:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.655+0000 7f51ada12640 1 -- 172.21.15.110:0/2155610903 wait complete. 2024-09-21T04:11:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 Processor -- start 2024-09-21T04:11:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 -- start start 2024-09-21T04:11:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 0x7f51a819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819c400 con 0x7f51a8105bf0 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51ada12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8195990 con 0x7f51a80ffb40 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53210/0 (socket says 172.21.15.110:53210) 2024-09-21T04:11:42.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 -- 172.21.15.110:0/3386115653 learned_addr learned my addr 172.21.15.110:0/3386115653 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f51a6ffd640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 0x7f51a819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 -- 172.21.15.110:0/3386115653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 msgr2=0x7f51a819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 0x7f51a819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 -- 172.21.15.110:0/3386115653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198009560 con 0x7f51a8105bf0 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.656+0000 7f519ffff640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f519000b420 tx=0x7f519000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.657+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5190011800 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.657+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5190011e20 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.657+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f519000f2e0 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.657+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a8195c70 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.657+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a8196240 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.658+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a81011f0 con 0x7f51a8105bf0 2024-09-21T04:11:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.658+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5190010380 con 0x7f51a8105bf0 2024-09-21T04:11:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.659+0000 7f51aca10640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 0x7f519406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.659+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f519008b7e0 con 0x7f51a8105bf0 2024-09-21T04:11:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.659+0000 7f51a6ffd640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 0x7f519406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.659+0000 7f51a6ffd640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 0x7f519406f110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.663+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5190055f70 con 0x7f51a8105bf0 2024-09-21T04:11:42.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.830+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51a8100020 con 0x7f51a8105bf0 2024-09-21T04:11:42.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.831+0000 7f51aca10640 1 -- 172.21.15.110:0/3386115653 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11324 (secure 0 0 0) 0x7f5190059590 con 0x7f51a8105bf0 2024-09-21T04:11:42.834 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:42.834 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","created":"2024-09-21T04:05:22.443593+0000","modified":"2024-09-21T04:11:22.087199+0000","last_up_change":"2024-09-21T04:11:20.850929+0000","last_in_change":"2024-09-21T04:11:01.869822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:10:12.442999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"373c3716-ec2e-4d0a-9eb8-c192e38b47ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6803","nonce":3868599904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6805","nonce":3868599904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6809","nonce":3868599904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6807","nonce":3868599904}]},"public_addr":"172.21.15.110:6803/3868599904","cluster_addr":"172.21.15.110:6805/3868599904","heartbeat_back_addr":"172.21.15.110:6809/3868599904","heartbeat_front_addr":"172.21.15.110:6807/3868599904","state":["exists","up"]},{"osd":1,"uuid":"a901efc6-b81c-4487-8056-0842cbb0dea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6811","nonce":18252912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6813","nonce":18252912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6817","nonce":18252912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6815","nonce":18252912}]},"public_addr":"172.21.15.110:6811/18252912","cluster_addr":"172.21.15.110:6813/18252912","heartbeat_back_addr":"172.21.15.110:6817/18252912","heartbeat_front_addr":"172.21.15.110:6815/18252912","state":["exists","up"]},{"osd":2,"uuid":"4ef1cc98-8f37-4031-ac9e-9bf871256b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6819","nonce":3408208533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6821","nonce":3408208533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6825","nonce":3408208533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6823","nonce":3408208533}]},"public_addr":"172.21.15.110:6819/3408208533","cluster_addr":"172.21.15.110:6821/3408208533","heartbeat_back_addr":"172.21.15.110:6825/3408208533","heartbeat_front_addr":"172.21.15.110:6823/3408208533","state":["exists","up"]},{"osd":3,"uuid":"80377dc2-d4fc-4bb6-8ae8-f295b4d9b774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6801","nonce":76623163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6803","nonce":76623163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6807","nonce":76623163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6805","nonce":76623163}]},"public_addr":"172.21.15.148:6801/76623163","cluster_addr":"172.21.15.148:6803/76623163","heartbeat_back_addr":"172.21.15.148:6807/76623163","heartbeat_front_addr":"172.21.15.148:6805/76623163","state":["exists","up"]},{"osd":4,"uuid":"cfbc2e07-ddea-44e4-8d0f-df489240ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6809","nonce":564615118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6811","nonce":564615118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6815","nonce":564615118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6813","nonce":564615118}]},"public_addr":"172.21.15.148:6809/564615118","cluster_addr":"172.21.15.148:6811/564615118","heartbeat_back_addr":"172.21.15.148:6815/564615118","heartbeat_front_addr":"172.21.15.148:6813/564615118","state":["exists","up"]},{"osd":5,"uuid":"f5dd029b-19d1-4ce9-9377-cce1d2b70348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6817","nonce":958827228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6819","nonce":958827228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6823","nonce":958827228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6821","nonce":958827228}]},"public_addr":"172.21.15.148:6817/958827228","cluster_addr":"172.21.15.148:6819/958827228","heartbeat_back_addr":"172.21.15.148:6823/958827228","heartbeat_front_addr":"172.21.15.148:6821/958827228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:13.833321+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:41.103978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:08.627858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:30.608053+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:54.266275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:11:18.341884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6800/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:6801/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:6801/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2087016304":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/602456430":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2783964987":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/1589768070":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/598619296":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6801/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/1485849066":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/3142935389":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/2973624819":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/3973251001":"2024-09-22T04:07:18.131661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 msgr2=0x7f519406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 0x7f519406f110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198005a20 comp rx=0 tx=0).stop 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 msgr2=0x7f51a819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f519000b420 tx=0x7f519000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 shutdown_connections 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f519406cc50 0x7f519406f110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51a8105bf0 0x7f51a819bde0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 --2- 172.21.15.110:0/3386115653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a80ffb40 0x7f51a819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 >> 172.21.15.110:0/3386115653 conn(0x7f51a80fb840 msgr2=0x7f51a80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 shutdown_connections 2024-09-21T04:11:42.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:42.833+0000 7f51ada12640 1 -- 172.21.15.110:0/3386115653 wait complete. 2024-09-21T04:11:43.575 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:43 smithi110 ceph-mon[27723]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:43.576 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:43 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3386115653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:43.597 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T04:11:43.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:11:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:43 smithi148 ceph-mon[34957]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:43 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3386115653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:45.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 -- 172.21.15.110:0/1472084761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac0ffdf0 msgr2=0x7fc6ac1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 --2- 172.21.15.110:0/1472084761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac1001d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009880 tx=0x7fc69c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 -- 172.21.15.110:0/1472084761 shutdown_connections 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 --2- 172.21.15.110:0/1472084761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 --2- 172.21.15.110:0/1472084761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac1001d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 -- 172.21.15.110:0/1472084761 >> 172.21.15.110:0/1472084761 conn(0x7fc6ac0fb8b0 msgr2=0x7fc6ac0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 -- 172.21.15.110:0/1472084761 shutdown_connections 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.422+0000 7fc6b440a640 1 -- 172.21.15.110:0/1472084761 wait complete. 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 Processor -- start 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 -- start start 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac198970 con 0x7fc6ac1007a0 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b440a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac19c6e0 con 0x7fc6ac0ffdf0 2024-09-21T04:11:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53218/0 (socket says 172.21.15.110:53218) 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 -- 172.21.15.110:0/2863359327 learned_addr learned my addr 172.21.15.110:0/2863359327 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b21a6640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 -- 172.21.15.110:0/2863359327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac0ffdf0 msgr2=0x7fc6ac197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 -- 172.21.15.110:0/2863359327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c009560 con 0x7fc6ac1007a0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.423+0000 7fc6b19a5640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b420 tx=0x7fc6a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.424+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a8011800 con 0x7fc6ac1007a0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.424+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6a8011e20 con 0x7fc6ac1007a0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.424+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19c9c0 con 0x7fc6ac1007a0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.424+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a800f2e0 con 0x7fc6ac1007a0 2024-09-21T04:11:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.424+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac19cf10 con 0x7fc6ac1007a0 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.425+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac1012d0 con 0x7fc6ac1007a0 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.425+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc6a8010380 con 0x7fc6ac1007a0 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.426+0000 7fc6a37fe640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 0x7fc69806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.426+0000 7fc6b21a6640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 0x7fc69806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.426+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc6a808b7e0 con 0x7fc6ac1007a0 2024-09-21T04:11:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.426+0000 7fc6b21a6640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 0x7fc69806f110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.430+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc6a8055f70 con 0x7fc6ac1007a0 2024-09-21T04:11:45.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.590+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6ac1001d0 con 0x7fc6ac1007a0 2024-09-21T04:11:45.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:45 smithi110 ceph-mon[27723]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:45.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.591+0000 7fc6a37fe640 1 -- 172.21.15.110:0/2863359327 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11324 (secure 0 0 0) 0x7fc6a8059590 con 0x7fc6ac1007a0 2024-09-21T04:11:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:11:45.595 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"77422328-77ce-11ef-baf5-efdc52797490","created":"2024-09-21T04:05:22.443593+0000","modified":"2024-09-21T04:11:22.087199+0000","last_up_change":"2024-09-21T04:11:20.850929+0000","last_in_change":"2024-09-21T04:11:01.869822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:10:12.442999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"373c3716-ec2e-4d0a-9eb8-c192e38b47ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6803","nonce":3868599904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6805","nonce":3868599904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6809","nonce":3868599904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3868599904},{"type":"v1","addr":"172.21.15.110:6807","nonce":3868599904}]},"public_addr":"172.21.15.110:6803/3868599904","cluster_addr":"172.21.15.110:6805/3868599904","heartbeat_back_addr":"172.21.15.110:6809/3868599904","heartbeat_front_addr":"172.21.15.110:6807/3868599904","state":["exists","up"]},{"osd":1,"uuid":"a901efc6-b81c-4487-8056-0842cbb0dea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6811","nonce":18252912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6813","nonce":18252912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6817","nonce":18252912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":18252912},{"type":"v1","addr":"172.21.15.110:6815","nonce":18252912}]},"public_addr":"172.21.15.110:6811/18252912","cluster_addr":"172.21.15.110:6813/18252912","heartbeat_back_addr":"172.21.15.110:6817/18252912","heartbeat_front_addr":"172.21.15.110:6815/18252912","state":["exists","up"]},{"osd":2,"uuid":"4ef1cc98-8f37-4031-ac9e-9bf871256b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6819","nonce":3408208533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6821","nonce":3408208533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6825","nonce":3408208533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3408208533},{"type":"v1","addr":"172.21.15.110:6823","nonce":3408208533}]},"public_addr":"172.21.15.110:6819/3408208533","cluster_addr":"172.21.15.110:6821/3408208533","heartbeat_back_addr":"172.21.15.110:6825/3408208533","heartbeat_front_addr":"172.21.15.110:6823/3408208533","state":["exists","up"]},{"osd":3,"uuid":"80377dc2-d4fc-4bb6-8ae8-f295b4d9b774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6801","nonce":76623163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6803","nonce":76623163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6807","nonce":76623163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":76623163},{"type":"v1","addr":"172.21.15.148:6805","nonce":76623163}]},"public_addr":"172.21.15.148:6801/76623163","cluster_addr":"172.21.15.148:6803/76623163","heartbeat_back_addr":"172.21.15.148:6807/76623163","heartbeat_front_addr":"172.21.15.148:6805/76623163","state":["exists","up"]},{"osd":4,"uuid":"cfbc2e07-ddea-44e4-8d0f-df489240ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6809","nonce":564615118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6811","nonce":564615118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6815","nonce":564615118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":564615118},{"type":"v1","addr":"172.21.15.148:6813","nonce":564615118}]},"public_addr":"172.21.15.148:6809/564615118","cluster_addr":"172.21.15.148:6811/564615118","heartbeat_back_addr":"172.21.15.148:6815/564615118","heartbeat_front_addr":"172.21.15.148:6813/564615118","state":["exists","up"]},{"osd":5,"uuid":"f5dd029b-19d1-4ce9-9377-cce1d2b70348","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6817","nonce":958827228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6819","nonce":958827228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6823","nonce":958827228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":958827228},{"type":"v1","addr":"172.21.15.148:6821","nonce":958827228}]},"public_addr":"172.21.15.148:6817/958827228","cluster_addr":"172.21.15.148:6819/958827228","heartbeat_back_addr":"172.21.15.148:6823/958827228","heartbeat_front_addr":"172.21.15.148:6821/958827228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:13.833321+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:09:41.103978+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:08.627858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:30.608053+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:10:54.266275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:11:18.341884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6800/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:6801/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:6801/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/833735978":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2087016304":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/602456430":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/2783964987":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/1589768070":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/598619296":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6800/1919392720":"2024-09-22T04:06:13.993411+0000","172.21.15.110:6801/539492599":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/1485849066":"2024-09-22T04:06:13.993411+0000","172.21.15.110:0/3142935389":"2024-09-22T04:07:18.131661+0000","172.21.15.110:0/2973624819":"2024-09-22T04:05:43.998468+0000","172.21.15.110:0/3973251001":"2024-09-22T04:07:18.131661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:11:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 msgr2=0x7fc69806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 0x7fc69806f110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c005a20 comp rx=0 tx=0).stop 2024-09-21T04:11:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 msgr2=0x7fc6ac1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b420 tx=0x7fc6a800b8e0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 shutdown_connections 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc69806cc50 0x7fc69806f110 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6ac1007a0 0x7fc6ac1982e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 --2- 172.21.15.110:0/2863359327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac0ffdf0 0x7fc6ac197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 >> 172.21.15.110:0/2863359327 conn(0x7fc6ac0fb8b0 msgr2=0x7fc6ac0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.593+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 shutdown_connections 2024-09-21T04:11:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:45.594+0000 7fc6b440a640 1 -- 172.21.15.110:0/2863359327 wait complete. 2024-09-21T04:11:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:45 smithi148 ceph-mon[34957]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:46.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:46 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2863359327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:46.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T04:11:46.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T04:11:46.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T04:11:46.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T04:11:46.396 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T04:11:46.396 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T04:11:46.652 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:46.675 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:46 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2863359327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:11:46.718 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:47 smithi110 ceph-mon[27723]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:47.434 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:47 smithi148 ceph-mon[34957]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:48.192 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:48.193 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:11:49.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:49 smithi110 ceph-mon[27723]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:49 smithi148 ceph-mon[34957]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:50.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 -- 172.21.15.110:0/789019329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5044104a20 msgr2=0x7f5044104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 --2- 172.21.15.110:0/789019329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5044104a20 0x7f5044104e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5034009880 tx=0x7f503402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 -- 172.21.15.110:0/789019329 shutdown_connections 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 --2- 172.21.15.110:0/789019329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50440fea20 0x7f50440fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 --2- 172.21.15.110:0/789019329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5044104a20 0x7f5044104e00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 -- 172.21.15.110:0/789019329 >> 172.21.15.110:0/789019329 conn(0x7f50440fa720 msgr2=0x7f50440fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:50.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 -- 172.21.15.110:0/789019329 shutdown_connections 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.622+0000 7f5049815640 1 -- 172.21.15.110:0/789019329 wait complete. 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 Processor -- start 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 -- start start 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 0x7f5044198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:50.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5044198990 con 0x7f50440fea20 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5049815640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504419c700 con 0x7f5044104a20 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53236/0 (socket says 172.21.15.110:53236) 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 -- 172.21.15.110:0/3760460125 learned_addr learned my addr 172.21.15.110:0/3760460125 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f503a7fc640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 0x7f5044198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 -- 172.21.15.110:0/3760460125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 msgr2=0x7f5044198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 0x7f5044198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f5042ffd640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034009560 con 0x7f50440fea20 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.623+0000 7f503a7fc640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 0x7f5044198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:50.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5042ffd640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f503402a9c0 tx=0x7f50340055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5034004aa0 con 0x7f50440fea20 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5034004c00 con 0x7f50440fea20 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5034003ab0 con 0x7f50440fea20 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504419c980 con 0x7f50440fea20 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.624+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504419ce70 con 0x7f50440fea20 2024-09-21T04:11:50.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.625+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f504410c010 con 0x7f50440fea20 2024-09-21T04:11:50.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.625+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5034031d20 con 0x7f50440fea20 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f5048813640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 0x7f503006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f503a7fc640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 0x7f503006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f50340acf20 con 0x7f50440fea20 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f5048813640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 0x7f5030074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50300754b0 con 0x7f50300729e0 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50340af5f0 con 0x7f50440fea20 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f50437fe640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 0x7f5030074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.626+0000 7f503a7fc640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 0x7f503006f110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f502c006fd0 tx=0x7f502c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:50.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.627+0000 7f50437fe640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 0x7f5030074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:50.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.627+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== osd.0 v2:172.21.15.110:6802/3868599904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f50300754b0 con 0x7f50300729e0 2024-09-21T04:11:50.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.640+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 --> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5044100160 con 0x7f50300729e0 2024-09-21T04:11:50.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.640+0000 7f5048813640 1 -- 172.21.15.110:0/3760460125 <== osd.0 v2:172.21.15.110:6802/3868599904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5044100160 con 0x7f50300729e0 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 msgr2=0x7f5030074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 0x7f5030074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 msgr2=0x7f503006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 0x7f503006f110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f502c006fd0 tx=0x7f502c008040 comp rx=0 tx=0).stop 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 msgr2=0x7f5044197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:50.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f503402a9c0 tx=0x7f50340055a0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 shutdown_connections 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5044104a20 0x7f5044198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6802/3868599904,v1:172.21.15.110:6803/3868599904] conn(0x7f50300729e0 0x7f5030074e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f503006cc50 0x7f503006f110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 --2- 172.21.15.110:0/3760460125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50440fea20 0x7f5044197dc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 >> 172.21.15.110:0/3760460125 conn(0x7f50440fa720 msgr2=0x7f50440fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 shutdown_connections 2024-09-21T04:11:50.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:50.641+0000 7f5049815640 1 -- 172.21.15.110:0/3760460125 wait complete. 2024-09-21T04:11:51.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:51 smithi110 ceph-mon[27723]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:51.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:51 smithi148 ceph-mon[34957]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:53.492 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:53 smithi148 ceph-mon[34957]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:53.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:53 smithi110 ceph-mon[27723]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1592510538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8100d60 msgr2=0x7ff8c8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1592510538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8100d60 0x7ff8c8101140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009880 tx=0x7ff8b802a530 comp rx=0 tx=0).stop 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1592510538 shutdown_connections 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1592510538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8101680 0x7ff8c8109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1592510538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8100d60 0x7ff8c8101140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1592510538 >> 172.21.15.110:0/1592510538 conn(0x7ff8c80fa780 msgr2=0x7ff8c80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1592510538 shutdown_connections 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.174+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1592510538 wait complete. 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 Processor -- start 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 -- start start 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8100d60 0x7ff8c80fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80ff440 con 0x7ff8c8101680 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8ce4ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80ff5b0 con 0x7ff8c8100d60 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44144/0 (socket says 172.21.15.110:44144) 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 -- 172.21.15.110:0/1354726699 learned_addr learned my addr 172.21.15.110:0/1354726699 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8c7fff640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8100d60 0x7ff8c80fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:54.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 -- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8100d60 msgr2=0x7ff8c80fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8100d60 0x7ff8c80fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.175+0000 7ff8bffff640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009560 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8bffff640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d6d0 tx=0x7ff8b000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b00051f0 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b0009ce0 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81a1d30 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b0004db0 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c81a2090 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.176+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8c810d090 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.177+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff8b0009820 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c5ffb640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 0x7ff8b406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff8b008bd90 con 0x7ff8c8101680 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c7fff640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 0x7ff8b406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:54.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c5ffb640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 0x7ff8b4074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b4075450 con 0x7ff8b4072980 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b008c150 con 0x7ff8c8101680 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8cca49640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 0x7ff8b4074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8c7fff640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 0x7ff8b406f0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009230 tx=0x7ff8b80058d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.178+0000 7ff8cca49640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 0x7ff8b4074da0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:54.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.179+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== osd.5 v2:172.21.15.148:6816/958827228 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff8b4075450 con 0x7ff8b4072980 2024-09-21T04:11:54.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.195+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 --> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8c8060b70 con 0x7ff8b4072980 2024-09-21T04:11:54.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8c5ffb640 1 -- 172.21.15.110:0/1354726699 <== osd.5 v2:172.21.15.148:6816/958827228 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff8c8060b70 con 0x7ff8b4072980 2024-09-21T04:11:54.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 msgr2=0x7ff8b4074da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:54.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 0x7ff8b4074da0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 msgr2=0x7ff8b406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 0x7ff8b406f0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009230 tx=0x7ff8b80058d0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 msgr2=0x7ff8c80fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d6d0 tx=0x7ff8b000db90 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 shutdown_connections 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:6816/958827228,v1:172.21.15.148:6817/958827228] conn(0x7ff8b4072980 0x7ff8b4074da0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff8b406cbf0 0x7ff8b406f0b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8c8101680 0x7ff8c80fef00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 --2- 172.21.15.110:0/1354726699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8c8100d60 0x7ff8c80fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 >> 172.21.15.110:0/1354726699 conn(0x7ff8c80fa780 msgr2=0x7ff8c8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.196+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 shutdown_connections 2024-09-21T04:11:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:54.197+0000 7ff8ce4ac640 1 -- 172.21.15.110:0/1354726699 wait complete. 2024-09-21T04:11:55.027 INFO:teuthology.orchestra.run.smithi110.stdout:38654705698 2024-09-21T04:11:55.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T04:11:55.330 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:55 smithi110 ceph-mon[27723]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- 172.21.15.110:0/3429754201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 msgr2=0x7f759c105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 --2- 172.21.15.110:0/3429754201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c105f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f758c009880 tx=0x7f758c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- 172.21.15.110:0/3429754201 shutdown_connections 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 --2- 172.21.15.110:0/3429754201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 --2- 172.21.15.110:0/3429754201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c105f70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- 172.21.15.110:0/3429754201 >> 172.21.15.110:0/3429754201 conn(0x7f759c0fb7e0 msgr2=0x7f759c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- 172.21.15.110:0/3429754201 shutdown_connections 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- 172.21.15.110:0/3429754201 wait complete. 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 Processor -- start 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.327+0000 7f75a188c640 1 -- start start 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c073230 con 0x7f759c105b90 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0733a0 con 0x7f759c0ffb00 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44164/0 (socket says 172.21.15.110:44164) 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 -- 172.21.15.110:0/1685163079 learned_addr learned my addr 172.21.15.110:0/1685163079 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759affd640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 -- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 msgr2=0x7f759c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c009560 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759affd640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f759a7fc640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f759000c8f0 tx=0x7f759000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7590007b80 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c1a1d50 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75900054e0 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759000f450 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.328+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c1a20b0 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.329+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f759c10cfd0 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.330+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7590004f60 con 0x7f759c105b90 2024-09-21T04:11:55.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.330+0000 7f75a088a640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 0x7f758406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f759008bd90 con 0x7f759c105b90 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f759affd640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 0x7f758406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f75a088a640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 0x7f7584074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75840754b0 con 0x7f75840729e0 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7590091470 con 0x7f759c105b90 2024-09-21T04:11:55.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f759b7fe640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 0x7f7584074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f759affd640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 0x7f758406f110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.331+0000 7f759b7fe640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 0x7f7584074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.332+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== osd.4 v2:172.21.15.148:6808/564615118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f75840754b0 con 0x7f75840729e0 2024-09-21T04:11:55.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.345+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 --> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f759c060b70 con 0x7f75840729e0 2024-09-21T04:11:55.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a088a640 1 -- 172.21.15.110:0/1685163079 <== osd.4 v2:172.21.15.148:6808/564615118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f759c060b70 con 0x7f75840729e0 2024-09-21T04:11:55.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 msgr2=0x7f7584074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 0x7f7584074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 msgr2=0x7f758406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 0x7f758406f110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c005a20 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 msgr2=0x7f759c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f759000c8f0 tx=0x7f759000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 shutdown_connections 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:6808/564615118,v1:172.21.15.148:6809/564615118] conn(0x7f75840729e0 0x7f7584074e00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f758406cc50 0x7f758406f110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c105b90 0x7f759c072cf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.346+0000 7f75a188c640 1 --2- 172.21.15.110:0/1685163079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffb00 0x7f759c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.347+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 >> 172.21.15.110:0/1685163079 conn(0x7f759c0fb7e0 msgr2=0x7f759c0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:55.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.347+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 shutdown_connections 2024-09-21T04:11:55.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.347+0000 7f75a188c640 1 -- 172.21.15.110:0/1685163079 wait complete. 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- 172.21.15.110:0/3568963230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 msgr2=0x7f1f880fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/3568963230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f880fee80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009880 tx=0x7f1f7802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- 172.21.15.110:0/3568963230 shutdown_connections 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/3568963230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f880fee80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/3568963230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 0x7f1f88104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- 172.21.15.110:0/3568963230 >> 172.21.15.110:0/3568963230 conn(0x7f1f880fa720 msgr2=0x7f1f880fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- 172.21.15.110:0/3568963230 shutdown_connections 2024-09-21T04:11:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- 172.21.15.110:0/3568963230 wait complete. 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 Processor -- start 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.544+0000 7f1f8f08c640 1 -- start start 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8f08c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8f08c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 0x7f1f8819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8f08c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819c390 con 0x7f1f880fea20 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8f08c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88195920 con 0x7f1f88104a20 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44180/0 (socket says 172.21.15.110:44180) 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 -- 172.21.15.110:0/2232439198 learned_addr learned my addr 172.21.15.110:0/2232439198 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f7ffff640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 0x7f1f8819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 -- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 msgr2=0x7f1f8819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 0x7f1f8819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009560 con 0x7f1f880fea20 2024-09-21T04:11:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8ce28640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400d6f0 tx=0x7f1f8400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f84009a30 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88195c00 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f840054e0 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.545+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f84010460 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.546+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f88196120 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.546+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f8810bfb0 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f1f84009b90 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 0x7f1f7006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1f8408ba10 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7ffff640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 0x7f1f7006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 0x7f1f70074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f700754b0 con 0x7f1f700729e0 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.547+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f840902a0 con 0x7f1f880fea20 2024-09-21T04:11:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.548+0000 7f1f8d629640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 0x7f1f70074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:55.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.548+0000 7f1f7ffff640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 0x7f1f7006f110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009230 tx=0x7f1f780058d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.548+0000 7f1f8d629640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 0x7f1f70074e00 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:55.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.549+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== osd.3 v2:172.21.15.148:6800/76623163 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1f700754b0 con 0x7f1f700729e0 2024-09-21T04:11:55.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.564+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 --> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f88196580 con 0x7f1f700729e0 2024-09-21T04:11:55.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f7dffb640 1 -- 172.21.15.110:0/2232439198 <== osd.3 v2:172.21.15.148:6800/76623163 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1f88196580 con 0x7f1f700729e0 2024-09-21T04:11:55.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 msgr2=0x7f1f70074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 0x7f1f70074e00 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 msgr2=0x7f1f7006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 0x7f1f7006f110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009230 tx=0x7f1f780058d0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 msgr2=0x7f1f8819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400d6f0 tx=0x7f1f8400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 shutdown_connections 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:6800/76623163,v1:172.21.15.148:6801/76623163] conn(0x7f1f700729e0 0x7f1f70074e00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f88104a20 0x7f1f8819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f7006cc50 0x7f1f7006f110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 --2- 172.21.15.110:0/2232439198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f880fea20 0x7f1f8819b830 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.565+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 >> 172.21.15.110:0/2232439198 conn(0x7f1f880fa720 msgr2=0x7f1f880688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.566+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 shutdown_connections 2024-09-21T04:11:55.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:55.566+0000 7f1f8f08c640 1 -- 172.21.15.110:0/2232439198 wait complete. 2024-09-21T04:11:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:55 smithi148 ceph-mon[34957]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:55.991 INFO:teuthology.orchestra.run.smithi110.stdout:167503724554 2024-09-21T04:11:55.991 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T04:11:57.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 -- 172.21.15.110:0/2445067004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb420100db0 msgr2=0x7fb420101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:57.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2445067004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb420100db0 0x7fb420101190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098b0 tx=0x7fb41c02a560 comp rx=0 tx=0).stop 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 -- 172.21.15.110:0/2445067004 shutdown_connections 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2445067004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4201016d0 0x7fb420109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2445067004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb420100db0 0x7fb420101190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 -- 172.21.15.110:0/2445067004 >> 172.21.15.110:0/2445067004 conn(0x7fb4200fa7b0 msgr2=0x7fb4200fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.039+0000 7fb427b9a640 1 -- 172.21.15.110:0/2445067004 shutdown_connections 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.040+0000 7fb427b9a640 1 -- 172.21.15.110:0/2445067004 wait complete. 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.040+0000 7fb427b9a640 1 Processor -- start 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.040+0000 7fb427b9a640 1 -- start start 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb420100db0 0x7fb42019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42019c470 con 0x7fb4201016d0 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb420195a00 con 0x7fb420100db0 2024-09-21T04:11:57.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44196/0 (socket says 172.21.15.110:44196) 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 -- 172.21.15.110:0/2697624811 learned_addr learned my addr 172.21.15.110:0/2697624811 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425936640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb420100db0 0x7fb42019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 -- 172.21.15.110:0/2697624811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb420100db0 msgr2=0x7fb42019b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb420100db0 0x7fb42019b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009560 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb425135640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c8f0 tx=0x7fb41000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb410007b80 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4100054e0 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb420195ce0 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41000f450 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.041+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb420196230 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.042+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb42010d0c0 con 0x7fb4201016d0 2024-09-21T04:11:57.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb410004f60 con 0x7fb4201016d0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 0x7fb40c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb41008bd70 con 0x7fb4201016d0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb425936640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 0x7fb40c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 0x7fb40c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb40c0754b0 con 0x7fb40c0729e0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb410091450 con 0x7fb4201016d0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.043+0000 7fb426137640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 0x7fb40c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.044+0000 7fb425936640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 0x7fb40c06f110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009230 tx=0x7fb41c005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.044+0000 7fb426137640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 0x7fb40c074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:57.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.044+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== osd.2 v2:172.21.15.110:6818/3408208533 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb40c0754b0 con 0x7fb40c0729e0 2024-09-21T04:11:57.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.057+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 --> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb420196750 con 0x7fb40c0729e0 2024-09-21T04:11:57.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.057+0000 7fb416ffd640 1 -- 172.21.15.110:0/2697624811 <== osd.2 v2:172.21.15.110:6818/3408208533 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb420196750 con 0x7fb40c0729e0 2024-09-21T04:11:57.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 msgr2=0x7fb40c074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:57.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 0x7fb40c074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 msgr2=0x7fb40c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 0x7fb40c06f110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009230 tx=0x7fb41c005a20 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 msgr2=0x7fb42019be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c8f0 tx=0x7fb41000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 shutdown_connections 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6818/3408208533,v1:172.21.15.110:6819/3408208533] conn(0x7fb40c0729e0 0x7fb40c074e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb40c06cc50 0x7fb40c06f110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4201016d0 0x7fb42019be50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 --2- 172.21.15.110:0/2697624811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb420100db0 0x7fb42019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 >> 172.21.15.110:0/2697624811 conn(0x7fb4200fa7b0 msgr2=0x7fb420102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 shutdown_connections 2024-09-21T04:11:57.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:57.058+0000 7fb427b9a640 1 -- 172.21.15.110:0/2697624811 wait complete. 2024-09-21T04:11:57.657 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:57 smithi110 ceph-mon[27723]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:57.679 INFO:teuthology.orchestra.run.smithi110.stdout:81604378649 2024-09-21T04:11:57.680 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T04:11:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:57 smithi148 ceph-mon[34957]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:58.066 INFO:teuthology.orchestra.run.smithi110.stdout:111669149715 2024-09-21T04:11:58.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T04:11:58.698 INFO:teuthology.orchestra.run.smithi110.stdout:146028888079 2024-09-21T04:11:58.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T04:11:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 -- 172.21.15.110:0/1594339779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffb40 msgr2=0x7fcf800fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 --2- 172.21.15.110:0/1594339779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffb40 0x7fcf800fffa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009880 tx=0x7fcf7002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 -- 172.21.15.110:0/1594339779 shutdown_connections 2024-09-21T04:11:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 --2- 172.21.15.110:0/1594339779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffb40 0x7fcf800fffa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 --2- 172.21.15.110:0/1594339779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf80105bf0 0x7fcf80105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 -- 172.21.15.110:0/1594339779 >> 172.21.15.110:0/1594339779 conn(0x7fcf800fb840 msgr2=0x7fcf800fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 -- 172.21.15.110:0/1594339779 shutdown_connections 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.119+0000 7fcf879e9640 1 -- 172.21.15.110:0/1594339779 wait complete. 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 Processor -- start 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 -- start start 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 0x7fcf800727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80073230 con 0x7fcf80105bf0 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf879e9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf800733a0 con 0x7fcf800ffb40 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44202/0 (socket says 172.21.15.110:44202) 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 -- 172.21.15.110:0/4007249888 learned_addr learned my addr 172.21.15.110:0/4007249888 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:11:59.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf85785640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 0x7fcf800727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 -- 172.21.15.110:0/4007249888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 msgr2=0x7fcf800727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 0x7fcf800727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf84f84640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70009560 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.120+0000 7fcf85785640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 0x7fcf800727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf84f84640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf7002a9c0 tx=0x7fcf700055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf70004aa0 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf70004c00 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf80073620 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf70003980 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.121+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801a1f10 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.122+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf8010d0b0 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.122+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fcf70003bf0 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf767fc640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 0x7fcf6c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcf700ace30 con 0x7fcf80105bf0 2024-09-21T04:11:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf85785640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 0x7fcf6c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf767fc640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 0x7fcf6c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf6c0754b0 con 0x7fcf6c0729e0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf700b2510 con 0x7fcf80105bf0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf85f86640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 0x7fcf6c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf85785640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 0x7fcf6c06f110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b40 tx=0x7fcf7c005e50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.123+0000 7fcf85f86640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 0x7fcf6c074e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.124+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== osd.1 v2:172.21.15.110:6810/18252912 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fcf6c0754b0 con 0x7fcf6c0729e0 2024-09-21T04:11:59.134 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:59.135 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:59.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.147+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 --> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf80060b70 con 0x7fcf6c0729e0 2024-09-21T04:11:59.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.147+0000 7fcf767fc640 1 -- 172.21.15.110:0/4007249888 <== osd.1 v2:172.21.15.110:6810/18252912 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf80060b70 con 0x7fcf6c0729e0 2024-09-21T04:11:59.150 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 msgr2=0x7fcf6c074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 0x7fcf6c074e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 msgr2=0x7fcf6c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 0x7fcf6c06f110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b40 tx=0x7fcf7c005e50 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 msgr2=0x7fcf80072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.148+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf7002a9c0 tx=0x7fcf700055a0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 shutdown_connections 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6810/18252912,v1:172.21.15.110:6811/18252912] conn(0x7fcf6c0729e0 0x7fcf6c074e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcf6c06cc50 0x7fcf6c06f110 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf80105bf0 0x7fcf80072cf0 secure :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf7002a9c0 tx=0x7fcf700055a0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 --2- 172.21.15.110:0/4007249888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffb40 0x7fcf800727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:59.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 >> 172.21.15.110:0/4007249888 conn(0x7fcf800fb840 msgr2=0x7fcf800fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:59.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 shutdown_connections 2024-09-21T04:11:59.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:11:59.149+0000 7fcf879e9640 1 -- 172.21.15.110:0/4007249888 wait complete. 2024-09-21T04:11:59.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:11:59 smithi110 ceph-mon[27723]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:11:59 smithi148 ceph-mon[34957]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:11:59.868 INFO:teuthology.orchestra.run.smithi110.stdout:60129542175 2024-09-21T04:11:59.869 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T04:11:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:11:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:01.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:01 smithi110 ceph-mon[27723]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:01.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:01 smithi148 ceph-mon[34957]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:03.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:03 smithi110 ceph-mon[27723]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:03.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:03 smithi148 ceph-mon[34957]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:05.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:05 smithi110 ceph-mon[27723]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:05.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:05 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/664144356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-21T04:12:05.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:05 smithi148 ceph-mon[34957]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:05.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:05 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/664144356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-21T04:12:05.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/2135627869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fee30 msgr2=0x7ff7c0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:05.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/2135627869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fee30 0x7ff7c0102e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009880 tx=0x7ff7b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/2135627869 shutdown_connections 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/2135627869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fee30 0x7ff7c0102e80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/2135627869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fe480 0x7ff7c00fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/2135627869 >> 172.21.15.110:0/2135627869 conn(0x7ff7c00f9ec0 msgr2=0x7ff7c00fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/2135627869 shutdown_connections 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.750+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/2135627869 wait complete. 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 Processor -- start 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 -- start start 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fee30 0x7ff7c019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c019b300 con 0x7ff7c00fe480 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.751+0000 7ff7c5bd4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0194890 con 0x7ff7c00fee30 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40210/0 (socket says 172.21.15.110:40210) 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 -- 172.21.15.110:0/664144356 learned_addr learned my addr 172.21.15.110:0/664144356 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stdout:81604378650 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7beffd640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fee30 0x7ff7c019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 -- 172.21.15.110:0/664144356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fee30 msgr2=0x7ff7c019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fee30 0x7ff7c019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 -- 172.21.15.110:0/664144356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4009560 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bf7fe640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000d6f0 tx=0x7ff7b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b0009a30 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c0194b70 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7b00054e0 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b0010460 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.752+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c0195090 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.753+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c00ff960 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.754+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7ff7b0009b90 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.754+0000 7ff7bcff9640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 0x7ff7a806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.754+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff7b008ba10 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.754+0000 7ff7beffd640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 0x7ff7a806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.755+0000 7ff7beffd640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 0x7ff7a806f110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009230 tx=0x7ff7b40058d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.758+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7b00561d0 con 0x7ff7c00fe480 2024-09-21T04:12:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.926+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff7c0106790 con 0x7ff7c00fe480 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.926+0000 7ff7bcff9640 1 -- 172.21.15.110:0/664144356 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7b0020090 con 0x7ff7c00fe480 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 msgr2=0x7ff7a806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 0x7ff7a806f110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009230 tx=0x7ff7b40058d0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 msgr2=0x7ff7c019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000d6f0 tx=0x7ff7b000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 shutdown_connections 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7c00fee30 0x7ff7c019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7ff7a806cc50 0x7ff7a806f110 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 --2- 172.21.15.110:0/664144356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7c00fe480 0x7ff7c019a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.928+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 >> 172.21.15.110:0/664144356 conn(0x7ff7c00f9ec0 msgr2=0x7ff7c00fb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.929+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 shutdown_connections 2024-09-21T04:12:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:04.929+0000 7ff7c5bd4640 1 -- 172.21.15.110:0/664144356 wait complete. 2024-09-21T04:12:06.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378650 for osd.2 2024-09-21T04:12:06.258 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- 172.21.15.110:0/2175649878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 msgr2=0x7f0274109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 --2- 172.21.15.110:0/2175649878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f0274109ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f025c009880 tx=0x7f025c02a530 comp rx=0 tx=0).stop 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- 172.21.15.110:0/2175649878 shutdown_connections 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 --2- 172.21.15.110:0/2175649878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f0274109ae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 --2- 172.21.15.110:0/2175649878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f02741010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- 172.21.15.110:0/2175649878 >> 172.21.15.110:0/2175649878 conn(0x7f02740fa730 msgr2=0x7f02740fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- 172.21.15.110:0/2175649878 shutdown_connections 2024-09-21T04:12:06.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- 172.21.15.110:0/2175649878 wait complete. 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 Processor -- start 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- start start 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f027419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274195a20 con 0x7f02741015f0 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.422+0000 7f027aecd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274195b60 con 0x7f0274100cd0 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40222/0 (socket says 172.21.15.110:40222) 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 -- 172.21.15.110:0/3780725447 learned_addr learned my addr 172.21.15.110:0/3780725447 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0278c69640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f027419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 -- 172.21.15.110:0/3780725447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 msgr2=0x7f027419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f027419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 -- 172.21.15.110:0/3780725447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009560 con 0x7f02741015f0 2024-09-21T04:12:06.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0278c69640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f027419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0273fff640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f025c02aa50 tx=0x7f025c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c004aa0 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f025c004c00 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274195de0 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c0039b0 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.423+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741962d0 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.424+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274068ce0 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.425+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f025c02ad50 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.425+0000 7f0271ffb640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 0x7f026006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.425+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f025c0acd90 con 0x7f02741015f0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.425+0000 7f0278c69640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 0x7f026006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.426+0000 7f0278c69640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 0x7f026006f110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0264009930 tx=0x7f0264008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:06.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.430+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f025c0774a0 con 0x7f02741015f0 2024-09-21T04:12:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.597+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0274105330 con 0x7f02741015f0 2024-09-21T04:12:06.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.597+0000 7f0271ffb640 1 -- 172.21.15.110:0/3780725447 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f025c07aac0 con 0x7f02741015f0 2024-09-21T04:12:06.600 INFO:teuthology.orchestra.run.smithi110.stdout:38654705701 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 msgr2=0x7f026006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 0x7f026006f110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0264009930 tx=0x7f0264008040 comp rx=0 tx=0).stop 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 msgr2=0x7f027419bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f025c02aa50 tx=0x7f025c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 shutdown_connections 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f026006cc50 0x7f026006f110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02741015f0 0x7f027419bf00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 --2- 172.21.15.110:0/3780725447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0274100cd0 0x7f027419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 >> 172.21.15.110:0/3780725447 conn(0x7f02740fa730 msgr2=0x7f0274101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 shutdown_connections 2024-09-21T04:12:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:06.599+0000 7f027aecd640 1 -- 172.21.15.110:0/3780725447 wait complete. 2024-09-21T04:12:07.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705701 for osd.0 2024-09-21T04:12:07.215 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 -- 172.21.15.110:0/4120424916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 msgr2=0x7f737c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 --2- 172.21.15.110:0/4120424916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c1038c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f736c009880 tx=0x7f736c02a530 comp rx=0 tx=0).stop 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 -- 172.21.15.110:0/4120424916 shutdown_connections 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 --2- 172.21.15.110:0/4120424916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c1038c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 --2- 172.21.15.110:0/4120424916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c0feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 -- 172.21.15.110:0/4120424916 >> 172.21.15.110:0/4120424916 conn(0x7f737c0fa720 msgr2=0x7f737c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 -- 172.21.15.110:0/4120424916 shutdown_connections 2024-09-21T04:12:07.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.389+0000 7f7383bc1640 1 -- 172.21.15.110:0/4120424916 wait complete. 2024-09-21T04:12:07.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 Processor -- start 2024-09-21T04:12:07.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 -- start start 2024-09-21T04:12:07.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c19cd30 con 0x7f737c0ff3f0 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f7383bc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1a0aa0 con 0x7f737c0fead0 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40240/0 (socket says 172.21.15.110:40240) 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 -- 172.21.15.110:0/3156974615 learned_addr learned my addr 172.21.15.110:0/3156974615 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738195d640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 -- 172.21.15.110:0/3156974615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 msgr2=0x7f737c19c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c19c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 -- 172.21.15.110:0/3156974615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c009560 con 0x7f737c0ff3f0 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738195d640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:12:07.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.390+0000 7f738115c640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f736c02aa50 tx=0x7f736c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.391+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c004aa0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.391+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f736c004c00 con 0x7f737c0ff3f0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.391+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c1a0d20 con 0x7f737c0ff3f0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.391+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c0039b0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.391+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c1a1210 con 0x7f737c0ff3f0 2024-09-21T04:12:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.392+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c068ce0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.392+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f736c02ad50 con 0x7f737c0ff3f0 2024-09-21T04:12:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.393+0000 7f7372ffd640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 0x7f736806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.393+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f736c0acd90 con 0x7f737c0ff3f0 2024-09-21T04:12:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.393+0000 7f738195d640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 0x7f736806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.393+0000 7f738195d640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 0x7f736806f110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73780058c0 tx=0x7f7378005d30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.397+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f736c0774a0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:07 smithi110 ceph-mon[27723]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:07.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:07 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3780725447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-21T04:12:07.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.569+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f737c1133d0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.570+0000 7f7372ffd640 1 -- 172.21.15.110:0/3156974615 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f736c07aac0 con 0x7f737c0ff3f0 2024-09-21T04:12:07.573 INFO:teuthology.orchestra.run.smithi110.stdout:167503724557 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 msgr2=0x7f736806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 0x7f736806f110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73780058c0 tx=0x7f7378005d30 comp rx=0 tx=0).stop 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 msgr2=0x7f737c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f736c02aa50 tx=0x7f736c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 shutdown_connections 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f736806cc50 0x7f736806f110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c0ff3f0 0x7f737c19c6a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 --2- 172.21.15.110:0/3156974615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c0fead0 0x7f737c19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 >> 172.21.15.110:0/3156974615 conn(0x7f737c0fa720 msgr2=0x7f737c10d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 shutdown_connections 2024-09-21T04:12:07.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.572+0000 7f7383bc1640 1 -- 172.21.15.110:0/3156974615 wait complete. 2024-09-21T04:12:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:07 smithi148 ceph-mon[34957]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:07 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3780725447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-21T04:12:08.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 -- 172.21.15.110:0/3253153193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 msgr2=0x7f747c0fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:08.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 --2- 172.21.15.110:0/3253153193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c0fee00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7468009880 tx=0x7f746802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 -- 172.21.15.110:0/3253153193 shutdown_connections 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 --2- 172.21.15.110:0/3253153193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c0fee00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 --2- 172.21.15.110:0/3253153193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 0x7f747c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 -- 172.21.15.110:0/3253153193 >> 172.21.15.110:0/3253153193 conn(0x7f747c0758c0 msgr2=0x7f747c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 -- 172.21.15.110:0/3253153193 shutdown_connections 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.558+0000 7f7482d4e640 1 -- 172.21.15.110:0/3253153193 wait complete. 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 Processor -- start 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 -- start start 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 0x7f747c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c19c3e0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7482d4e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c195970 con 0x7f747c1049a0 2024-09-21T04:12:08.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7480aea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7480aea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40248/0 (socket says 172.21.15.110:40248) 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.559+0000 7f7480aea640 1 -- 172.21.15.110:0/981144385 learned_addr learned my addr 172.21.15.110:0/981144385 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f747bfff640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 0x7f747c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7480aea640 1 -- 172.21.15.110:0/981144385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 msgr2=0x7f747c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7480aea640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 0x7f747c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7480aea640 1 -- 172.21.15.110:0/981144385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468009560 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7480aea640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f747000d6f0 tx=0x7f747000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7470009a30 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74700054e0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c195c50 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7470010460 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.560+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c196170 con 0x7f747c0fe9a0 2024-09-21T04:12:08.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.561+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c1000e0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.562+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7470009b90 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.562+0000 7f7479ffb640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 0x7f746c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.563+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f747008bac0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.563+0000 7f747bfff640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 0x7f746c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.563+0000 7f747bfff640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 0x7f746c06f110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7468000c00 tx=0x7f7468005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.567+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f74700561d0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.735+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f747c0fee00 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.736+0000 7f7479ffb640 1 -- 172.21.15.110:0/981144385 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74700597f0 con 0x7f747c0fe9a0 2024-09-21T04:12:08.204 INFO:teuthology.orchestra.run.smithi110.stdout:146028888081 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 msgr2=0x7f746c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 0x7f746c06f110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7468000c00 tx=0x7f7468005a20 comp rx=0 tx=0).stop 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 msgr2=0x7f747c19b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f747000d6f0 tx=0x7f747000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 shutdown_connections 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f747c1049a0 0x7f747c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f746c06cc50 0x7f746c06f110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 --2- 172.21.15.110:0/981144385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c0fe9a0 0x7f747c19b880 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 >> 172.21.15.110:0/981144385 conn(0x7f747c0758c0 msgr2=0x7f747c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 shutdown_connections 2024-09-21T04:12:08.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:07.738+0000 7f7482d4e640 1 -- 172.21.15.110:0/981144385 wait complete. 2024-09-21T04:12:08.496 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:08 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3156974615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-21T04:12:08.496 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:08 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/981144385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-21T04:12:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:08 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3156974615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-21T04:12:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:08 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/981144385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-21T04:12:08.753 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-09-21T04:12:08.753 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:08.770 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:09.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-09-21T04:12:09.166 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:09.393 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:09 smithi110 ceph-mon[27723]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:09 smithi148 ceph-mon[34957]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:09.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.707+0000 7fb5e957a640 1 -- 172.21.15.110:0/890823403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 msgr2=0x7fb5e41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:09.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.707+0000 7fb5e957a640 1 --2- 172.21.15.110:0/890823403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e41038c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009880 tx=0x7fb5cc02a530 comp rx=0 tx=0).stop 2024-09-21T04:12:09.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- 172.21.15.110:0/890823403 shutdown_connections 2024-09-21T04:12:09.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 --2- 172.21.15.110:0/890823403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e41038c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 --2- 172.21.15.110:0/890823403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 0x7fb5e40feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- 172.21.15.110:0/890823403 >> 172.21.15.110:0/890823403 conn(0x7fb5e40fa720 msgr2=0x7fb5e40fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- 172.21.15.110:0/890823403 shutdown_connections 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- 172.21.15.110:0/890823403 wait complete. 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 Processor -- start 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- start start 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 0x7fb5e419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e419cd30 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5e957a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41a0aa0 con 0x7fb5e40fead0 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40270/0 (socket says 172.21.15.110:40270) 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.708+0000 7fb5dbfff640 1 -- 172.21.15.110:0/2385307005 learned_addr learned my addr 172.21.15.110:0/2385307005 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e2ffd640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 0x7fb5e419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5dbfff640 1 -- 172.21.15.110:0/2385307005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 msgr2=0x7fb5e419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5dbfff640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 0x7fb5e419c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5dbfff640 1 -- 172.21.15.110:0/2385307005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009560 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5dbfff640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5cc0048d0 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5cc031640 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5cc003820 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e41a0d20 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.709+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e41a1210 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.710+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e4068ce0 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.711+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb5cc02ad50 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.711+0000 7fb5e0ff9640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 0x7fb5d006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:09.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.711+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb5cc0acbb0 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.711+0000 7fb5e2ffd640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 0x7fb5d006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.711+0000 7fb5e2ffd640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 0x7fb5d006f120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005fd0 tx=0x7fb5d4005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.715+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb5cc0b3080 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.880+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb5e40feeb0 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.881+0000 7fb5e0ff9640 1 -- 172.21.15.110:0/2385307005 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5cc077340 con 0x7fb5e40ff3f0 2024-09-21T04:12:09.883 INFO:teuthology.orchestra.run.smithi110.stdout:111669149718 2024-09-21T04:12:09.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 msgr2=0x7fb5d006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:09.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 0x7fb5d006f120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005fd0 tx=0x7fb5d4005850 comp rx=0 tx=0).stop 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 msgr2=0x7fb5e419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006010 tx=0x7fb5cc0055a0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 shutdown_connections 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb5d006cc60 0x7fb5d006f120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e40ff3f0 0x7fb5e419c6a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 --2- 172.21.15.110:0/2385307005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e40fead0 0x7fb5e419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 >> 172.21.15.110:0/2385307005 conn(0x7fb5e40fa720 msgr2=0x7fb5e410d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 shutdown_connections 2024-09-21T04:12:09.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:09.883+0000 7fb5e957a640 1 -- 172.21.15.110:0/2385307005 wait complete. 2024-09-21T04:12:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:10 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2385307005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-21T04:12:10.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:10 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2385307005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-21T04:12:10.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149718 for osd.3 2024-09-21T04:12:10.751 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:11.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:11 smithi110 ceph-mon[27723]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:11 smithi148 ceph-mon[34957]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:12.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- 172.21.15.110:0/40607876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 msgr2=0x7f3f380fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:12.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 --2- 172.21.15.110:0/40607876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f380fee00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009880 tx=0x7f3f2002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- 172.21.15.110:0/40607876 shutdown_connections 2024-09-21T04:12:12.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 --2- 172.21.15.110:0/40607876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f380fee00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 --2- 172.21.15.110:0/40607876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 0x7f3f38104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- 172.21.15.110:0/40607876 >> 172.21.15.110:0/40607876 conn(0x7f3f380758c0 msgr2=0x7f3f380fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- 172.21.15.110:0/40607876 shutdown_connections 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- 172.21.15.110:0/40607876 wait complete. 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 Processor -- start 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.112+0000 7f3f3de80640 1 -- start start 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 0x7f3f381983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38198a40 con 0x7f3f380fe9a0 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3819c7b0 con 0x7f3f381049a0 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58796/0 (socket says 172.21.15.110:58796) 2024-09-21T04:12:12.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 -- 172.21.15.110:0/3067010513 learned_addr learned my addr 172.21.15.110:0/3067010513 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f2effd640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 0x7f3f381983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 -- 172.21.15.110:0/3067010513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 msgr2=0x7f3f381983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 0x7f3f381983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 -- 172.21.15.110:0/3067010513 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009560 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f377fe640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b420 tx=0x7f3f2800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f28011800 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f28011e20 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3819ca90 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2800f2e0 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.113+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3819cfe0 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.115+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f381000e0 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.115+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f3f28010380 con 0x7f3f380fe9a0 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.115+0000 7f3f357fa640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 0x7f3f2406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:12.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.116+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3f2808b880 con 0x7f3f380fe9a0 2024-09-21T04:12:12.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.116+0000 7f3f2effd640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 0x7f3f2406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:12.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.116+0000 7f3f2effd640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 0x7f3f2406f110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20004120 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:12.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.120+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3f28055f90 con 0x7f3f380fe9a0 2024-09-21T04:12:12.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.283+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f380fee80 con 0x7f3f380fe9a0 2024-09-21T04:12:12.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.284+0000 7f3f357fa640 1 -- 172.21.15.110:0/3067010513 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f280595b0 con 0x7f3f380fe9a0 2024-09-21T04:12:12.287 INFO:teuthology.orchestra.run.smithi110.stdout:60129542177 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 msgr2=0x7f3f2406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 0x7f3f2406f110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20004120 comp rx=0 tx=0).stop 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 msgr2=0x7f3f38197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b420 tx=0x7f3f2800b8e0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 shutdown_connections 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f381049a0 0x7f3f381983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f3f2406cc50 0x7f3f2406f110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 --2- 172.21.15.110:0/3067010513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f380fe9a0 0x7f3f38197e70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 >> 172.21.15.110:0/3067010513 conn(0x7f3f380758c0 msgr2=0x7f3f38102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 shutdown_connections 2024-09-21T04:12:12.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:12.286+0000 7f3f3de80640 1 -- 172.21.15.110:0/3067010513 wait complete. 2024-09-21T04:12:12.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-09-21T04:12:12.966 DEBUG:teuthology.parallel:result is None 2024-09-21T04:12:12.966 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T04:12:12.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:12:13.164 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:13.433 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:13 smithi110 ceph-mon[27723]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:13.433 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:13 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3067010513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-21T04:12:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:13 smithi148 ceph-mon[34957]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:13 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3067010513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-21T04:12:14.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 -- 172.21.15.110:0/1175494206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 msgr2=0x7efc7c104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:14.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 --2- 172.21.15.110:0/1175494206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c104d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009880 tx=0x7efc6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 -- 172.21.15.110:0/1175494206 shutdown_connections 2024-09-21T04:12:14.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 --2- 172.21.15.110:0/1175494206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 0x7efc7c0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 --2- 172.21.15.110:0/1175494206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c104d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 -- 172.21.15.110:0/1175494206 >> 172.21.15.110:0/1175494206 conn(0x7efc7c0758c0 msgr2=0x7efc7c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:14.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 -- 172.21.15.110:0/1175494206 shutdown_connections 2024-09-21T04:12:14.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.790+0000 7efc8182b640 1 -- 172.21.15.110:0/1175494206 wait complete. 2024-09-21T04:12:14.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 Processor -- start 2024-09-21T04:12:14.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 -- start start 2024-09-21T04:12:14.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 0x7efc7c196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:14.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:14.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c197390 con 0x7efc7c1049a0 2024-09-21T04:12:14.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc8182b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c197500 con 0x7efc7c0fe9a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58816/0 (socket says 172.21.15.110:58816) 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 -- 172.21.15.110:0/3092710547 learned_addr learned my addr 172.21.15.110:0/3092710547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7affd640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 0x7efc7c196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 -- 172.21.15.110:0/3092710547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 msgr2=0x7efc7c196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 0x7efc7c196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.791+0000 7efc7a7fc640 1 -- 172.21.15.110:0/3092710547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c009560 con 0x7efc7c1049a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc7a7fc640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c8a0 tx=0x7efc6400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc64007b80 con 0x7efc7c1049a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc64007ce0 con 0x7efc7c1049a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c19f980 con 0x7efc7c1049a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc64018810 con 0x7efc7c1049a0 2024-09-21T04:12:14.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.792+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c194700 con 0x7efc7c1049a0 2024-09-21T04:12:14.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.793+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c1000e0 con 0x7efc7c1049a0 2024-09-21T04:12:14.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.794+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7efc64018a80 con 0x7efc7c1049a0 2024-09-21T04:12:14.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.794+0000 7efc80829640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 0x7efc68073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:14.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.794+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efc6408c110 con 0x7efc7c1049a0 2024-09-21T04:12:14.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.794+0000 7efc7affd640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 0x7efc68073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:14.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.795+0000 7efc7affd640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 0x7efc68073520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000c00 tx=0x7efc6c005a20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:14.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.798+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efc64056890 con 0x7efc7c1049a0 2024-09-21T04:12:14.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.963+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc7c10bc10 con 0x7efc68071060 2024-09-21T04:12:14.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.965+0000 7efc80829640 1 -- 172.21.15.110:0/3092710547 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27624 (secure 0 0 0) 0x7efc7c10bc10 con 0x7efc68071060 2024-09-21T04:12:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:14.972 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-21T04:12:14.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 msgr2=0x7efc68073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:14.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 0x7efc68073520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000c00 tx=0x7efc6c005a20 comp rx=0 tx=0).stop 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 msgr2=0x7efc7c196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c8a0 tx=0x7efc6400cd60 comp rx=0 tx=0).stop 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 shutdown_connections 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efc68071060 0x7efc68073520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c1049a0 0x7efc7c196e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 --2- 172.21.15.110:0/3092710547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc7c0fe9a0 0x7efc7c196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 >> 172.21.15.110:0/3092710547 conn(0x7efc7c0758c0 msgr2=0x7efc7c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 shutdown_connections 2024-09-21T04:12:14.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:14.967+0000 7efc8182b640 1 -- 172.21.15.110:0/3092710547 wait complete. 2024-09-21T04:12:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:15 smithi110 ceph-mon[27723]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:15 smithi148 ceph-mon[34957]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:15.751 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-21T04:12:14.182157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":215156,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":211190,"kb_avail":562255756,"statfs":{"total":575970213888,"available":575749894144,"internally_reserved":0,"allocated":3964928,"data_stored":2609187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":216259238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001908"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T04:11:20.854163+0000","last_change":"2024-09-21T04:10:55.521168+0000","last_active":"2024-09-21T04:11:20.854163+0000","last_peered":"2024-09-21T04:11:20.854163+0000","last_clean":"2024-09-21T04:11:20.854163+0000","last_became_active":"2024-09-21T04:10:55.520895+0000","last_became_peered":"2024-09-21T04:10:55.520895+0000","last_unstale":"2024-09-21T04:11:20.854163+0000","last_undegraded":"2024-09-21T04:11:20.854163+0000","last_fullsized":"2024-09-21T04:11:20.854163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:10:12.443385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:10:12.443385+0000","last_clean_scrub_stamp":"2024-09-21T04:10:12.443385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:27.828433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713680,"statfs":{"total":95995035648,"available":95962808320,"internally_reserved":0,"allocated":364544,"data_stored":139792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.352}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713680,"statfs":{"total":95995035648,"available":95962808320,"internally_reserved":0,"allocated":364544,"data_stored":139792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.504}]},{"osd":1,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46100000000000002}]},{"osd":3,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.61499999999999999}]},{"osd":1,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Sep 21 04:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":39664,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93705488,"statfs":{"total":95995035648,"available":95954419712,"internally_reserved":0,"allocated":360448,"data_stored":139123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Sep 21 04:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.254},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Sat Sep 21 04:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Sep 21 04:11:59 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40248,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704904,"statfs":{"total":95995035648,"available":95953821696,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28699999999999998},"min":{"1min":0.222,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.437,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.371}]},{"osd":2,"last update":"Sat Sep 21 04:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.42199999999999999}]},{"osd":3,"last update":"Sat Sep 21 04:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Sep 21 04:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":18.997,"5min":18.997,"15min":18.997},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":19.015999999999998,"5min":19.015999999999998,"15min":19.015999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40244,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704908,"statfs":{"total":95995035648,"available":95953825792,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 04:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.307,"15min":0.30399999999999999},"min":{"1min":0.223,"5min":0.184,"15min":0.184},"max":{"1min":0.45500000000000002,"5min":0.497,"15min":0.497},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Sep 21 04:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.309},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Sep 21 04:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:12:15.753 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:12:15.956 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:16.357 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:16 smithi110 ceph-mon[27723]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:12:16.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:16 smithi148 ceph-mon[34957]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:12:17.322 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:17 smithi110 ceph-mon[27723]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:17.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 -- 172.21.15.110:0/182773152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 msgr2=0x7fe8140fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/182773152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe8140fee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe808009880 tx=0x7fe80802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 -- 172.21.15.110:0/182773152 shutdown_connections 2024-09-21T04:12:17.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/182773152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe8140fee00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/182773152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 0x7fe814104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 -- 172.21.15.110:0/182773152 >> 172.21.15.110:0/182773152 conn(0x7fe8140758c0 msgr2=0x7fe8140fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:17.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 -- 172.21.15.110:0/182773152 shutdown_connections 2024-09-21T04:12:17.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.507+0000 7fe81a2ad640 1 -- 172.21.15.110:0/182773152 wait complete. 2024-09-21T04:12:17.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.508+0000 7fe81a2ad640 1 Processor -- start 2024-09-21T04:12:17.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.508+0000 7fe81a2ad640 1 -- start start 2024-09-21T04:12:17.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 0x7fe814194f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe814195620 con 0x7fe8140fe9a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe814199370 con 0x7fe8141049a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58836/0 (socket says 172.21.15.110:58836) 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 -- 172.21.15.110:0/3971789391 learned_addr learned my addr 172.21.15.110:0/3971789391 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe8137fe640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 0x7fe814194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 -- 172.21.15.110:0/3971789391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 msgr2=0x7fe814194f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 0x7fe814194f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 -- 172.21.15.110:0/3971789391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009560 con 0x7fe8140fe9a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe813fff640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe80400d6f0 tx=0x7fe80400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe804009a30 con 0x7fe8140fe9a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8040054e0 con 0x7fe8140fe9a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe814199650 con 0x7fe8140fe9a0 2024-09-21T04:12:17.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe804010460 con 0x7fe8140fe9a0 2024-09-21T04:12:17.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.509+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe814199b70 con 0x7fe8140fe9a0 2024-09-21T04:12:17.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.510+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe814103fe0 con 0x7fe8140fe9a0 2024-09-21T04:12:17.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.511+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe804009b90 con 0x7fe8140fe9a0 2024-09-21T04:12:17.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.511+0000 7fe8117fa640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 0x7fe7fc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.511+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe80408ba10 con 0x7fe8140fe9a0 2024-09-21T04:12:17.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.511+0000 7fe8137fe640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 0x7fe7fc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.512+0000 7fe8137fe640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 0x7fe7fc06f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe808009230 tx=0x7fe8080058d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:17.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.516+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe8040561d0 con 0x7fe8140fe9a0 2024-09-21T04:12:17.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.675+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe814195d60 con 0x7fe7fc06cc50 2024-09-21T04:12:17.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.677+0000 7fe8117fa640 1 -- 172.21.15.110:0/3971789391 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27623 (secure 0 0 0) 0x7fe814195d60 con 0x7fe7fc06cc50 2024-09-21T04:12:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:17.680 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-21T04:12:17.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 msgr2=0x7fe7fc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 0x7fe7fc06f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe808009230 tx=0x7fe8080058d0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 msgr2=0x7fe814194a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe80400d6f0 tx=0x7fe80400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 shutdown_connections 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8141049a0 0x7fe814194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7fc06cc50 0x7fe7fc06f110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 --2- 172.21.15.110:0/3971789391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8140fe9a0 0x7fe814194a00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 >> 172.21.15.110:0/3971789391 conn(0x7fe8140758c0 msgr2=0x7fe8140fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 shutdown_connections 2024-09-21T04:12:17.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:17.679+0000 7fe81a2ad640 1 -- 172.21.15.110:0/3971789391 wait complete. 2024-09-21T04:12:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:17 smithi148 ceph-mon[34957]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:18.318 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-21T04:12:16.182639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":215156,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":211190,"kb_avail":562255756,"statfs":{"total":575970213888,"available":575749894144,"internally_reserved":0,"allocated":3964928,"data_stored":2609187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":216259238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002012"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T04:11:20.854163+0000","last_change":"2024-09-21T04:10:55.521168+0000","last_active":"2024-09-21T04:11:20.854163+0000","last_peered":"2024-09-21T04:11:20.854163+0000","last_clean":"2024-09-21T04:11:20.854163+0000","last_became_active":"2024-09-21T04:10:55.520895+0000","last_became_peered":"2024-09-21T04:10:55.520895+0000","last_unstale":"2024-09-21T04:11:20.854163+0000","last_undegraded":"2024-09-21T04:11:20.854163+0000","last_fullsized":"2024-09-21T04:11:20.854163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:10:12.443385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:10:12.443385+0000","last_clean_scrub_stamp":"2024-09-21T04:10:12.443385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:27.828433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713680,"statfs":{"total":95995035648,"available":95962808320,"internally_reserved":0,"allocated":364544,"data_stored":139792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713680,"statfs":{"total":95995035648,"available":95962808320,"internally_reserved":0,"allocated":364544,"data_stored":139792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.504}]},{"osd":1,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46100000000000002}]},{"osd":3,"last update":"Sat Sep 21 04:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.61499999999999999}]},{"osd":1,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Sep 21 04:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":39664,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93705488,"statfs":{"total":95995035648,"available":95954419712,"internally_reserved":0,"allocated":360448,"data_stored":139123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26000000000000001}]},{"osd":1,"last update":"Sat Sep 21 04:11:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.254},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Sat Sep 21 04:11:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Sep 21 04:11:59 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40248,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704904,"statfs":{"total":95995035648,"available":95953821696,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 04:11:43 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28699999999999998},"min":{"1min":0.222,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.437,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.371}]},{"osd":2,"last update":"Sat Sep 21 04:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.42199999999999999}]},{"osd":3,"last update":"Sat Sep 21 04:11:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Sep 21 04:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":18.997,"5min":18.997,"15min":18.997},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":19.015999999999998,"5min":19.015999999999998,"15min":19.015999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40244,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704908,"statfs":{"total":95995035648,"available":95953825792,"internally_reserved":0,"allocated":958464,"data_stored":730160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 04:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.307,"15min":0.30399999999999999},"min":{"1min":0.223,"5min":0.184,"15min":0.184},"max":{"1min":0.45500000000000002,"5min":0.497,"15min":0.497},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.311,"15min":0.307},"min":{"1min":0.22900000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Sep 21 04:11:13 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.309},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Sat Sep 21 04:11:32 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Sep 21 04:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:12:18.319 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T04:12:18.319 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T04:12:18.319 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T04:12:18.319 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T04:12:18.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:18.518 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:18.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:19.570 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:19 smithi110 ceph-mon[27723]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:12:19.570 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:19 smithi110 ceph-mon[27723]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:19.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:19 smithi148 ceph-mon[34957]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:12:19.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:19 smithi148 ceph-mon[34957]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:20.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/2854350954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 msgr2=0x7fbcd80fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/2854350954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd80fe860 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009880 tx=0x7fbcd402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/2854350954 shutdown_connections 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/2854350954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 0x7fbcd8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/2854350954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd80fe860 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/2854350954 >> 172.21.15.110:0/2854350954 conn(0x7fbcd80f9ec0 msgr2=0x7fbcd80fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/2854350954 shutdown_connections 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/2854350954 wait complete. 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 Processor -- start 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.022+0000 7fbcdfcd8640 1 -- start start 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 0x7fbcd819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd819c4b0 con 0x7fbcd80fe480 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8195a40 con 0x7fbcd80fee30 2024-09-21T04:12:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58852/0 (socket says 172.21.15.110:58852) 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 -- 172.21.15.110:0/4269314542 learned_addr learned my addr 172.21.15.110:0/4269314542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdd273640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 0x7fbcd819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 -- 172.21.15.110:0/4269314542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 msgr2=0x7fbcd819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 0x7fbcd819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 -- 172.21.15.110:0/4269314542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4009560 con 0x7fbcd80fe480 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdda74640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402a9c0 tx=0x7fbcd40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd4004aa0 con 0x7fbcd80fe480 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcd4004c00 con 0x7fbcd80fe480 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd8195cc0 con 0x7fbcd80fe480 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd4003ab0 con 0x7fbcd80fe480 2024-09-21T04:12:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.023+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd81961b0 con 0x7fbcd80fe480 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.024+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd80ff960 con 0x7fbcd80fe480 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.025+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fbcd4031d20 con 0x7fbcd80fe480 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.025+0000 7fbcceffd640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 0x7fbcc406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.025+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbcd40acf20 con 0x7fbcd80fe480 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.025+0000 7fbcdd273640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 0x7fbcc406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.026+0000 7fbcdd273640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 0x7fbcc406f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8196eb0 tx=0x7fbcc8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.029+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbcd40e0e20 con 0x7fbcd80fe480 2024-09-21T04:12:20.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.233+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbcd810b510 con 0x7fbcd80fe480 2024-09-21T04:12:20.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.234+0000 7fbcceffd640 1 -- 172.21.15.110:0/4269314542 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbcd407b140 con 0x7fbcd80fe480 2024-09-21T04:12:20.237 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:20.237 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 msgr2=0x7fbcc406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 0x7fbcc406f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8196eb0 tx=0x7fbcc8008040 comp rx=0 tx=0).stop 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 msgr2=0x7fbcd819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402a9c0 tx=0x7fbcd40055a0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 shutdown_connections 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcd80fee30 0x7fbcd819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbcc406cc50 0x7fbcc406f110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 --2- 172.21.15.110:0/4269314542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd80fe480 0x7fbcd819b950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 >> 172.21.15.110:0/4269314542 conn(0x7fbcd80f9ec0 msgr2=0x7fbcd8102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 shutdown_connections 2024-09-21T04:12:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:20.236+0000 7fbcdfcd8640 1 -- 172.21.15.110:0/4269314542 wait complete. 2024-09-21T04:12:20.542 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:20 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4269314542' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-21T04:12:20.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:20 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4269314542' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-21T04:12:20.849 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T04:12:20.849 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T04:12:20.850 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:12:20.858 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-09-21T04:12:20.858 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:20.869 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:20.869 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-21T04:12:21.068 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:21.522 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:21 smithi110 ceph-mon[27723]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:21.522 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:21 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:21 smithi148 ceph-mon[34957]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:21 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.650+0000 7f3577786640 1 -- 172.21.15.110:0/139995113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700fea30 msgr2=0x7f35700fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.650+0000 7f3577786640 1 --2- 172.21.15.110:0/139995113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700fea30 0x7f35700fee30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f356c009880 tx=0x7f356c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.650+0000 7f3577786640 1 -- 172.21.15.110:0/139995113 shutdown_connections 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.650+0000 7f3577786640 1 --2- 172.21.15.110:0/139995113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700ffc30 0x7f35701000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 --2- 172.21.15.110:0/139995113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700fea30 0x7f35700fee30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- 172.21.15.110:0/139995113 >> 172.21.15.110:0/139995113 conn(0x7f35700fa1c0 msgr2=0x7f35700fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- 172.21.15.110:0/139995113 shutdown_connections 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- 172.21.15.110:0/139995113 wait complete. 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 Processor -- start 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- start start 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 0x7f3570068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357019aa20 con 0x7f35700ffc30 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3577786640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357019ab60 con 0x7f35700fea30 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.651+0000 7f3574d21640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48014/0 (socket says 172.21.15.110:48014) 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 -- 172.21.15.110:0/312401178 learned_addr learned my addr 172.21.15.110:0/312401178 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3575522640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 0x7f3570068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 -- 172.21.15.110:0/312401178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 msgr2=0x7f3570068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 0x7f3570068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 -- 172.21.15.110:0/312401178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009560 con 0x7f35700ffc30 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3575522640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 0x7f3570068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3574d21640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f356000b420 tx=0x7f356000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3560011800 con 0x7f35700ffc30 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3560011e20 con 0x7f35700ffc30 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357019ae10 con 0x7f35700ffc30 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356000f2e0 con 0x7f35700ffc30 2024-09-21T04:12:22.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.652+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357019b3b0 con 0x7f35700ffc30 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.653+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35700feeb0 con 0x7f35700ffc30 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.654+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f3560010380 con 0x7f35700ffc30 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.654+0000 7f35667fc640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 0x7f355c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.654+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f356008b7e0 con 0x7f35700ffc30 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.654+0000 7f3575522640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 0x7f355c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.654+0000 7f3575522640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 0x7f355c06f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f356c031540 tx=0x7f356c005de0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.658+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3560055f70 con 0x7f35700ffc30 2024-09-21T04:12:22.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.823+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f35701076a0 con 0x7f35700ffc30 2024-09-21T04:12:22.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.827+0000 7f35667fc640 1 -- 172.21.15.110:0/312401178 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v44)=0 v44) v1 ==== 143+0+0 (secure 0 0 0) 0x7f3560059590 con 0x7f35700ffc30 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 msgr2=0x7f355c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 0x7f355c06f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f356c031540 tx=0x7f356c005de0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 msgr2=0x7f357019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f356000b420 tx=0x7f356000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 shutdown_connections 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f355c06cc50 0x7f355c06f110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35700ffc30 0x7f357019a450 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 --2- 172.21.15.110:0/312401178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700fea30 0x7f3570068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 >> 172.21.15.110:0/312401178 conn(0x7f35700fa1c0 msgr2=0x7f3570102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 shutdown_connections 2024-09-21T04:12:22.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:22.830+0000 7f3577786640 1 -- 172.21.15.110:0/312401178 wait complete. 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/312401178' entity='client.admin' 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:23.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:23.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:23.447 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:12:23.467 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-21T04:12:23.467 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:23.511 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:23.511 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T04:12:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/312401178' entity='client.admin' 2024-09-21T04:12:23.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:23.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:23.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:23.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:23.712 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 -- 172.21.15.110:0/329350957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c8100d40 msgr2=0x7f16c81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 --2- 172.21.15.110:0/329350957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c8100d40 0x7f16c81011c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009880 tx=0x7f16b002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 -- 172.21.15.110:0/329350957 shutdown_connections 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 --2- 172.21.15.110:0/329350957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c8100d40 0x7f16c81011c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 --2- 172.21.15.110:0/329350957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c80ffb40 0x7f16c80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 -- 172.21.15.110:0/329350957 >> 172.21.15.110:0/329350957 conn(0x7f16c80fb2f0 msgr2=0x7f16c80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.204+0000 7f16ccd08640 1 -- 172.21.15.110:0/329350957 shutdown_connections 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.205+0000 7f16ccd08640 1 -- 172.21.15.110:0/329350957 wait complete. 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.205+0000 7f16ccd08640 1 Processor -- start 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.205+0000 7f16ccd08640 1 -- start start 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c8100d40 0x7f16c8195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8196220 con 0x7f16c80ffb40 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8196390 con 0x7f16c8100d40 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48022/0 (socket says 172.21.15.110:48022) 2024-09-21T04:12:25.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 -- 172.21.15.110:0/432075695 learned_addr learned my addr 172.21.15.110:0/432075695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16bdd9b640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c8100d40 0x7f16c8195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 -- 172.21.15.110:0/432075695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c8100d40 msgr2=0x7f16c8195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c8100d40 0x7f16c8195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 -- 172.21.15.110:0/432075695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0009560 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16c659c640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f16b800b400 tx=0x7f16b800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b8011800 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b8011e20 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c819ae30 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b800f2e0 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.206+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c819b3a0 con 0x7f16c80ffb40 2024-09-21T04:12:25.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.207+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c81087d0 con 0x7f16c80ffb40 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.208+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f16b8010380 con 0x7f16c80ffb40 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.208+0000 7f16bffff640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 0x7f16b406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.208+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f16b808bbe0 con 0x7f16c80ffb40 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.208+0000 7f16bdd9b640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 0x7f16b406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.209+0000 7f16bdd9b640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 0x7f16b406f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031540 tx=0x7f16b0000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:25.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.212+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f16b80562f0 con 0x7f16c80ffb40 2024-09-21T04:12:25.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.389+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16c819b620 con 0x7f16b406cc50 2024-09-21T04:12:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.390+0000 7f16bffff640 1 -- 172.21.15.110:0/432075695 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f16c819b620 con 0x7f16b406cc50 2024-09-21T04:12:25.392 INFO:teuthology.orchestra.run.smithi110.stdout:Backend: cephadm 2024-09-21T04:12:25.393 INFO:teuthology.orchestra.run.smithi110.stdout:Available: Yes 2024-09-21T04:12:25.393 INFO:teuthology.orchestra.run.smithi110.stdout:Paused: No 2024-09-21T04:12:25.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 msgr2=0x7f16b406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 0x7f16b406f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16b0031540 tx=0x7f16b0000bc0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 msgr2=0x7f16c8195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f16b800b400 tx=0x7f16b800b8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 shutdown_connections 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c8100d40 0x7f16c8195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f16b406cc50 0x7f16b406f110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 --2- 172.21.15.110:0/432075695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16c80ffb40 0x7f16c8195710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 >> 172.21.15.110:0/432075695 conn(0x7f16c80fb2f0 msgr2=0x7f16c8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 shutdown_connections 2024-09-21T04:12:25.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:25.392+0000 7f16ccd08640 1 -- 172.21.15.110:0/432075695 wait complete. 2024-09-21T04:12:25.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:25 smithi110 ceph-mon[27723]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:25 smithi148 ceph-mon[34957]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:26.047 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T04:12:26.250 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:26.610 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:26 smithi110 ceph-mon[27723]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:26.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:26 smithi148 ceph-mon[34957]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:27.620 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:27 smithi110 ceph-mon[27723]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:27 smithi148 ceph-mon[34957]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:27.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2408024745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0fea80 msgr2=0x7f1fac0fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:27.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2408024745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0fea80 0x7f1fac0fee80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009880 tx=0x7f1fa002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:27.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2408024745 shutdown_connections 2024-09-21T04:12:27.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2408024745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0ffc80 0x7f1fac100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2408024745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0fea80 0x7f1fac0fee80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2408024745 >> 172.21.15.110:0/2408024745 conn(0x7f1fac0fa230 msgr2=0x7f1fac0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2408024745 shutdown_connections 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.803+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2408024745 wait complete. 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 Processor -- start 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 -- start start 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 0x7f1fac197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac198330 con 0x7f1fac0ffc80 2024-09-21T04:12:27.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb2ba7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac1984a0 con 0x7f1fac0fea80 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48032/0 (socket says 172.21.15.110:48032) 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 -- 172.21.15.110:0/2072875554 learned_addr learned my addr 172.21.15.110:0/2072875554 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb0943640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 0x7f1fac197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 -- 172.21.15.110:0/2072875554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 msgr2=0x7f1fac197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 0x7f1fac197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 -- 172.21.15.110:0/2072875554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009560 con 0x7f1fac0ffc80 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fb0943640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 0x7f1fac197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.804+0000 7f1fabfff640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c840 tx=0x7f1f9800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:27.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.805+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f980041f0 con 0x7f1fac0ffc80 2024-09-21T04:12:27.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.805+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f98007e70 con 0x7f1fac0ffc80 2024-09-21T04:12:27.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.805+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f980049d0 con 0x7f1fac0ffc80 2024-09-21T04:12:27.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.805+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac19cef0 con 0x7f1fac0ffc80 2024-09-21T04:12:27.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.805+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac19d490 con 0x7f1fac0ffc80 2024-09-21T04:12:27.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.806+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0fee80 con 0x7f1fac0ffc80 2024-09-21T04:12:27.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.806+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f1f980079b0 con 0x7f1fac0ffc80 2024-09-21T04:12:27.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.807+0000 7f1fa9ffb640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 0x7f1f9c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:27.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.807+0000 7f1fb0943640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 0x7f1f9c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:27.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.807+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1f9808bca0 con 0x7f1fac0ffc80 2024-09-21T04:12:27.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.807+0000 7f1fb0943640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 0x7f1f9c06f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009230 tx=0x7f1fa00314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:27.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.811+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1f98090070 con 0x7f1fac0ffc80 2024-09-21T04:12:27.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.984+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fac0608f0 con 0x7f1f9c06cc50 2024-09-21T04:12:27.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.992+0000 7f1fa9ffb640 1 -- 172.21.15.110:0/2072875554 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3080 (secure 0 0 0) 0x7f1fac0608f0 con 0x7f1f9c06cc50 2024-09-21T04:12:27.999 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:12:27.999 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (3m) 2m ago 5m 26.6M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:12:27.999 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (5m) 2m ago 5m 7050k - 18.2.4-871-g6a1bdf85 0adab276c97e b0186c860d81 2024-09-21T04:12:27.999 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (5m) 70s ago 4m 7042k - 18.2.4-871-g6a1bdf85 0adab276c97e 6e1e8fb80a91 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 2m ago 5m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (4m) 70s ago 4m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (3m) 2m ago 5m 76.6M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (6m) 2m ago 6m 477M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (4m) 70s ago 4m 439M - 18.2.4-871-g6a1bdf85 0adab276c97e 4f74b6e9b876 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 2m ago 7m 45.1M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (4m) 70s ago 4m 41.3M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (5m) 2m ago 5m 16.5M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 70s ago 4m 15.0M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (3m) 2m ago 3m 48.7M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:12:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (2m) 2m ago 2m 45.8M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 2m ago 2m 12.0M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (2m) 70s ago 2m 48.6M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (99s) 70s ago 98s 44.8M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (75s) 70s ago 74s 14.3M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 2m ago 5m 44.6M - 2.43.0 a07b618ecd1d 2d686ff7a6ad 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 msgr2=0x7f1f9c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:28.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 0x7f1f9c06f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009230 tx=0x7f1fa00314e0 comp rx=0 tx=0).stop 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 msgr2=0x7f1fac197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c840 tx=0x7f1f9800cd00 comp rx=0 tx=0).stop 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 shutdown_connections 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f1f9c06cc50 0x7f1f9c06f110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fac0ffc80 0x7f1fac197d60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 --2- 172.21.15.110:0/2072875554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fac0fea80 0x7f1fac197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 >> 172.21.15.110:0/2072875554 conn(0x7f1fac0fa230 msgr2=0x7f1fac0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 shutdown_connections 2024-09-21T04:12:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:27.994+0000 7f1fb2ba7640 1 -- 172.21.15.110:0/2072875554 wait complete. 2024-09-21T04:12:28.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T04:12:28.830 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:29.564 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:29 smithi110 ceph-mon[27723]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:29.564 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:29 smithi110 ceph-mon[27723]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:29.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:29 smithi148 ceph-mon[34957]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:29.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:29 smithi148 ceph-mon[34957]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:30.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 -- 172.21.15.110:0/1691611138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 msgr2=0x7f5418101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:30.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 --2- 172.21.15.110:0/1691611138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418101210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5400009880 tx=0x7f540002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 -- 172.21.15.110:0/1691611138 shutdown_connections 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 --2- 172.21.15.110:0/1691611138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418101210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 --2- 172.21.15.110:0/1691611138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180ffb90 0x7f54180fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 -- 172.21.15.110:0/1691611138 >> 172.21.15.110:0/1691611138 conn(0x7f54180fb320 msgr2=0x7f54180fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 -- 172.21.15.110:0/1691611138 shutdown_connections 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.376+0000 7f541d327640 1 -- 172.21.15.110:0/1691611138 wait complete. 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 Processor -- start 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 -- start start 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180ffb90 0x7f5418195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418196240 con 0x7f5418100d90 2024-09-21T04:12:30.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f541d327640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181963b0 con 0x7f54180ffb90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48046/0 (socket says 172.21.15.110:48046) 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 -- 172.21.15.110:0/2811430682 learned_addr learned my addr 172.21.15.110:0/2811430682 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 -- 172.21.15.110:0/2811430682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180ffb90 msgr2=0x7f5418195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180ffb90 0x7f5418195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.377+0000 7f540ffff640 1 -- 172.21.15.110:0/2811430682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5400009560 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f540ffff640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5400005640 tx=0x7f5400005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5400004ba0 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5400004d00 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5400003c30 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541819adf0 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.378+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541819b330 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.379+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5418100010 con 0x7f5418100d90 2024-09-21T04:12:30.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.379+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f5400031640 con 0x7f5418100d90 2024-09-21T04:12:30.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.380+0000 7f5414ff9640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 0x7f540406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:30.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.380+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f54000ad1b0 con 0x7f5418100d90 2024-09-21T04:12:30.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.380+0000 7f5416ffd640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 0x7f540406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:30.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.380+0000 7f5416ffd640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 0x7f540406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5418100bf0 tx=0x7f5408006c90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:30.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.384+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f54000778c0 con 0x7f5418100d90 2024-09-21T04:12:30.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.553+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f54181056e0 con 0x7f540406cc50 2024-09-21T04:12:30.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.558+0000 7f5414ff9640 1 -- 172.21.15.110:0/2811430682 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f54181056e0 con 0x7f540406cc50 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter 2/2 2m ago 6m * 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:crash 2/2 2m ago 6m * 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:mgr 2/2 2m ago 6m count:2 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:mon 2/2 2m ago 5m smithi110:172.21.15.110=smithi110;smithi148:172.21.15.148=smithi148;count:2 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:osd 6 2m ago - 2024-09-21T04:12:30.561 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 msgr2=0x7f540406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 0x7f540406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5418100bf0 tx=0x7f5408006c90 comp rx=0 tx=0).stop 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 msgr2=0x7f5418195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5400005640 tx=0x7f5400005670 comp rx=0 tx=0).stop 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 shutdown_connections 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f540406cc50 0x7f540406f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5418100d90 0x7f5418195c70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 --2- 172.21.15.110:0/2811430682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180ffb90 0x7f5418195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 >> 172.21.15.110:0/2811430682 conn(0x7f54180fb320 msgr2=0x7f5418103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 shutdown_connections 2024-09-21T04:12:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:30.560+0000 7f541d327640 1 -- 172.21.15.110:0/2811430682 wait complete. 2024-09-21T04:12:31.066 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T04:12:31.264 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:31.580 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:31 smithi110 ceph-mon[27723]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:31 smithi148 ceph-mon[34957]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:32.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:32 smithi110 ceph-mon[27723]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:32 smithi148 ceph-mon[34957]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- 172.21.15.110:0/1473282160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 msgr2=0x7fae700fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 --2- 172.21.15.110:0/1473282160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae700fee30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fae68009880 tx=0x7fae6802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- 172.21.15.110:0/1473282160 shutdown_connections 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 --2- 172.21.15.110:0/1473282160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 0x7fae701000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 --2- 172.21.15.110:0/1473282160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae700fee30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- 172.21.15.110:0/1473282160 >> 172.21.15.110:0/1473282160 conn(0x7fae700fa1c0 msgr2=0x7fae700fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:32.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- 172.21.15.110:0/1473282160 shutdown_connections 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- 172.21.15.110:0/1473282160 wait complete. 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 Processor -- start 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.765+0000 7fae785b3640 1 -- start start 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 0x7fae70101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae70102f20 con 0x7fae700fea30 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae70101580 con 0x7fae700ffc30 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55704/0 (socket says 172.21.15.110:55704) 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 -- 172.21.15.110:0/3559159012 learned_addr learned my addr 172.21.15.110:0/3559159012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae75b4e640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 0x7fae70101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:32.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 -- 172.21.15.110:0/3559159012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 msgr2=0x7fae70101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 0x7fae70101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 -- 172.21.15.110:0/3559159012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009560 con 0x7fae700fea30 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae7634f640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae6802a9c0 tx=0x7fae68005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68004b20 con 0x7fae700fea30 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae68004c80 con 0x7fae700fea30 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70101800 con 0x7fae700fea30 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68003b80 con 0x7fae700fea30 2024-09-21T04:12:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.766+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae70101d70 con 0x7fae700fea30 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.767+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae700feeb0 con 0x7fae700fea30 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.768+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fae68031640 con 0x7fae700fea30 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.768+0000 7fae6f7fe640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 0x7fae5806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.768+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fae680ad270 con 0x7fae700fea30 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.768+0000 7fae75b4e640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 0x7fae5806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.769+0000 7fae75b4e640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 0x7fae5806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae701026d0 tx=0x7fae64008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.772+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fae68077980 con 0x7fae700fea30 2024-09-21T04:12:32.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.941+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fae700682f0 con 0x7fae5806cc50 2024-09-21T04:12:32.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.942+0000 7fae6f7fe640 1 -- 172.21.15.110:0/3559159012 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fae700682f0 con 0x7fae5806cc50 2024-09-21T04:12:32.945 INFO:teuthology.orchestra.run.smithi110.stdout:HOST ADDR LABELS STATUS 2024-09-21T04:12:32.945 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 172.21.15.110 2024-09-21T04:12:32.945 INFO:teuthology.orchestra.run.smithi110.stdout:smithi148 172.21.15.148 2024-09-21T04:12:32.945 INFO:teuthology.orchestra.run.smithi110.stdout:2 hosts in cluster 2024-09-21T04:12:32.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 msgr2=0x7fae5806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:32.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 0x7fae5806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae701026d0 tx=0x7fae64008040 comp rx=0 tx=0).stop 2024-09-21T04:12:32.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 msgr2=0x7fae70102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:32.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae6802a9c0 tx=0x7fae68005610 comp rx=0 tx=0).stop 2024-09-21T04:12:32.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 shutdown_connections 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae700ffc30 0x7fae70101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fae5806cc50 0x7fae5806f110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 --2- 172.21.15.110:0/3559159012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae700fea30 0x7fae70102950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 >> 172.21.15.110:0/3559159012 conn(0x7fae700fa1c0 msgr2=0x7fae70067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 shutdown_connections 2024-09-21T04:12:32.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:32.944+0000 7fae785b3640 1 -- 172.21.15.110:0/3559159012 wait complete. 2024-09-21T04:12:33.570 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:33 smithi110 ceph-mon[27723]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:33.570 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:33.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T04:12:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:33 smithi148 ceph-mon[34957]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:33.768 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:34.575 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:34 smithi110 ceph-mon[27723]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:34.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:34 smithi148 ceph-mon[34957]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:35.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 -- 172.21.15.110:0/2782459157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c101280 msgr2=0x7f8e7c101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 --2- 172.21.15.110:0/2782459157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c101280 0x7f8e7c101680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009c20 tx=0x7f8e7802a970 comp rx=0 tx=0).stop 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 -- 172.21.15.110:0/2782459157 shutdown_connections 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 --2- 172.21.15.110:0/2782459157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 --2- 172.21.15.110:0/2782459157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c101280 0x7f8e7c101680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 -- 172.21.15.110:0/2782459157 >> 172.21.15.110:0/2782459157 conn(0x7f8e7c0fb330 msgr2=0x7f8e7c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 -- 172.21.15.110:0/2782459157 shutdown_connections 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.322+0000 7f8e83acd640 1 -- 172.21.15.110:0/2782459157 wait complete. 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 Processor -- start 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 -- start start 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c101280 0x7f8e7c195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c196220 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e83acd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c196390 con 0x7f8e7c101280 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e81869640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e81869640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55714/0 (socket says 172.21.15.110:55714) 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e81869640 1 -- 172.21.15.110:0/3334375481 learned_addr learned my addr 172.21.15.110:0/3334375481 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.323+0000 7f8e81068640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c101280 0x7f8e7c195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e81869640 1 -- 172.21.15.110:0/3334375481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c101280 msgr2=0x7f8e7c195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e81869640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c101280 0x7f8e7c195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e81869640 1 -- 172.21.15.110:0/3334375481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009900 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e81869640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e78032000 tx=0x7f8e7802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e78030670 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e78030c90 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e7803a660 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c19add0 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.324+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c19b2e0 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.325+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c0ffc40 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.325+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8e7800a040 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.326+0000 7f8e72ffd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 0x7f8e6806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.326+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8e7802c040 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.326+0000 7f8e81068640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 0x7f8e6806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:35.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.326+0000 7f8e81068640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 0x7f8e6806f120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005fd0 tx=0x7f8e6c005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:35.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.330+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8e78077d10 con 0x7f8e7c0ff7c0 2024-09-21T04:12:35.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:35 smithi110 ceph-mon[27723]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:35.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.499+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e7c02c430 con 0x7f8e6806cc60 2024-09-21T04:12:35.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.500+0000 7f8e72ffd640 1 -- 172.21.15.110:0/3334375481 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f8e7c02c430 con 0x7f8e6806cc60 2024-09-21T04:12:35.503 INFO:teuthology.orchestra.run.smithi110.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T04:12:35.503 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-21T04:12:35.503 INFO:teuthology.orchestra.run.smithi110.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 74s ago Has a FileSystem, LVM detected 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 msgr2=0x7f8e6806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 0x7f8e6806f120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005fd0 tx=0x7f8e6c005850 comp rx=0 tx=0).stop 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 msgr2=0x7f8e7c195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e78032000 tx=0x7f8e7802f5f0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 shutdown_connections 2024-09-21T04:12:35.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e7c101280 0x7f8e7c195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8e6806cc60 0x7f8e6806f120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 --2- 172.21.15.110:0/3334375481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e7c0ff7c0 0x7f8e7c195710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:35.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 >> 172.21.15.110:0/3334375481 conn(0x7f8e7c0fb330 msgr2=0x7f8e7c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:35.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 shutdown_connections 2024-09-21T04:12:35.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:35.502+0000 7f8e83acd640 1 -- 172.21.15.110:0/3334375481 wait complete. 2024-09-21T04:12:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:35 smithi148 ceph-mon[34957]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:36.127 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:36.140 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:36.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-21T04:12:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:36.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:36 smithi110 ceph-mon[27723]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:36.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:36 smithi148 ceph-mon[34957]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:37.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:37 smithi110 ceph-mon[27723]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:37 smithi148 ceph-mon[34957]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 -- 172.21.15.110:0/134662538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19740ffb70 msgr2=0x7f19740fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 --2- 172.21.15.110:0/134662538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19740ffb70 0x7f19740fff70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1968009880 tx=0x7f196802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 -- 172.21.15.110:0/134662538 shutdown_connections 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 --2- 172.21.15.110:0/134662538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1974100d70 0x7f19741011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 --2- 172.21.15.110:0/134662538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19740ffb70 0x7f19740fff70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 -- 172.21.15.110:0/134662538 >> 172.21.15.110:0/134662538 conn(0x7f19740fb2e0 msgr2=0x7f19740fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 -- 172.21.15.110:0/134662538 shutdown_connections 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.947+0000 7f197ae4b640 1 -- 172.21.15.110:0/134662538 wait complete. 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 Processor -- start 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 -- start start 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740ffb70 0x7f1974074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:37.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974072de0 con 0x7f1974100d70 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f197ae4b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974072f50 con 0x7f19740ffb70 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f1973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f1973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55730/0 (socket says 172.21.15.110:55730) 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f1973fff640 1 -- 172.21.15.110:0/1936773127 learned_addr learned my addr 172.21.15.110:0/1936773127 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.948+0000 7f1978be7640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740ffb70 0x7f1974074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1973fff640 1 -- 172.21.15.110:0/1936773127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740ffb70 msgr2=0x7f1974074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1973fff640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740ffb70 0x7f1974074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1973fff640 1 -- 172.21.15.110:0/1936773127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009560 con 0x7f1974100d70 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1973fff640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f196000d6f0 tx=0x7f196000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1960009a30 con 0x7f1974100d70 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1974073230 con 0x7f1974100d70 2024-09-21T04:12:37.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19600054e0 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1960010460 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.949+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197419d6d0 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.950+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197418f710 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.951+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f1960009b90 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.951+0000 7f1971ffb640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 0x7f196406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.951+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f196008bac0 con 0x7f1974100d70 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.951+0000 7f1978be7640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 0x7f196406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.952+0000 7f1978be7640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 0x7f196406f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1968009230 tx=0x7f1968005d50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:37.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:37.956+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f19600561d0 con 0x7f1974100d70 2024-09-21T04:12:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:38.139+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f197402c450 con 0x7f196406cc50 2024-09-21T04:12:38.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:38 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-21T04:12:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:38 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-21T04:12:39.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.458+0000 7f1971ffb640 1 -- 172.21.15.110:0/1936773127 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f197402c450 con 0x7f196406cc50 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.461+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 msgr2=0x7f196406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.461+0000 7f197ae4b640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 0x7f196406f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1968009230 tx=0x7f1968005d50 comp rx=0 tx=0).stop 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.461+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 msgr2=0x7f19740728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.461+0000 7f197ae4b640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f196000d6f0 tx=0x7f196000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.461+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 shutdown_connections 2024-09-21T04:12:39.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f196406cc50 0x7f196406f110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:39.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974100d70 0x7f19740728a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:39.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 --2- 172.21.15.110:0/1936773127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740ffb70 0x7f1974074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:39.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 >> 172.21.15.110:0/1936773127 conn(0x7f19740fb2e0 msgr2=0x7f19741044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:39.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 shutdown_connections 2024-09-21T04:12:39.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:39.462+0000 7f197ae4b640 1 -- 172.21.15.110:0/1936773127 wait complete. 2024-09-21T04:12:39.797 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[27719]: 2024-09-21T04:12:39.436+0000 7fed937e7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:12:39.797 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: osdmap e42: 6 total, 6 up, 6 in 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: osdmap e43: 6 total, 6 up, 6 in 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: fsmap cephfs:0 2024-09-21T04:12:39.798 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: Saving service mds.cephfs spec with placement count:4 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:39.799 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:39 smithi110 ceph-mon[27723]: Deploying daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:12:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:12:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: osdmap e42: 6 total, 6 up, 6 in 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: osdmap e43: 6 total, 6 up, 6 in 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: fsmap cephfs:0 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: Saving service mds.cephfs spec with placement count:4 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:39.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:39 smithi148 ceph-mon[34957]: Deploying daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:12:40.102 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:12:40.843 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:41.711 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:41 smithi110 ceph-mon[27723]: pgmap v192: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:41.711 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:41 smithi110 ceph-mon[27723]: osdmap e44: 6 total, 6 up, 6 in 2024-09-21T04:12:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:41 smithi148 ceph-mon[34957]: pgmap v192: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:41.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:41 smithi148 ceph-mon[34957]: osdmap e44: 6 total, 6 up, 6 in 2024-09-21T04:12:42.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:42 smithi110 ceph-mon[27723]: osdmap e45: 6 total, 6 up, 6 in 2024-09-21T04:12:42.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:42 smithi148 ceph-mon[34957]: osdmap e45: 6 total, 6 up, 6 in 2024-09-21T04:12:43.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/2548493327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf480ffb70 msgr2=0x7fbf480fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/2548493327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf480ffb70 0x7fbf480fff70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009880 tx=0x7fbf3802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/2548493327 shutdown_connections 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/2548493327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf48100d70 0x7fbf481011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/2548493327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf480ffb70 0x7fbf480fff70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/2548493327 >> 172.21.15.110:0/2548493327 conn(0x7fbf480fb2e0 msgr2=0x7fbf480fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/2548493327 shutdown_connections 2024-09-21T04:12:43.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.257+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/2548493327 wait complete. 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 Processor -- start 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 -- start start 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf480ffb70 0x7fbf48074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf48072de0 con 0x7fbf48100d70 2024-09-21T04:12:43.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.258+0000 7fbf4cd6e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf48072f50 con 0x7fbf480ffb70 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46584/0 (socket says 172.21.15.110:46584) 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf4659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf480ffb70 0x7fbf48074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 -- 172.21.15.110:0/317974970 learned_addr learned my addr 172.21.15.110:0/317974970 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 -- 172.21.15.110:0/317974970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf480ffb70 msgr2=0x7fbf48074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf480ffb70 0x7fbf48074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 -- 172.21.15.110:0/317974970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf38009560 con 0x7fbf48100d70 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf45d9b640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c810 tx=0x7fbf3c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:43.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c0041f0 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf3c007e70 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf48073230 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c0049d0 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.259+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4819d680 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.260+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf48108760 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.261+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fbf3c004b30 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.261+0000 7fbf377fe640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 0x7fbf3006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.261+0000 7fbf4659c640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 0x7fbf3006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.262+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbf3c08bfd0 con 0x7fbf48100d70 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.262+0000 7fbf4659c640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 0x7fbf3006f110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009230 tx=0x7fbf380314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:43.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.266+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbf3c0bfe20 con 0x7fbf48100d70 2024-09-21T04:12:43.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.475+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf481088f0 con 0x7fbf48100d70 2024-09-21T04:12:43.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.476+0000 7fbf377fe640 1 -- 172.21.15.110:0/317974970 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1287 (secure 0 0 0) 0x7fbf3c01d070 con 0x7fbf48100d70 2024-09-21T04:12:43.479 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 4 2024-09-21T04:12:43.479 INFO:teuthology.orchestra.run.smithi110.stdout:e4 2024-09-21T04:12:43.479 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 4 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:43.458699+0000 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:12:43.480 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14542} 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:12:43.481 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:12:43.482 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:12:43.482 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 0 2024-09-21T04:12:43.482 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:14542} state up:creating seq 1 addr [v2:172.21.15.110:6826/1662909837,v1:172.21.15.110:6827/1662909837] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:12:43.482 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:43.482 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 msgr2=0x7fbf3006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 0x7fbf3006f110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009230 tx=0x7fbf380314e0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 msgr2=0x7fbf480728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c810 tx=0x7fbf3c00ccd0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 shutdown_connections 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fbf3006cc50 0x7fbf3006f110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100d70 0x7fbf480728a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 --2- 172.21.15.110:0/317974970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf480ffb70 0x7fbf48074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 >> 172.21.15.110:0/317974970 conn(0x7fbf480fb2e0 msgr2=0x7fbf48103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:43.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 shutdown_connections 2024-09-21T04:12:43.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:43.479+0000 7fbf4cd6e640 1 -- 172.21.15.110:0/317974970 wait complete. 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: pgmap v195: 65 pgs: 27 unknown, 38 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:43.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:43.803 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: pgmap v195: 65 pgs: 27 unknown, 38 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:43.804 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:44.137 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:44.147 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:44.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-09-21T04:12:44.349 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: Deploying daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6826/1662909837,v1:172.21.15.110:6827/1662909837] up:boot 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: daemon mds.cephfs.smithi110.trhyxr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: Cluster is now healthy 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: fsmap cephfs:0 1 up:standby 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:12:44.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:creating} 2024-09-21T04:12:44.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: daemon mds.cephfs.smithi110.trhyxr is now active in filesystem cephfs as rank 0 2024-09-21T04:12:44.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:44 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/317974970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: Deploying daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6826/1662909837,v1:172.21.15.110:6827/1662909837] up:boot 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: daemon mds.cephfs.smithi110.trhyxr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: Cluster is now healthy 2024-09-21T04:12:44.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: fsmap cephfs:0 1 up:standby 2024-09-21T04:12:44.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:12:44.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:creating} 2024-09-21T04:12:44.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: daemon mds.cephfs.smithi110.trhyxr is now active in filesystem cephfs as rank 0 2024-09-21T04:12:44.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:44 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/317974970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: pgmap v196: 65 pgs: 65 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6826/1662909837,v1:172.21.15.110:6827/1662909837] up:active 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:45.596 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:45.596 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:45 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: pgmap v196: 65 pgs: 65 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6826/1662909837,v1:172.21.15.110:6827/1662909837] up:active 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:45 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:45.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 -- 172.21.15.110:0/1935514590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20102fb0 msgr2=0x7f4b201053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:45.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 --2- 172.21.15.110:0/1935514590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20102fb0 0x7f4b201053a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009880 tx=0x7f4b0802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:45.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 -- 172.21.15.110:0/1935514590 shutdown_connections 2024-09-21T04:12:45.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 --2- 172.21.15.110:0/1935514590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20102fb0 0x7f4b201053a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:45.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 --2- 172.21.15.110:0/1935514590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20100680 0x7f4b20102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 -- 172.21.15.110:0/1935514590 >> 172.21.15.110:0/1935514590 conn(0x7f4b200fa1c0 msgr2=0x7f4b200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 -- 172.21.15.110:0/1935514590 shutdown_connections 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.916+0000 7f4b25f49640 1 -- 172.21.15.110:0/1935514590 wait complete. 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 Processor -- start 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 -- start start 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20102fb0 0x7f4b20195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20196230 con 0x7f4b20100680 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b25f49640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201963a0 con 0x7f4b20102fb0 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46608/0 (socket says 172.21.15.110:46608) 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 -- 172.21.15.110:0/201302573 learned_addr learned my addr 172.21.15.110:0/201302573 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 -- 172.21.15.110:0/201302573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20102fb0 msgr2=0x7f4b20195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b17fff640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20102fb0 0x7f4b20195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20102fb0 0x7f4b20195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 -- 172.21.15.110:0/201302573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08009560 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.917+0000 7f4b1f7fe640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000b450 tx=0x7f4b1000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.918+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b10011860 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.918+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2019ae40 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.918+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b10011e80 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.918+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1000f570 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.918+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2019b3b0 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.919+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b201087a0 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.919+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f4b100103e0 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.920+0000 7f4b1d7fa640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 0x7f4b0c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.920+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4b1008c160 con 0x7f4b20100680 2024-09-21T04:12:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.920+0000 7f4b17fff640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 0x7f4b0c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:45.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.920+0000 7f4b17fff640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 0x7f4b0c06f110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b08031500 tx=0x7f4b08000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:45.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:45.924+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4b10056530 con 0x7f4b20100680 2024-09-21T04:12:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.134+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f4b2019b630 con 0x7f4b20100680 2024-09-21T04:12:46.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.477+0000 7f4b1d7fa640 1 -- 172.21.15.110:0/201302573 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4b10059b50 con 0x7f4b20100680 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 msgr2=0x7f4b0c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 0x7f4b0c06f110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b08031500 tx=0x7f4b08000bc0 comp rx=0 tx=0).stop 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 msgr2=0x7f4b20195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000b450 tx=0x7f4b1000b910 comp rx=0 tx=0).stop 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 shutdown_connections 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b20102fb0 0x7f4b20195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b0c06cc50 0x7f4b0c06f110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 --2- 172.21.15.110:0/201302573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b20100680 0x7f4b20195720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 >> 172.21.15.110:0/201302573 conn(0x7f4b200fa1c0 msgr2=0x7f4b200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 shutdown_connections 2024-09-21T04:12:46.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:46.480+0000 7f4b25f49640 1 -- 172.21.15.110:0/201302573 wait complete. 2024-09-21T04:12:46.711 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: Deploying daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:12:46.711 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:boot 2024-09-21T04:12:46.711 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:46.711 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:12:46.711 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:46.712 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:46 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/201302573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: Deploying daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:boot 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:46 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/201302573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-09-21T04:12:47.234 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:47.244 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:47.244 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-09-21T04:12:47.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:47 smithi110 ceph-mon[27723]: pgmap v197: 65 pgs: 65 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:47.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:47 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/201302573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-09-21T04:12:47.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:47 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:47.799 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:47 smithi148 ceph-mon[34957]: pgmap v197: 65 pgs: 65 active+clean; 577 KiB data, 210 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:12:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:47 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/201302573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-09-21T04:12:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:47 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 1 up:standby 2024-09-21T04:12:48.728 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:12:49.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- 172.21.15.110:0/1389328612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0ffc30 msgr2=0x7f76ec1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 --2- 172.21.15.110:0/1389328612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0ffc30 0x7f76ec1000b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009880 tx=0x7f76d402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- 172.21.15.110:0/1389328612 shutdown_connections 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 --2- 172.21.15.110:0/1389328612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0ffc30 0x7f76ec1000b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 --2- 172.21.15.110:0/1389328612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0fea30 0x7f76ec0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- 172.21.15.110:0/1389328612 >> 172.21.15.110:0/1389328612 conn(0x7f76ec0fa1c0 msgr2=0x7f76ec0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- 172.21.15.110:0/1389328612 shutdown_connections 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- 172.21.15.110:0/1389328612 wait complete. 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 Processor -- start 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.514+0000 7f76f2226640 1 -- start start 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76f2226640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76f2226640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 0x7f76ec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76f2226640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec19aa70 con 0x7f76ec0fea30 2024-09-21T04:12:49.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76f2226640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec19abb0 con 0x7f76ec0ffc30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46626/0 (socket says 172.21.15.110:46626) 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 -- 172.21.15.110:0/3884985499 learned_addr learned my addr 172.21.15.110:0/3884985499 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76e2ffd640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 0x7f76ec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 -- 172.21.15.110:0/3884985499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 msgr2=0x7f76ec19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 0x7f76ec19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 -- 172.21.15.110:0/3884985499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009560 con 0x7f76ec0fea30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76e2ffd640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 0x7f76ec19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76eb7fe640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b3d0 tx=0x7f76dc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76dc011800 con 0x7f76ec0fea30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76dc011e20 con 0x7f76ec0fea30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec19ae60 con 0x7f76ec0fea30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.515+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76dc00f510 con 0x7f76ec0fea30 2024-09-21T04:12:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.516+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec19b3a0 con 0x7f76ec0fea30 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.517+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ec0feeb0 con 0x7f76ec0fea30 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.517+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f76dc010380 con 0x7f76ec0fea30 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.517+0000 7f76e97fa640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 0x7f76d806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.518+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f76dc08bf10 con 0x7f76ec0fea30 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.518+0000 7f76e2ffd640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 0x7f76d806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.518+0000 7f76e2ffd640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 0x7f76d806f110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d40314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:49.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.522+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f76dc0562e0 con 0x7f76ec0fea30 2024-09-21T04:12:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:49 smithi110 ceph-mon[27723]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T04:12:49.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:49.737+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f76ec19b650 con 0x7f76ec0fea30 2024-09-21T04:12:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:49 smithi148 ceph-mon[34957]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T04:12:50.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.490+0000 7f76e97fa640 1 -- 172.21.15.110:0/3884985499 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76dc059900 con 0x7f76ec0fea30 2024-09-21T04:12:50.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 msgr2=0x7f76d806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 0x7f76d806f110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d40314e0 comp rx=0 tx=0).stop 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 msgr2=0x7f76ec068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b3d0 tx=0x7f76dc00b890 comp rx=0 tx=0).stop 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 shutdown_connections 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76ec0ffc30 0x7f76ec19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f76d806cc50 0x7f76d806f110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 --2- 172.21.15.110:0/3884985499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76ec0fea30 0x7f76ec068740 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 >> 172.21.15.110:0/3884985499 conn(0x7f76ec0fa1c0 msgr2=0x7f76ec102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 shutdown_connections 2024-09-21T04:12:50.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:50.493+0000 7f76f2226640 1 -- 172.21.15.110:0/3884985499 wait complete. 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3884985499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:50.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:50 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3884985499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-21T04:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:50.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:12:50.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:12:50.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:50 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:12:51.119 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:51.130 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:51.130 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs inline_data false' 2024-09-21T04:12:51.331 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: Deploying daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3884985499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:boot 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2 up:standby 2024-09-21T04:12:51.743 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:51 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:12:51.812 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: Deploying daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:12:51.812 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-21T04:12:51.812 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3884985499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-21T04:12:51.812 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:boot 2024-09-21T04:12:51.812 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2 up:standby 2024-09-21T04:12:51.813 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:51 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 -- 172.21.15.110:0/81229438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de80fe420 msgr2=0x7f0de8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 --2- 172.21.15.110:0/81229438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de80fe420 0x7f0de8100830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009880 tx=0x7f0de402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 -- 172.21.15.110:0/81229438 shutdown_connections 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 --2- 172.21.15.110:0/81229438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de8100d70 0x7f0de81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 --2- 172.21.15.110:0/81229438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de80fe420 0x7f0de8100830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 -- 172.21.15.110:0/81229438 >> 172.21.15.110:0/81229438 conn(0x7f0de80fa230 msgr2=0x7f0de80fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 -- 172.21.15.110:0/81229438 shutdown_connections 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.989+0000 7f0def9a6640 1 -- 172.21.15.110:0/81229438 wait complete. 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 Processor -- start 2024-09-21T04:12:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 -- start start 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 0x7f0de819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819c6d0 con 0x7f0de8100d70 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0def9a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819c840 con 0x7f0de80fe420 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0decf41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0decf41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39238/0 (socket says 172.21.15.110:39238) 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0decf41640 1 -- 172.21.15.110:0/3970358579 learned_addr learned my addr 172.21.15.110:0/3970358579 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:53.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.990+0000 7f0ded742640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 0x7f0de819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0decf41640 1 -- 172.21.15.110:0/3970358579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 msgr2=0x7f0de819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0decf41640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 0x7f0de819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0decf41640 1 -- 172.21.15.110:0/3970358579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de4009560 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0ded742640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 0x7f0de819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0decf41640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d900 tx=0x7f0dd800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dd8009c00 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de81a12e0 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dd8009d60 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dd80049d0 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.991+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81a1830 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.992+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de810ed70 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.992+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f0dd8004020 con 0x7f0de8100d70 2024-09-21T04:12:53.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.993+0000 7f0dde7fc640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 0x7f0dd406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:53.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.993+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0dd808bfb0 con 0x7f0de8100d70 2024-09-21T04:12:53.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.993+0000 7f0ded742640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 0x7f0dd406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:53.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.993+0000 7f0ded742640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 0x7f0dd406f110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000c00 tx=0x7f0de40314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:53.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:52.998+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0dd8056380 con 0x7f0de8100d70 2024-09-21T04:12:53.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:53.215+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f0de81a1ab0 con 0x7f0de8100d70 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:boot 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:53.828 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.829 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:53 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3970358579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:boot 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:12:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:53.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:53 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3970358579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-09-21T04:12:54.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.193+0000 7f0dde7fc640 1 -- 172.21.15.110:0/3970358579 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0dd80599a0 con 0x7f0de8100d70 2024-09-21T04:12:54.196 INFO:teuthology.orchestra.run.smithi110.stderr:inline data disabled 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 msgr2=0x7f0dd406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 0x7f0dd406f110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000c00 tx=0x7f0de40314e0 comp rx=0 tx=0).stop 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 msgr2=0x7f0de819c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d900 tx=0x7f0dd800ddc0 comp rx=0 tx=0).stop 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 shutdown_connections 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0dd406cc50 0x7f0dd406f110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:54.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0de8100d70 0x7f0de819c100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:54.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 --2- 172.21.15.110:0/3970358579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0de80fe420 0x7f0de819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:54.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.196+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 >> 172.21.15.110:0/3970358579 conn(0x7f0de80fa230 msgr2=0x7f0de80fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:54.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.197+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 shutdown_connections 2024-09-21T04:12:54.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:54.197+0000 7f0def9a6640 1 -- 172.21.15.110:0/3970358579 wait complete. 2024-09-21T04:12:54.949 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:12:54.959 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:12:54.959 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:12:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-21T04:12:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3970358579' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-09-21T04:12:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:standby 2024-09-21T04:12:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:standby 2024-09-21T04:12:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: Dropping low affinity active daemon mds.cephfs.smithi110.trhyxr in favor of higher affinity standby. 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: Replacing daemon mds.cephfs.smithi110.trhyxr as rank 0 with standby daemon mds.cephfs.smithi110.faxwju 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:replay} 2 up:standby 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:55.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:55 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3970358579' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:standby 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:standby 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: Dropping low affinity active daemon mds.cephfs.smithi110.trhyxr in favor of higher affinity standby. 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: Replacing daemon mds.cephfs.smithi110.trhyxr as rank 0 with standby daemon mds.cephfs.smithi110.faxwju 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:replay} 2 up:standby 2024-09-21T04:12:55.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:55.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:55 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:12:55.738 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:12:56.502 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:56 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:reconnect 2024-09-21T04:12:56.502 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:56 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] up:boot 2024-09-21T04:12:56.503 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:56 smithi110 ceph-mon[27723]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:reconnect} 3 up:standby 2024-09-21T04:12:56.503 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:56 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:12:56.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:56 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:reconnect 2024-09-21T04:12:56.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:56 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] up:boot 2024-09-21T04:12:56.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:56 smithi148 ceph-mon[34957]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:reconnect} 3 up:standby 2024-09-21T04:12:56.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:56 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:rejoin 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:rejoin} 3 up:standby 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: daemon mds.cephfs.smithi110.faxwju is now active in filesystem cephfs as rank 0 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:12:57.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:57 smithi110 ceph-mon[27723]: Cluster is now healthy 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:rejoin 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: fsmap cephfs:1/1 {0=cephfs.smithi110.faxwju=up:rejoin} 3 up:standby 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: daemon mds.cephfs.smithi110.faxwju is now active in filesystem cephfs as rank 0 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:12:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:57 smithi148 ceph-mon[34957]: Cluster is now healthy 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.272+0000 7f46caf11640 1 -- 172.21.15.110:0/6404201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40fea80 msgr2=0x7f46c40fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.272+0000 7f46caf11640 1 --2- 172.21.15.110:0/6404201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40fea80 0x7f46c40fee80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009880 tx=0x7f46b802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- 172.21.15.110:0/6404201 shutdown_connections 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 --2- 172.21.15.110:0/6404201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40ffc80 0x7f46c4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 --2- 172.21.15.110:0/6404201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40fea80 0x7f46c40fee80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- 172.21.15.110:0/6404201 >> 172.21.15.110:0/6404201 conn(0x7f46c40fa230 msgr2=0x7f46c40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- 172.21.15.110:0/6404201 shutdown_connections 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- 172.21.15.110:0/6404201 wait complete. 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 Processor -- start 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- start start 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40fea80 0x7f46c4068f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c419a920 con 0x7f46c40ffc80 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46caf11640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c419aa60 con 0x7f46c40fea80 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39270/0 (socket says 172.21.15.110:39270) 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.273+0000 7f46c3fff640 1 -- 172.21.15.110:0/3349127152 learned_addr learned my addr 172.21.15.110:0/3349127152 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c8cad640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40fea80 0x7f46c4068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c3fff640 1 -- 172.21.15.110:0/3349127152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40fea80 msgr2=0x7f46c4068f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c3fff640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40fea80 0x7f46c4068f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c3fff640 1 -- 172.21.15.110:0/3349127152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009560 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c3fff640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46b000d6f0 tx=0x7f46b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b0009a30 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46b00054e0 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c419acf0 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b0010460 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.274+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c419b2c0 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.275+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c418e620 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.276+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f46b0009b90 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.276+0000 7f46c1ffb640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 0x7f46b406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.276+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f46b008be40 con 0x7f46c40ffc80 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.276+0000 7f46c8cad640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 0x7f46b406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.277+0000 7f46c8cad640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 0x7f46b406f110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009230 tx=0x7f46b8005d50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.280+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f46b00561b0 con 0x7f46c40ffc80 2024-09-21T04:12:58.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.486+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46c4107690 con 0x7f46c40ffc80 2024-09-21T04:12:58.489 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:58 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:active 2024-09-21T04:12:58.489 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:58 smithi110 ceph-mon[27723]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:standby 2024-09-21T04:12:58.489 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:58 smithi110 ceph-mon[27723]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:12:58.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.486+0000 7f46c1ffb640 1 -- 172.21.15.110:0/3349127152 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f46b0020750 con 0x7f46c40ffc80 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:12:58.490 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:12:58.491 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:12:58.492 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:12:58.493 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:12:58.493 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:12:58.493 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 msgr2=0x7f46b406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 0x7f46b406f110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009230 tx=0x7f46b8005d50 comp rx=0 tx=0).stop 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 msgr2=0x7f46c419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46b000d6f0 tx=0x7f46b000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 shutdown_connections 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f46b406cc50 0x7f46b406f110 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c40ffc80 0x7f46c419a350 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 --2- 172.21.15.110:0/3349127152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c40fea80 0x7f46c4068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 >> 172.21.15.110:0/3349127152 conn(0x7f46c40fa230 msgr2=0x7f46c4103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 shutdown_connections 2024-09-21T04:12:58.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:12:58.489+0000 7f46caf11640 1 -- 172.21.15.110:0/3349127152 wait complete. 2024-09-21T04:12:58.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:58 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] up:active 2024-09-21T04:12:58.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:58 smithi148 ceph-mon[34957]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:standby 2024-09-21T04:12:58.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:58 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:12:59.345 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:59 smithi110 ceph-mon[27723]: pgmap v204: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 204 B/s wr, 3 op/s 2024-09-21T04:12:59.345 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:12:59 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3349127152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:12:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:59 smithi148 ceph-mon[34957]: pgmap v204: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 204 B/s wr, 3 op/s 2024-09-21T04:12:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:12:59 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3349127152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:12:59.758 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-21T04:12:59.972 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:00 smithi148 ceph-mon[34957]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 3 op/s 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:01.213 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:00 smithi110 ceph-mon[27723]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 3 op/s 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 -- 172.21.15.110:0/1027359523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 msgr2=0x7fd15c100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 --2- 172.21.15.110:0/1027359523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c100120 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd150009880 tx=0x7fd15002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 -- 172.21.15.110:0/1027359523 shutdown_connections 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 --2- 172.21.15.110:0/1027359523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c100120 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 --2- 172.21.15.110:0/1027359523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 0x7fd15c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 -- 172.21.15.110:0/1027359523 >> 172.21.15.110:0/1027359523 conn(0x7fd15c0fa230 msgr2=0x7fd15c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 -- 172.21.15.110:0/1027359523 shutdown_connections 2024-09-21T04:13:01.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.549+0000 7fd16095b640 1 -- 172.21.15.110:0/1027359523 wait complete. 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 Processor -- start 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 -- start start 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 0x7fd15c1944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c195000 con 0x7fd15c0ffca0 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.550+0000 7fd16095b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c195170 con 0x7fd15c0feaa0 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39294/0 (socket says 172.21.15.110:39294) 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 -- 172.21.15.110:0/1258646076 learned_addr learned my addr 172.21.15.110:0/1258646076 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd15a59c640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 0x7fd15c1944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 -- 172.21.15.110:0/1258646076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 msgr2=0x7fd15c1944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 0x7fd15c1944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 -- 172.21.15.110:0/1258646076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150009560 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd159d9b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd150005520 tx=0x7fd150005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd150004aa0 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c199bb0 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd150004c00 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd150003b10 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.551+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c19a0a0 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.552+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15c107690 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.553+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fd150003d40 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.553+0000 7fd14b7fe640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 0x7fd14406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.553+0000 7fd15a59c640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 0x7fd14406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.553+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd1500ad3f0 con 0x7fd15c0ffca0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.554+0000 7fd15a59c640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 0x7fd14406f110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.558+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd150077810 con 0x7fd15c0ffca0 2024-09-21T04:13:01.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.758+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd15c19a2f0 con 0x7fd15c0ffca0 2024-09-21T04:13:01.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.759+0000 7fd14b7fe640 1 -- 172.21.15.110:0/1258646076 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4831 (secure 0 0 0) 0x7fd150081070 con 0x7fd15c0ffca0 2024-09-21T04:13:01.761 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:13:01.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 msgr2=0x7fd14406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 0x7fd14406f110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c00b410 comp rx=0 tx=0).stop 2024-09-21T04:13:01.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 msgr2=0x7fd15c194a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd150005520 tx=0x7fd150005550 comp rx=0 tx=0).stop 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 shutdown_connections 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fd14406cc50 0x7fd14406f110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd15c0ffca0 0x7fd15c194a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 --2- 172.21.15.110:0/1258646076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd15c0feaa0 0x7fd15c1944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 >> 172.21.15.110:0/1258646076 conn(0x7fd15c0fa230 msgr2=0x7fd15c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 shutdown_connections 2024-09-21T04:13:01.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:01.761+0000 7fd16095b640 1 -- 172.21.15.110:0/1258646076 wait complete. 2024-09-21T04:13:01.772 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:13:02.033 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:01 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1258646076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:13:02.034 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:01 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:02.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:01 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1258646076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:13:02.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:01 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:02.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-21T04:13:03.061 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:03.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:02 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:02 smithi110 ceph-mon[27723]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 7 op/s 2024-09-21T04:13:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:02 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:02 smithi148 ceph-mon[34957]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 7 op/s 2024-09-21T04:13:04.120 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:05.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:04 smithi110 ceph-mon[27723]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T04:13:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:04 smithi148 ceph-mon[34957]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T04:13:06.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 -- 172.21.15.110:0/1612534237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 msgr2=0x7fc08c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 --2- 172.21.15.110:0/1612534237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c102a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098b0 tx=0x7fc07c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 -- 172.21.15.110:0/1612534237 shutdown_connections 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 --2- 172.21.15.110:0/1612534237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 0x7fc08c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 --2- 172.21.15.110:0/1612534237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c102a20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 -- 172.21.15.110:0/1612534237 >> 172.21.15.110:0/1612534237 conn(0x7fc08c0fa230 msgr2=0x7fc08c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 -- 172.21.15.110:0/1612534237 shutdown_connections 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.478+0000 7fc0924e6640 1 -- 172.21.15.110:0/1612534237 wait complete. 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 Processor -- start 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 -- start start 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 0x7fc08c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1960c0 con 0x7fc08c100630 2024-09-21T04:13:06.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc0924e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c196230 con 0x7fc08c102f60 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48942/0 (socket says 172.21.15.110:48942) 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 -- 172.21.15.110:0/879759234 learned_addr learned my addr 172.21.15.110:0/879759234 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08b7fe640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 0x7fc08c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 -- 172.21.15.110:0/879759234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 msgr2=0x7fc08c195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 0x7fc08c195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 -- 172.21.15.110:0/879759234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009560 con 0x7fc08c100630 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.479+0000 7fc08bfff640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.480+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07c004b20 con 0x7fc08c100630 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.480+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc07c004c80 con 0x7fc08c100630 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.480+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c19acb0 con 0x7fc08c100630 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.480+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07c003b80 con 0x7fc08c100630 2024-09-21T04:13:06.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.480+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c19b150 con 0x7fc08c100630 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.481+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c1087c0 con 0x7fc08c100630 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.481+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fc07c003ce0 con 0x7fc08c100630 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.482+0000 7fc0897fa640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 0x7fc07406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.482+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc07c0ad3f0 con 0x7fc08c100630 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.482+0000 7fc08b7fe640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 0x7fc07406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.482+0000 7fc08b7fe640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 0x7fc07406f110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc08c196b60 tx=0x7fc08000a2d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:06.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.486+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc07c077760 con 0x7fc08c100630 2024-09-21T04:13:06.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.736+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fc08c1089a0 con 0x7fc08c100630 2024-09-21T04:13:06.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.737+0000 7fc0897fa640 1 -- 172.21.15.110:0/879759234 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+97 (secure 0 0 0) 0x7fc07c081070 con 0x7fc08c100630 2024-09-21T04:13:06.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 msgr2=0x7fc07406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 0x7fc07406f110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc08c196b60 tx=0x7fc08000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 msgr2=0x7fc08c195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006010 tx=0x7fc07c005610 comp rx=0 tx=0).stop 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 shutdown_connections 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc08c102f60 0x7fc08c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fc07406cc50 0x7fc07406f110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 --2- 172.21.15.110:0/879759234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc08c100630 0x7fc08c195640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 >> 172.21.15.110:0/879759234 conn(0x7fc08c0fa230 msgr2=0x7fc08c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 shutdown_connections 2024-09-21T04:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:06.739+0000 7fc0924e6640 1 -- 172.21.15.110:0/879759234 wait complete. 2024-09-21T04:13:06.750 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:13:07.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:07 smithi110 ceph-mon[27723]: pgmap v208: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T04:13:07.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:07 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/879759234' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-21T04:13:07.463 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-21T04:13:08.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:13:08.589 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:07 smithi148 ceph-mon[34957]: pgmap v208: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T04:13:08.589 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:07 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/879759234' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-21T04:13:08.788 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:09.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: pgmap v209: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T04:13:09.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:09.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:09.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:09.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:09.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:09 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: pgmap v209: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:09 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 -- 172.21.15.110:0/2511483758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7050105bf0 msgr2=0x7f7050105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 --2- 172.21.15.110:0/2511483758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7050105bf0 0x7f7050105fd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7040009880 tx=0x7f704002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 -- 172.21.15.110:0/2511483758 shutdown_connections 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 --2- 172.21.15.110:0/2511483758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70500ffb40 0x7f70500fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 --2- 172.21.15.110:0/2511483758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7050105bf0 0x7f7050105fd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.386+0000 7f705546e640 1 -- 172.21.15.110:0/2511483758 >> 172.21.15.110:0/2511483758 conn(0x7f70500fb840 msgr2=0x7f70500fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 -- 172.21.15.110:0/2511483758 shutdown_connections 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 -- 172.21.15.110:0/2511483758 wait complete. 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 Processor -- start 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 -- start start 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050105bf0 0x7f7050072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.387+0000 7f705546e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050073230 con 0x7f70500ffb40 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f705546e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70500733a0 con 0x7f7050105bf0 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48972/0 (socket says 172.21.15.110:48972) 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 -- 172.21.15.110:0/3425514812 learned_addr learned my addr 172.21.15.110:0/3425514812 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704e7fc640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050105bf0 0x7f7050072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 -- 172.21.15.110:0/3425514812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050105bf0 msgr2=0x7f7050072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050105bf0 0x7f7050072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 -- 172.21.15.110:0/3425514812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009560 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f704effd640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f704002a9c0 tx=0x7f70400055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7040004aa0 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7040004c00 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7050073620 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7040003ab0 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.388+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501a2010 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.389+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70501011f0 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.390+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f7040031d20 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.390+0000 7f7047fff640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 0x7f703c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.390+0000 7f704e7fc640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 0x7f703c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.390+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f70400ad200 con 0x7f70500ffb40 2024-09-21T04:13:10.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.390+0000 7f704e7fc640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 0x7f703c06f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7050073f30 tx=0x7f7038008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.394+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7040077570 con 0x7f70500ffb40 2024-09-21T04:13:10.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.588+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7050073620 con 0x7f70500ffb40 2024-09-21T04:13:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.589+0000 7f7047fff640 1 -- 172.21.15.110:0/3425514812 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4831 (secure 0 0 0) 0x7f7050073620 con 0x7f70500ffb40 2024-09-21T04:13:10.593 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:13:10.593 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2135055413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2135055413},{"type":"v1","addr":"172.21.15.110:6827","nonce":2135055413}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:12:57.204430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"cephfs.smithi110.faxwju","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6829/1332815334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1332815334},{"type":"v1","addr":"172.21.15.110:6829","nonce":1332815334}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-21T04:13:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:13:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 msgr2=0x7f703c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 0x7f703c06f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7050073f30 tx=0x7f7038008040 comp rx=0 tx=0).stop 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 msgr2=0x7f70500727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f704002a9c0 tx=0x7f70400055a0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 shutdown_connections 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050105bf0 0x7f7050072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f703c06cc50 0x7f703c06f110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 --2- 172.21.15.110:0/3425514812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70500ffb40 0x7f70500727b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 >> 172.21.15.110:0/3425514812 conn(0x7f70500fb840 msgr2=0x7f70500fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 shutdown_connections 2024-09-21T04:13:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:10.591+0000 7f705546e640 1 -- 172.21.15.110:0/3425514812 wait complete. 2024-09-21T04:13:11.331 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-09-21T04:13:11.331 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T04:13:11.395 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T04:13:11.395 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-09-21T04:13:11.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:11.395 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-21T04:13:11.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:11.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-21T04:13:11.484 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-09-21T04:13:11.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:13:11.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:11.486 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T04:13:11.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:11.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T04:13:11.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:11 smithi148 ceph-mon[34957]: pgmap v210: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-21T04:13:11.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:11 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3425514812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:13:11.579 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-09-21T04:13:11.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:13:11.581 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T04:13:11.581 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T04:13:11.581 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:11.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T04:13:11.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:11 smithi110 ceph-mon[27723]: pgmap v210: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-21T04:13:11.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:11 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3425514812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:13:11.784 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:13.421 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:13 smithi110 ceph-mon[27723]: pgmap v211: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T04:13:13.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 -- 172.21.15.110:0/2116857821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 msgr2=0x7fcaec104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/2116857821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec104f20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009880 tx=0x7fcae002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 -- 172.21.15.110:0/2116857821 shutdown_connections 2024-09-21T04:13:13.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/2116857821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec0fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/2116857821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec104f20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 -- 172.21.15.110:0/2116857821 >> 172.21.15.110:0/2116857821 conn(0x7fcaec0f9e70 msgr2=0x7fcaec0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 -- 172.21.15.110:0/2116857821 shutdown_connections 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.418+0000 7fcaf33c2640 1 -- 172.21.15.110:0/2116857821 wait complete. 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 Processor -- start 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 -- start start 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec19c410 con 0x7fcaec104b40 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf33c2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1959a0 con 0x7fcaec0fdf60 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf095d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf095d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37042/0 (socket says 172.21.15.110:37042) 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.419+0000 7fcaf095d640 1 -- 172.21.15.110:0/1167589316 learned_addr learned my addr 172.21.15.110:0/1167589316 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf115e640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf095d640 1 -- 172.21.15.110:0/1167589316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 msgr2=0x7fcaec19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf095d640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf095d640 1 -- 172.21.15.110:0/1167589316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0009560 con 0x7fcaec104b40 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf115e640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf095d640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cc60 tx=0x7fcae800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae8004b20 con 0x7fcaec104b40 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec195c80 con 0x7fcaec104b40 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcae8004c80 con 0x7fcaec104b40 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae800f750 con 0x7fcaec104b40 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.420+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec1961d0 con 0x7fcaec104b40 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.421+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec101280 con 0x7fcaec104b40 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.422+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fcae800f9c0 con 0x7fcaec104b40 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.422+0000 7fcade7fc640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 0x7fcad806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.422+0000 7fcaf115e640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 0x7fcad806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.422+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcae808c450 con 0x7fcaec104b40 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.423+0000 7fcaf115e640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 0x7fcad806f110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae0005e60 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.427+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcae80567c0 con 0x7fcaec104b40 2024-09-21T04:13:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.626+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcaec1080f0 con 0x7fcaec104b40 2024-09-21T04:13:13.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.626+0000 7fcade7fc640 1 -- 172.21.15.110:0/1167589316 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fcae8059de0 con 0x7fcaec104b40 2024-09-21T04:13:13.629 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 msgr2=0x7fcad806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 0x7fcad806f110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae0005e60 comp rx=0 tx=0).stop 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 msgr2=0x7fcaec19bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cc60 tx=0x7fcae800a620 comp rx=0 tx=0).stop 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 shutdown_connections 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcad806cc50 0x7fcad806f110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaec104b40 0x7fcaec19bdf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 --2- 172.21.15.110:0/1167589316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaec0fdf60 0x7fcaec19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 >> 172.21.15.110:0/1167589316 conn(0x7fcaec0f9e70 msgr2=0x7fcaec0fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 shutdown_connections 2024-09-21T04:13:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:13.628+0000 7fcaf33c2640 1 -- 172.21.15.110:0/1167589316 wait complete. 2024-09-21T04:13:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:13 smithi148 ceph-mon[34957]: pgmap v211: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T04:13:14.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:14 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1167589316' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T04:13:14.402 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:13:14.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:14.403 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-21T04:13:14.459 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T04:13:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T04:13:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T04:13:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T04:13:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42235sec preferred_lft 42235sec 2024-09-21T04:13:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link noprefixroute 2024-09-21T04:13:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:14.461 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T04:13:14.461 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:14.461 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-21T04:13:14.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add name ceph-brx type bridge 2024-09-21T04:13:14.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr flush dev ceph-brx 2024-09-21T04:13:14.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set ceph-brx up 2024-09-21T04:13:14.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T04:13:14.462 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-21T04:13:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:14.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:14.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:14.672 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T04:13:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:14 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1167589316' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:13:14.697 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-21T04:13:14.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:14.699 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-21T04:13:14.770 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-21T04:13:14.770 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T04:13:14.770 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T04:13:14.771 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T04:13:14.771 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T04:13:14.771 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-21T04:13:14.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:15.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:15.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:15.130 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-21T04:13:15.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:15.147 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list-id 2024-09-21T04:13:15.203 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:15.203 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-21T04:13:15.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:13:15.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T04:13:15.203 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-21T04:13:15.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:15 smithi110 ceph-mon[27723]: pgmap v212: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-09-21T04:13:15.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:15.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:15.323 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T04:13:15.323 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:15.323 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-21T04:13:15.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T04:13:15.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T04:13:15.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T04:13:15.324 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T04:13:15.324 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T04:13:15.324 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-21T04:13:15.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:15.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:15.493 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:15.494 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-21T04:13:15.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-21T04:13:15.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T04:13:15.494 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-21T04:13:15.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:15.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:15.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:13:15.571 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:15.586 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:13:15.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:13:15.586 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:15.642 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T04:13:15.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:15.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T04:13:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:15 smithi148 ceph-mon[34957]: pgmap v212: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-09-21T04:13:15.740 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T04:13:15.741 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T04:13:15.741 INFO:teuthology.orchestra.run.smithi110.stdout:invalid new device string format 2024-09-21T04:13:15.854 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T04:13:15.854 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T04:13:15.855 INFO:teuthology.orchestra.run.smithi110.stdout:invalid new device string format 2024-09-21T04:13:15.855 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.148:3300" 2024-09-21T04:13:15.855 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T04:13:15.862 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T04:13:15.862 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:invalid new device string format 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.148:3300" 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:6789,172.21.15.148:6789" 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.110:6789,172.21.15.148:6789:/ 2024-09-21T04:13:15.863 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=77422328-77ce-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T04:13:15.864 INFO:teuthology.orchestra.run.smithi110.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T04:13:15.864 INFO:teuthology.orchestra.run.smithi110.stdout: You just mounted a file system that supports labels which does not 2024-09-21T04:13:15.864 INFO:teuthology.orchestra.run.smithi110.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T04:13:15.865 INFO:teuthology.orchestra.run.smithi110.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T04:13:15.865 INFO:teuthology.orchestra.run.smithi110.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T04:13:15.866 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T04:13:15.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:15.867 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:15.892 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:15.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:15.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:15.995 INFO:teuthology.orchestra.run.smithi110.stdout:client24325 2024-09-21T04:13:15.995 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:13:15.995 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:13:16.012 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:16.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:16.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:16.113 INFO:teuthology.orchestra.run.smithi110.stdout:client24325 2024-09-21T04:13:16.114 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:16.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:16.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:16.174 INFO:teuthology.orchestra.run.smithi110.stdout:client24325 2024-09-21T04:13:16.175 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77422328-77ce-11ef-baf5-efdc52797490.client24325/status') 2024-09-21T04:13:16.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:16.235 INFO:teuthology.orchestra.run.smithi110.stdout:instance: client.24325 (3)172.21.15.110:0/290757620 2024-09-21T04:13:16.236 INFO:teuthology.orchestra.run.smithi110.stdout:blocklisted: false 2024-09-21T04:13:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records in 2024-09-21T04:13:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records out 2024-09-21T04:13:16.236 INFO:teuthology.orchestra.run.smithi110.stderr:71 bytes copied, 2.303e-05 s, 3.1 MB/s 2024-09-21T04:13:16.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:16.243 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T04:13:16.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T04:13:16.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:16.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T04:13:16.442 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:17 smithi110 ceph-mon[27723]: pgmap v213: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:17 smithi148 ceph-mon[34957]: pgmap v213: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 -- 172.21.15.110:0/3574934014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4280fea10 msgr2=0x7fe4280fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 --2- 172.21.15.110:0/3574934014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4280fea10 0x7fe4280fee70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009c20 tx=0x7fe41c02a970 comp rx=0 tx=0).stop 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 -- 172.21.15.110:0/3574934014 shutdown_connections 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 --2- 172.21.15.110:0/3574934014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4280fea10 0x7fe4280fee70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 --2- 172.21.15.110:0/3574934014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe428104a10 0x7fe428104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 -- 172.21.15.110:0/3574934014 >> 172.21.15.110:0/3574934014 conn(0x7fe4280fa730 msgr2=0x7fe4280fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:18.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 -- 172.21.15.110:0/3574934014 shutdown_connections 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.337+0000 7fe43033b640 1 -- 172.21.15.110:0/3574934014 wait complete. 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 Processor -- start 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 -- start start 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280fea10 0x7fe42819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42819c390 con 0x7fe428104a10 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe43033b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe428195920 con 0x7fe4280fea10 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37060/0 (socket says 172.21.15.110:37060) 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 -- 172.21.15.110:0/826283570 learned_addr learned my addr 172.21.15.110:0/826283570 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42e0d7640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280fea10 0x7fe42819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 -- 172.21.15.110:0/826283570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280fea10 msgr2=0x7fe42819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280fea10 0x7fe42819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.338+0000 7fe42d8d6640 1 -- 172.21.15.110:0/826283570 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c009900 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe42d8d6640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe41c02ae90 tx=0x7fe41c030880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41c02f3c0 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe41c02f9e0 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe428195ba0 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41c039450 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.339+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe428196090 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.340+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe428100150 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.340+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe41c00a040 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.341+0000 7fe4277fe640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 0x7fe41006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.341+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe41c02c040 con 0x7fe428104a10 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.341+0000 7fe42e0d7640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 0x7fe41006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.341+0000 7fe42e0d7640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 0x7fe41006f110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe420005fd0 tx=0x7fe42000b390 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:18.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.345+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe41c077a70 con 0x7fe428104a10 2024-09-21T04:13:18.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:18.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.543+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4280fee70 con 0x7fe428104a10 2024-09-21T04:13:18.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.543+0000 7fe4277fe640 1 -- 172.21.15.110:0/826283570 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe41c07b090 con 0x7fe428104a10 2024-09-21T04:13:18.546 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 msgr2=0x7fe41006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 0x7fe41006f110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe420005fd0 tx=0x7fe42000b390 comp rx=0 tx=0).stop 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 msgr2=0x7fe42819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe41c02ae90 tx=0x7fe41c030880 comp rx=0 tx=0).stop 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 shutdown_connections 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe41006cc50 0x7fe41006f110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe428104a10 0x7fe42819bd70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 --2- 172.21.15.110:0/826283570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280fea10 0x7fe42819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:18.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 >> 172.21.15.110:0/826283570 conn(0x7fe4280fa730 msgr2=0x7fe4280fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:18.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 shutdown_connections 2024-09-21T04:13:18.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:18.545+0000 7fe43033b640 1 -- 172.21.15.110:0/826283570 wait complete. 2024-09-21T04:13:18.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:19.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T04:13:19.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T04:13:19.194 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T04:13:19.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:19.194 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-09-21T04:13:19.213 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:19.214 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-09-21T04:13:19.215 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2024-09-21T04:13:19.216 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:13:19.216 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T04:13:19.216 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T04:13:19.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:19.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:19.440 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T04:13:19.468 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-21T04:13:19.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:19.469 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-09-21T04:13:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-09-21T04:13:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T04:13:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T04:13:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T04:13:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T04:13:19.537 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T04:13:19.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:19 smithi148 ceph-mon[34957]: pgmap v214: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:19.591 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:19 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/826283570' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:13:19.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:19.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:19 smithi110 ceph-mon[27723]: pgmap v214: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:19.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:19 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/826283570' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:13:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:19.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:19.824 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T04:13:19.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:19.881 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-09-21T04:13:19.936 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:19.936 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T04:13:19.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:13:19.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T04:13:19.936 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T04:13:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:20.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.056 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T04:13:20.056 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:20.056 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T04:13:20.057 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T04:13:20.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:20.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.188 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:13:20.188 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T04:13:20.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-09-21T04:13:20.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T04:13:20.188 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T04:13:20.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:20.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:13:20.264 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:20.279 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T04:13:20.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:13:20.279 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:20.334 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T04:13:20.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:20.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T04:13:20.442 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T04:13:20.442 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T04:13:20.442 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-09-21T04:13:20.571 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T04:13:20.571 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T04:13:20.571 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-09-21T04:13:20.571 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.148:3300" 2024-09-21T04:13:20.571 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T04:13:20.576 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T04:13:20.576 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T04:13:20.576 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-09-21T04:13:20.576 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.148:3300" 2024-09-21T04:13:20.576 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.110:6789,172.21.15.148:6789" 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.110:6789,172.21.15.148:6789:/ 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=77422328-77ce-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T04:13:20.577 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T04:13:20.578 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T04:13:20.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T04:13:20.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:13:20.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:20.610 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:20.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.706 INFO:teuthology.orchestra.run.smithi148.stdout:client14600 2024-09-21T04:13:20.706 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:13:20.706 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:13:20.723 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:20.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:20.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.820 INFO:teuthology.orchestra.run.smithi148.stdout:client14600 2024-09-21T04:13:20.820 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:13:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:13:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.875 INFO:teuthology.orchestra.run.smithi148.stdout:client14600 2024-09-21T04:13:20.876 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77422328-77ce-11ef-baf5-efdc52797490.client14600/status') 2024-09-21T04:13:20.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:13:20.934 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.14600 (3)172.21.15.148:0/3441795895 2024-09-21T04:13:20.934 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-09-21T04:13:20.934 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-09-21T04:13:20.934 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-09-21T04:13:20.934 INFO:teuthology.orchestra.run.smithi148.stderr:72 bytes copied, 2.3926e-05 s, 3.0 MB/s 2024-09-21T04:13:20.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T04:13:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:13:20.941 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:13:20.956 INFO:teuthology.task.print:**** done client 2024-09-21T04:13:20.956 INFO:teuthology.run_tasks:Running task parallel... 2024-09-21T04:13:21.015 INFO:teuthology.task.parallel:starting parallel... 2024-09-21T04:13:21.016 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-21T04:13:21.016 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:13:21.017 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:13:21.017 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-09-21T04:13:21.017 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-21T04:13:21.017 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-21T04:13:21.021 INFO:tasks.workunit:Pulling workunits from ref f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:21.021 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T04:13:21.022 INFO:tasks.workunit:timeout=3h 2024-09-21T04:13:21.022 INFO:tasks.workunit:cleanup=True 2024-09-21T04:13:21.022 DEBUG:teuthology.orchestra.run.smithi110:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6ch/108d Inode: 1 Links: 2 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 04:12:43.462941905 +0000 2024-09-21T04:13:21.043 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 04:13:15.888893793 +0000 2024-09-21T04:13:21.044 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 04:12:43.462941905 +0000 2024-09-21T04:13:21.044 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T04:13:21.044 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T04:13:21.114 DEBUG:teuthology.orchestra.run.smithi148:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:21.130 INFO:teuthology.orchestra.run.smithi148.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 04:13:21.106805345 +0000 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 04:13:21.106805345 +0000 2024-09-21T04:13:21.131 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 04:12:43.462941905 +0000 2024-09-21T04:13:21.131 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-21T04:13:21.131 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-21T04:13:21.199 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:21.199 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:21.216 INFO:tasks.workunit.client.0.smithi110.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T04:13:21.241 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:21.256 INFO:tasks.workunit.client.1.smithi148.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-21T04:13:21.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:21 smithi110 ceph-mon[27723]: pgmap v215: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:21 smithi148 ceph-mon[34957]: pgmap v215: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:22.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/3448240254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 msgr2=0x7fb0bc0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/3448240254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc0fee30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009880 tx=0x7fb0ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/3448240254 shutdown_connections 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/3448240254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/3448240254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc0fee30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/3448240254 >> 172.21.15.110:0/3448240254 conn(0x7fb0bc0fa1c0 msgr2=0x7fb0bc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/3448240254 shutdown_connections 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/3448240254 wait complete. 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.872+0000 7fb0c2dfe640 1 Processor -- start 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c2dfe640 1 -- start start 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c2dfe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c2dfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c2dfe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc19a960 con 0x7fb0bc0fea30 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c2dfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc19aaa0 con 0x7fb0bc0ffc30 2024-09-21T04:13:22.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37806/0 (socket says 172.21.15.110:37806) 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 -- 172.21.15.110:0/85616012 learned_addr learned my addr 172.21.15.110:0/85616012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0bbfff640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 -- 172.21.15.110:0/85616012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 msgr2=0x7fb0bc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 -- 172.21.15.110:0/85616012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac009560 con 0x7fb0bc0fea30 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0bbfff640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.873+0000 7fb0c0b9a640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a9c0 tx=0x7fb0ac005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.874+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ac004b20 con 0x7fb0bc0fea30 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.874+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0ac004c80 con 0x7fb0bc0fea30 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.874+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc19acf0 con 0x7fb0bc0fea30 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.874+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ac003b80 con 0x7fb0bc0fea30 2024-09-21T04:13:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.874+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc19b260 con 0x7fb0bc0fea30 2024-09-21T04:13:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.875+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc107640 con 0x7fb0bc0fea30 2024-09-21T04:13:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.875+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb0ac031640 con 0x7fb0bc0fea30 2024-09-21T04:13:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.876+0000 7fb0b9ffb640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 0x7fb0a806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.876+0000 7fb0bbfff640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 0x7fb0a806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.876+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb0ac0ad5d0 con 0x7fb0bc0fea30 2024-09-21T04:13:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.876+0000 7fb0bbfff640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 0x7fb0a806f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc068cc0 tx=0x7fb0a4005d30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:22.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:22.880+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb0ac077940 con 0x7fb0bc0fea30 2024-09-21T04:13:23.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.043+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fb0bc19b450 con 0x7fb0bc0fea30 2024-09-21T04:13:23.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.047+0000 7fb0b9ffb640 1 -- 172.21.15.110:0/85616012 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 126+0+0 (secure 0 0 0) 0x7fb0ac07af60 con 0x7fb0bc0fea30 2024-09-21T04:13:23.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 msgr2=0x7fb0a806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:23.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 0x7fb0a806f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc068cc0 tx=0x7fb0a4005d30 comp rx=0 tx=0).stop 2024-09-21T04:13:23.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 msgr2=0x7fb0bc068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a9c0 tx=0x7fb0ac005610 comp rx=0 tx=0).stop 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 shutdown_connections 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0bc0ffc30 0x7fb0bc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb0a806cc50 0x7fb0a806f110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 --2- 172.21.15.110:0/85616012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0bc0fea30 0x7fb0bc068740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.049+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 >> 172.21.15.110:0/85616012 conn(0x7fb0bc0fa1c0 msgr2=0x7fb0bc102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.050+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 shutdown_connections 2024-09-21T04:13:23.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:23.050+0000 7fb0c2dfe640 1 -- 172.21.15.110:0/85616012 wait complete. 2024-09-21T04:13:23.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:23 smithi110 ceph-mon[27723]: pgmap v216: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:23 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/85616012' entity='client.admin' 2024-09-21T04:13:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:23 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:23.784 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:13:23.784 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:13:23.784 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-21T04:13:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:23 smithi148 ceph-mon[34957]: pgmap v216: 65 pgs: 65 active+clean; 579 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:13:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:23 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/85616012' entity='client.admin' 2024-09-21T04:13:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:23 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:24.033 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:24.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:24.820 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:24.820 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:24 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:24 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:24 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:24 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:25.732 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:25 smithi110 ceph-mon[27723]: pgmap v217: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 -- 172.21.15.110:0/2470017167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 msgr2=0x7f53600fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 --2- 172.21.15.110:0/2470017167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53600fee30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f535c009880 tx=0x7f535c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 -- 172.21.15.110:0/2470017167 shutdown_connections 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 --2- 172.21.15.110:0/2470017167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f53601000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 --2- 172.21.15.110:0/2470017167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53600fee30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 -- 172.21.15.110:0/2470017167 >> 172.21.15.110:0/2470017167 conn(0x7f53600fa1c0 msgr2=0x7f53600fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 -- 172.21.15.110:0/2470017167 shutdown_connections 2024-09-21T04:13:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.725+0000 7f5367578640 1 -- 172.21.15.110:0/2470017167 wait complete. 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 Processor -- start 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 -- start start 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f5360195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53601961c0 con 0x7f53600fea30 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5367578640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360196330 con 0x7f53600ffc30 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5365314640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5365314640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37826/0 (socket says 172.21.15.110:37826) 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5365314640 1 -- 172.21.15.110:0/67607572 learned_addr learned my addr 172.21.15.110:0/67607572 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.726+0000 7f5364b13640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f5360195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5365314640 1 -- 172.21.15.110:0/67607572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 msgr2=0x7f5360195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5365314640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f5360195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5365314640 1 -- 172.21.15.110:0/67607572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c009560 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5364b13640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f5360195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5365314640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f535c004650 tx=0x7f535c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c004aa0 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536019acb0 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f535c005d50 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c0039e0 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.727+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536019b1a0 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.728+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360107640 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.728+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f535c003ca0 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.729+0000 7f53567fc640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 0x7f534c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.729+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f535c0ad330 con 0x7f53600fea30 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.729+0000 7f5364b13640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 0x7f534c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.730+0000 7f5364b13640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 0x7f534c06f110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5360196b10 tx=0x7f5350005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.734+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f535c077750 con 0x7f53600fea30 2024-09-21T04:13:25.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.899+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f536019b3c0 con 0x7f53600fea30 2024-09-21T04:13:25.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.899+0000 7f53567fc640 1 -- 172.21.15.110:0/67607572 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f535c07ad70 con 0x7f53600fea30 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 msgr2=0x7f534c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 0x7f534c06f110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5360196b10 tx=0x7f5350005f90 comp rx=0 tx=0).stop 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 msgr2=0x7f53601956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f535c004650 tx=0x7f535c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 shutdown_connections 2024-09-21T04:13:25.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53600ffc30 0x7f5360195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f534c06cc50 0x7f534c06f110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 --2- 172.21.15.110:0/67607572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53600fea30 0x7f53601956b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:25.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 >> 172.21.15.110:0/67607572 conn(0x7f53600fa1c0 msgr2=0x7f5360067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:25.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 shutdown_connections 2024-09-21T04:13:25.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:25.901+0000 7f5367578640 1 -- 172.21.15.110:0/67607572 wait complete. 2024-09-21T04:13:25.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:25 smithi148 ceph-mon[34957]: pgmap v217: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:26.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-21T04:13:26.888 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:27.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:27 smithi110 ceph-mon[27723]: pgmap v218: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:27.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:27 smithi148 ceph-mon[34957]: pgmap v218: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 -- 172.21.15.110:0/1061350367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040ffc80 msgr2=0x7f9d04100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 --2- 172.21.15.110:0/1061350367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040ffc80 0x7f9d04100100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009880 tx=0x7f9cf402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 -- 172.21.15.110:0/1061350367 shutdown_connections 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 --2- 172.21.15.110:0/1061350367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040ffc80 0x7f9d04100100 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 --2- 172.21.15.110:0/1061350367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fea80 0x7f9d040fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 -- 172.21.15.110:0/1061350367 >> 172.21.15.110:0/1061350367 conn(0x7f9d040fa230 msgr2=0x7f9d040fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 -- 172.21.15.110:0/1061350367 shutdown_connections 2024-09-21T04:13:28.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 -- 172.21.15.110:0/1061350367 wait complete. 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.464+0000 7f9d0b606640 1 Processor -- start 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d0b606640 1 -- start start 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d0b606640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d0b606640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040ffc80 0x7f9d04193900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d0b606640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04193ed0 con 0x7f9d040fea80 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d0b606640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04194040 con 0x7f9d040ffc80 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:28.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37848/0 (socket says 172.21.15.110:37848) 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 -- 172.21.15.110:0/3189485295 learned_addr learned my addr 172.21.15.110:0/3189485295 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d08ba1640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040ffc80 0x7f9d04193900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 -- 172.21.15.110:0/3189485295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040ffc80 msgr2=0x7f9d04193900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040ffc80 0x7f9d04193900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 -- 172.21.15.110:0/3189485295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4009560 con 0x7f9d040fea80 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.465+0000 7f9d093a2640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d900 tx=0x7f9d0000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.466+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d00009c00 con 0x7f9d040fea80 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.466+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d00009d60 con 0x7f9d040fea80 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.466+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04198ae0 con 0x7f9d040fea80 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.466+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d0000f7b0 con 0x7f9d040fea80 2024-09-21T04:13:28.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.466+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04199000 con 0x7f9d040fea80 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.467+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d040fee80 con 0x7f9d040fea80 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.467+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9d00010460 con 0x7f9d040fea80 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.467+0000 7f9cfa7fc640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 0x7f9cf0071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.468+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9d0008c530 con 0x7f9d040fea80 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.468+0000 7f9d08ba1640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 0x7f9cf0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:28.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.468+0000 7f9d08ba1640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 0x7f9cf0071330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf40314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:28.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.472+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9d000568a0 con 0x7f9d040fea80 2024-09-21T04:13:28.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.638+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9d041067f0 con 0x7f9d040fea80 2024-09-21T04:13:28.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.638+0000 7f9cfa7fc640 1 -- 172.21.15.110:0/3189485295 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9d00056070 con 0x7f9d040fea80 2024-09-21T04:13:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.640+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 msgr2=0x7f9cf0071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.640+0000 7f9d0b606640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 0x7f9cf0071330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf40314e0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.640+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 msgr2=0x7f9d041933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.640+0000 7f9d0b606640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d900 tx=0x7f9d0000ddc0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.640+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 shutdown_connections 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040ffc80 0x7f9d04193900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9cf006ee70 0x7f9cf0071330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 --2- 172.21.15.110:0/3189485295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d040fea80 0x7f9d041933c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 >> 172.21.15.110:0/3189485295 conn(0x7f9d040fa230 msgr2=0x7f9d040fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 shutdown_connections 2024-09-21T04:13:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:28.641+0000 7f9d0b606640 1 -- 172.21.15.110:0/3189485295 wait complete. 2024-09-21T04:13:29.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-21T04:13:29.488 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:29.596 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:29 smithi110 ceph-mon[27723]: pgmap v219: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:29.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:29 smithi148 ceph-mon[34957]: pgmap v219: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 -- 172.21.15.110:0/724920381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0feaa0 msgr2=0x7f429c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/724920381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0feaa0 0x7f429c0feea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f428c009880 tx=0x7f428c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 -- 172.21.15.110:0/724920381 shutdown_connections 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/724920381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429c0ffca0 0x7f429c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/724920381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0feaa0 0x7f429c0feea0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 -- 172.21.15.110:0/724920381 >> 172.21.15.110:0/724920381 conn(0x7f429c0fa230 msgr2=0x7f429c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 -- 172.21.15.110:0/724920381 shutdown_connections 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.096+0000 7f42a0ad1640 1 -- 172.21.15.110:0/724920381 wait complete. 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 Processor -- start 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 -- start start 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429c0feaa0 0x7f429c068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19a920 con 0x7f429c0ffca0 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f42a0ad1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19aa60 con 0x7f429c0feaa0 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37862/0 (socket says 172.21.15.110:37862) 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 -- 172.21.15.110:0/1111579017 learned_addr learned my addr 172.21.15.110:0/1111579017 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 -- 172.21.15.110:0/1111579017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429c0feaa0 msgr2=0x7f429c068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:31.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429c0feaa0 0x7f429c068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 -- 172.21.15.110:0/1111579017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c009560 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f4299d9b640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f429000c8a0 tx=0x7f429000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.097+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4290007b80 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.098+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4290007ce0 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.098+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4290018810 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.098+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c19ad10 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.098+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c19b2e0 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.099+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c18e620 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.099+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f4290018a80 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.099+0000 7f428b7fe640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 0x7f428406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.100+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f429008c4c0 con 0x7f429c0ffca0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.100+0000 7f429a59c640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 0x7f428406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:31.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.100+0000 7f429a59c640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 0x7f428406f110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f428c02a9c0 tx=0x7f428c000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:31.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.104+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f42900568e0 con 0x7f429c0ffca0 2024-09-21T04:13:31.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.269+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f429c107690 con 0x7f429c0ffca0 2024-09-21T04:13:31.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.270+0000 7f428b7fe640 1 -- 172.21.15.110:0/1111579017 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4290059f00 con 0x7f429c0ffca0 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 msgr2=0x7f428406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 0x7f428406f110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f428c02a9c0 tx=0x7f428c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 msgr2=0x7f429c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f429000c8a0 tx=0x7f429000cd60 comp rx=0 tx=0).stop 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 shutdown_connections 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f428406cc50 0x7f428406f110 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f429c0ffca0 0x7f429c19a3e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 --2- 172.21.15.110:0/1111579017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429c0feaa0 0x7f429c068f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 >> 172.21.15.110:0/1111579017 conn(0x7f429c0fa230 msgr2=0x7f429c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 shutdown_connections 2024-09-21T04:13:31.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:31.272+0000 7f42a0ad1640 1 -- 172.21.15.110:0/1111579017 wait complete. 2024-09-21T04:13:31.576 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:31 smithi110 ceph-mon[27723]: pgmap v220: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:31.882 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-21T04:13:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:31 smithi148 ceph-mon[34957]: pgmap v220: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:32.081 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/2158093643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00fea50 msgr2=0x7fb3e00fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/2158093643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00fea50 0x7fb3e00fee50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009c20 tx=0x7fb3c802a970 comp rx=0 tx=0).stop 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/2158093643 shutdown_connections 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/2158093643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00ffc50 0x7fb3e01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/2158093643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00fea50 0x7fb3e00fee50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/2158093643 >> 172.21.15.110:0/2158093643 conn(0x7fb3e00fa1c0 msgr2=0x7fb3e00fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/2158093643 shutdown_connections 2024-09-21T04:13:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/2158093643 wait complete. 2024-09-21T04:13:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.677+0000 7fb3e6cb6640 1 Processor -- start 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 -- start start 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00fea50 0x7fb3e0195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e0196180 con 0x7fb3e00ffc50 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e01962f0 con 0x7fb3e00fea50 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47202/0 (socket says 172.21.15.110:47202) 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 -- 172.21.15.110:0/1911292226 learned_addr learned my addr 172.21.15.110:0/1911292226 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e4a52640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00fea50 0x7fb3e0195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 -- 172.21.15.110:0/1911292226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00fea50 msgr2=0x7fb3e0195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00fea50 0x7fb3e0195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 -- 172.21.15.110:0/1911292226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8009900 con 0x7fb3e00ffc50 2024-09-21T04:13:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3dffff640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d900 tx=0x7fb3d000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d0009c00 con 0x7fb3e00ffc50 2024-09-21T04:13:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.678+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e019ad90 con 0x7fb3e00ffc50 2024-09-21T04:13:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.679+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3d0009d60 con 0x7fb3e00ffc50 2024-09-21T04:13:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.679+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d00049d0 con 0x7fb3e00ffc50 2024-09-21T04:13:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.679+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e019b2b0 con 0x7fb3e00ffc50 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.680+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e00fee50 con 0x7fb3e00ffc50 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.680+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fb3d0004020 con 0x7fb3e00ffc50 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.680+0000 7fb3ddffb640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 0x7fb3cc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.681+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb3d008bf70 con 0x7fb3e00ffc50 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.681+0000 7fb3e4a52640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 0x7fb3cc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.681+0000 7fb3e4a52640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 0x7fb3cc06f110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c8004a80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.685+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb3d00bfe20 con 0x7fb3e00ffc50 2024-09-21T04:13:33.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.862+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3e00fc3e0 con 0x7fb3cc06cc50 2024-09-21T04:13:33.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:33 smithi110 ceph-mon[27723]: pgmap v221: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:33.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:33.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.867+0000 7fb3ddffb640 1 -- 172.21.15.110:0/1911292226 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb3e00fc3e0 con 0x7fb3cc06cc50 2024-09-21T04:13:33.870 INFO:teuthology.orchestra.run.smithi110.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.869+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 msgr2=0x7fb3cc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.869+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 0x7fb3cc06f110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c8004a80 comp rx=0 tx=0).stop 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.869+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 msgr2=0x7fb3e0195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.869+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d900 tx=0x7fb3d000ddc0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 shutdown_connections 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fb3cc06cc50 0x7fb3cc06f110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3e00ffc50 0x7fb3e0195bb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 --2- 172.21.15.110:0/1911292226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e00fea50 0x7fb3e0195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 >> 172.21.15.110:0/1911292226 conn(0x7fb3e00fa1c0 msgr2=0x7fb3e00fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 shutdown_connections 2024-09-21T04:13:33.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:33.870+0000 7fb3e6cb6640 1 -- 172.21.15.110:0/1911292226 wait complete. 2024-09-21T04:13:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:33 smithi148 ceph-mon[34957]: pgmap v221: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:34.624 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:13:34.624 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:13:34.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-21T04:13:34.834 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:13:35.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:35.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:34 smithi110 ceph-mon[27723]: pgmap v222: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:13:35.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:13:35.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:13:35.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:13:35.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:13:35.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:34 smithi148 ceph-mon[34957]: pgmap v222: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 -- 172.21.15.110:0/1588811321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 msgr2=0x7f64fc0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1588811321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc0fee30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009880 tx=0x7f64f802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 -- 172.21.15.110:0/1588811321 shutdown_connections 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1588811321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1588811321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc0fee30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 -- 172.21.15.110:0/1588811321 >> 172.21.15.110:0/1588811321 conn(0x7f64fc0fa1c0 msgr2=0x7f64fc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 -- 172.21.15.110:0/1588811321 shutdown_connections 2024-09-21T04:13:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.668+0000 7f6502bc0640 1 -- 172.21.15.110:0/1588811321 wait complete. 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 Processor -- start 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 -- start start 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc198350 con 0x7f64fc0fea30 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f6502bc0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1984c0 con 0x7f64fc0ffc30 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47222/0 (socket says 172.21.15.110:47222) 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 -- 172.21.15.110:0/1192429260 learned_addr learned my addr 172.21.15.110:0/1192429260 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f64f3fff640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 -- 172.21.15.110:0/1192429260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 msgr2=0x7f64fc197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f650095c640 1 -- 172.21.15.110:0/1192429260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009560 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.669+0000 7f64f3fff640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f650095c640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64f80054f0 tx=0x7f64f8005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f8004aa0 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64f8004c00 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64f8003ac0 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc19cf00 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.670+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc19d3f0 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.671+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc107620 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.672+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f64f8003d30 con 0x7f64fc0fea30 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.672+0000 7f64f1ffb640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 0x7f64e406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.672+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f64f80ad4b0 con 0x7f64fc0fea30 2024-09-21T04:13:36.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.672+0000 7f64f3fff640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 0x7f64e406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.673+0000 7f64f3fff640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 0x7f64e406f110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64fc198d60 tx=0x7f64ec005e50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:36.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.677+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f64f80778a0 con 0x7f64fc0fea30 2024-09-21T04:13:36.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.890+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64fc19d6c0 con 0x7f64e406cc50 2024-09-21T04:13:36.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.891+0000 7f64f1ffb640 1 -- 172.21.15.110:0/1192429260 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f64fc19d6c0 con 0x7f64e406cc50 2024-09-21T04:13:36.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 msgr2=0x7f64e406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 0x7f64e406f110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64fc198d60 tx=0x7f64ec005e50 comp rx=0 tx=0).stop 2024-09-21T04:13:36.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 msgr2=0x7f64fc197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64f80054f0 tx=0x7f64f8005520 comp rx=0 tx=0).stop 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 shutdown_connections 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64fc0ffc30 0x7f64fc197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f64e406cc50 0x7f64e406f110 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 --2- 172.21.15.110:0/1192429260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64fc0fea30 0x7f64fc197840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 >> 172.21.15.110:0/1192429260 conn(0x7f64fc0fa1c0 msgr2=0x7f64fc0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 shutdown_connections 2024-09-21T04:13:36.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.894+0000 7f6502bc0640 1 -- 172.21.15.110:0/1192429260 wait complete. 2024-09-21T04:13:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 -- 172.21.15.110:0/1584578550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 msgr2=0x7feec00fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 --2- 172.21.15.110:0/1584578550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec00fee30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009880 tx=0x7feeb002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 -- 172.21.15.110:0/1584578550 shutdown_connections 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 --2- 172.21.15.110:0/1584578550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 --2- 172.21.15.110:0/1584578550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec00fee30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 -- 172.21.15.110:0/1584578550 >> 172.21.15.110:0/1584578550 conn(0x7feec00fa1c0 msgr2=0x7feec00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:36.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 -- 172.21.15.110:0/1584578550 shutdown_connections 2024-09-21T04:13:36.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.983+0000 7feec4ce5640 1 -- 172.21.15.110:0/1584578550 wait complete. 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 Processor -- start 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 -- start start 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec0195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec0196080 con 0x7feec00fea30 2024-09-21T04:13:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feec4ce5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01961f0 con 0x7feec00ffc30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feebe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feebe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47232/0 (socket says 172.21.15.110:47232) 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feebe59c640 1 -- 172.21.15.110:0/2229221080 learned_addr learned my addr 172.21.15.110:0/2229221080 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feebe59c640 1 -- 172.21.15.110:0/2229221080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 msgr2=0x7feec0195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feeb7fff640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec0195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.984+0000 7feebe59c640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec0195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feebe59c640 1 -- 172.21.15.110:0/2229221080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0009560 con 0x7feec00fea30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feeb7fff640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec0195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feebe59c640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feeb002a9c0 tx=0x7feeb0005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb0004b20 con 0x7feec00fea30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeb0004c80 con 0x7feec00fea30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec019ac70 con 0x7feec00fea30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeb0003b80 con 0x7feec00fea30 2024-09-21T04:13:36.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.985+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec019b110 con 0x7feec00fea30 2024-09-21T04:13:36.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.986+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec0107620 con 0x7feec00fea30 2024-09-21T04:13:36.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.987+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7feeb0031640 con 0x7feec00fea30 2024-09-21T04:13:36.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.987+0000 7feeb77fe640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 0x7feeac06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:36.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.987+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7feeb00ad530 con 0x7feec00fea30 2024-09-21T04:13:36.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.987+0000 7feeb7fff640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 0x7feeac06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:36.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.988+0000 7feeb7fff640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 0x7feeac06f110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feec0196b20 tx=0x7feea8005d90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:36.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:36.992+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feeb00778a0 con 0x7feec00fea30 2024-09-21T04:13:37.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.202+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feec019b440 con 0x7feeac06cc50 2024-09-21T04:13:37.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.203+0000 7feeb77fe640 1 -- 172.21.15.110:0/2229221080 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7feec019b440 con 0x7feeac06cc50 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.205+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 msgr2=0x7feeac06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.205+0000 7feec4ce5640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 0x7feeac06f110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feec0196b20 tx=0x7feea8005d90 comp rx=0 tx=0).stop 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.205+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 msgr2=0x7feec0195600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.205+0000 7feec4ce5640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feeb002a9c0 tx=0x7feeb0005610 comp rx=0 tx=0).stop 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 shutdown_connections 2024-09-21T04:13:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00ffc30 0x7feec0195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feeac06cc50 0x7feeac06f110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 --2- 172.21.15.110:0/2229221080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feec00fea30 0x7feec0195600 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 >> 172.21.15.110:0/2229221080 conn(0x7feec00fa1c0 msgr2=0x7feec0067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 shutdown_connections 2024-09-21T04:13:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.206+0000 7feec4ce5640 1 -- 172.21.15.110:0/2229221080 wait complete. 2024-09-21T04:13:37.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 -- 172.21.15.110:0/1250829428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40ffc30 msgr2=0x7fa9b41000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/1250829428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40ffc30 0x7fa9b41000b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009880 tx=0x7fa99c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 -- 172.21.15.110:0/1250829428 shutdown_connections 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/1250829428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40ffc30 0x7fa9b41000b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/1250829428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9b40fea30 0x7fa9b40fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 -- 172.21.15.110:0/1250829428 >> 172.21.15.110:0/1250829428 conn(0x7fa9b40fa1c0 msgr2=0x7fa9b40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 -- 172.21.15.110:0/1250829428 shutdown_connections 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 -- 172.21.15.110:0/1250829428 wait complete. 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.296+0000 7fa9b89a6640 1 Processor -- start 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b89a6640 1 -- start start 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b89a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b89a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9b40ffc30 0x7fa9b4195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b89a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4196180 con 0x7fa9b40fea30 2024-09-21T04:13:37.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b89a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41962f0 con 0x7fa9b40ffc30 2024-09-21T04:13:37.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47250/0 (socket says 172.21.15.110:47250) 2024-09-21T04:13:37.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 -- 172.21.15.110:0/214819590 learned_addr learned my addr 172.21.15.110:0/214819590 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 -- 172.21.15.110:0/214819590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9b40ffc30 msgr2=0x7fa9b4195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9b40ffc30 0x7fa9b4195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 -- 172.21.15.110:0/214819590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009560 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.297+0000 7fa9b259c640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d900 tx=0x7fa9a400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.298+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9a4009c00 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.298+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b419ad20 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.298+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9a4009d60 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.298+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9a400f660 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.298+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b419b2f0 con 0x7fa9b40fea30 2024-09-21T04:13:37.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.299+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4107640 con 0x7fa9b40fea30 2024-09-21T04:13:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.299+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fa9a4010460 con 0x7fa9b40fea30 2024-09-21T04:13:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.300+0000 7fa9ab7fe640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 0x7fa9a006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.300+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa9a408c180 con 0x7fa9b40fea30 2024-09-21T04:13:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.300+0000 7fa9abfff640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 0x7fa9a006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.300+0000 7fa9abfff640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 0x7fa9a006f110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa99c02a9c0 tx=0x7fa99c000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.304+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa9a40564f0 con 0x7fa9b40fea30 2024-09-21T04:13:37.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.506+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9b419b4e0 con 0x7fa9a006cc50 2024-09-21T04:13:37.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:37 smithi110 ceph-mon[27723]: pgmap v223: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:37.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.517+0000 7fa9ab7fe640 1 -- 172.21.15.110:0/214819590 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fa9b419b4e0 con 0x7fa9a006cc50 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (4m) 29s ago 6m 25.3M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (7m) 29s ago 7m 7524k - 18.2.4-871-g6a1bdf85 0adab276c97e b0186c860d81 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (6m) 35s ago 6m 7247k - 18.2.4-871-g6a1bdf85 0adab276c97e 6e1e8fb80a91 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (7m) 29s ago 7m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (6m) 35s ago 6m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (4m) 29s ago 6m 84.8M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (47s) 29s ago 47s 16.3M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (55s) 29s ago 54s 11.8M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (45s) 35s ago 45s 13.6M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (52s) 35s ago 52s 15.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:13:37.521 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (8m) 29s ago 8m 486M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (5m) 35s ago 5m 440M - 18.2.4-871-g6a1bdf85 0adab276c97e 4f74b6e9b876 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (8m) 29s ago 8m 47.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (5m) 35s ago 5m 36.0M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (6m) 29s ago 6m 16.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 35s ago 6m 15.7M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 29s ago 4m 57.7M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 29s ago 4m 54.6M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (3m) 29s ago 3m 53.1M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (3m) 35s ago 3m 53.1M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (2m) 35s ago 2m 49.9M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (2m) 35s ago 2m 53.7M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:13:37.522 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 29s ago 6m 59.4M - 2.43.0 a07b618ecd1d 2d686ff7a6ad 2024-09-21T04:13:37.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 msgr2=0x7fa9a006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 0x7fa9a006f110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa99c02a9c0 tx=0x7fa99c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 msgr2=0x7fa9b4195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d900 tx=0x7fa9a400ddc0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 shutdown_connections 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9b40ffc30 0x7fa9b4195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fa9a006cc50 0x7fa9a006f110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 --2- 172.21.15.110:0/214819590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9b40fea30 0x7fa9b4195670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 >> 172.21.15.110:0/214819590 conn(0x7fa9b40fa1c0 msgr2=0x7fa9b4067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 shutdown_connections 2024-09-21T04:13:37.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.520+0000 7fa9b89a6640 1 -- 172.21.15.110:0/214819590 wait complete. 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.609+0000 7f83da2d8640 1 -- 172.21.15.110:0/131784711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40ffc30 msgr2=0x7f83d41000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.609+0000 7f83da2d8640 1 --2- 172.21.15.110:0/131784711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40ffc30 0x7f83d41000b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009880 tx=0x7f83c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.609+0000 7f83da2d8640 1 -- 172.21.15.110:0/131784711 shutdown_connections 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.609+0000 7f83da2d8640 1 --2- 172.21.15.110:0/131784711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40ffc30 0x7f83d41000b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.609+0000 7f83da2d8640 1 --2- 172.21.15.110:0/131784711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40fea30 0x7f83d40fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.610+0000 7f83da2d8640 1 -- 172.21.15.110:0/131784711 >> 172.21.15.110:0/131784711 conn(0x7f83d40fa1c0 msgr2=0x7f83d40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.610+0000 7f83da2d8640 1 -- 172.21.15.110:0/131784711 shutdown_connections 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.610+0000 7f83da2d8640 1 -- 172.21.15.110:0/131784711 wait complete. 2024-09-21T04:13:37.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.610+0000 7f83da2d8640 1 Processor -- start 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.610+0000 7f83da2d8640 1 -- start start 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83da2d8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83da2d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40ffc30 0x7f83d419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83da2d8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d41961d0 con 0x7f83d40fea30 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83da2d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d419cca0 con 0x7f83d40ffc30 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47260/0 (socket says 172.21.15.110:47260) 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 -- 172.21.15.110:0/2561228914 learned_addr learned my addr 172.21.15.110:0/2561228914 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d37fe640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40ffc30 0x7f83d419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 -- 172.21.15.110:0/2561228914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40ffc30 msgr2=0x7f83d419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40ffc30 0x7f83d419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 -- 172.21.15.110:0/2561228914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8009560 con 0x7f83d40fea30 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d3fff640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83c400d8d0 tx=0x7f83c400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c4009a20 con 0x7f83d40fea30 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83c4009b80 con 0x7f83d40fea30 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.611+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d419cf30 con 0x7f83d40fea30 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.612+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c400f5d0 con 0x7f83d40fea30 2024-09-21T04:13:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.612+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d419d4d0 con 0x7f83d40fea30 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.613+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c0005280 con 0x7f83d40fea30 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.613+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f83c4010460 con 0x7f83d40fea30 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.614+0000 7f83d17fa640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 0x7f83bc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.614+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f83c408c080 con 0x7f83d40fea30 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.614+0000 7f83d37fe640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 0x7f83bc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.614+0000 7f83d37fe640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 0x7f83bc06f110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006010 tx=0x7f83c8000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:37.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.619+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f83c40563f0 con 0x7f83d40fea30 2024-09-21T04:13:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:37 smithi148 ceph-mon[34957]: pgmap v223: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:37.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.890+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83c0005d40 con 0x7f83d40fea30 2024-09-21T04:13:37.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.890+0000 7f83d17fa640 1 -- 172.21.15.110:0/2561228914 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f83c4059a10 con 0x7f83d40fea30 2024-09-21T04:13:37.893 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:13:37.893 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:13:37.893 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 14 2024-09-21T04:13:37.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:13:37.895 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 msgr2=0x7f83bc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 0x7f83bc06f110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006010 tx=0x7f83c8000bc0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 msgr2=0x7f83d4068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83c400d8d0 tx=0x7f83c400dd90 comp rx=0 tx=0).stop 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 shutdown_connections 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40ffc30 0x7f83d419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f83bc06cc50 0x7f83bc06f110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 --2- 172.21.15.110:0/2561228914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83d40fea30 0x7f83d4068740 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 >> 172.21.15.110:0/2561228914 conn(0x7f83d40fa1c0 msgr2=0x7f83d4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 shutdown_connections 2024-09-21T04:13:37.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.893+0000 7f83da2d8640 1 -- 172.21.15.110:0/2561228914 wait complete. 2024-09-21T04:13:37.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/4105560272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc041005e0 msgr2=0x7fcc041029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/4105560272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc041005e0 0x7fcc041029d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40098b0 tx=0x7fcbf402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/4105560272 shutdown_connections 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/4105560272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc04102f10 0x7fcc04105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/4105560272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc041005e0 0x7fcc041029d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/4105560272 >> 172.21.15.110:0/4105560272 conn(0x7fcc040fa1e0 msgr2=0x7fcc040fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/4105560272 shutdown_connections 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.989+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/4105560272 wait complete. 2024-09-21T04:13:37.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 Processor -- start 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 -- start start 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc041005e0 0x7fcc040fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040fead0 con 0x7fcc04102f10 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.990+0000 7fcc0a4eb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040fec40 con 0x7fcc041005e0 2024-09-21T04:13:37.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47284/0 (socket says 172.21.15.110:47284) 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 -- 172.21.15.110:0/2471937248 learned_addr learned my addr 172.21.15.110:0/2471937248 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc041005e0 0x7fcc040fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 -- 172.21.15.110:0/2471937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc041005e0 msgr2=0x7fcc040fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc041005e0 0x7fcc040fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:37.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 -- 172.21.15.110:0/2471937248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf4009560 con 0x7fcc04102f10 2024-09-21T04:13:37.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc037fe640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d6c0 tx=0x7fcbf800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:37.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf80051f0 con 0x7fcc04102f10 2024-09-21T04:13:37.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbf8009ce0 con 0x7fcc04102f10 2024-09-21T04:13:37.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.991+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040fef20 con 0x7fcc04102f10 2024-09-21T04:13:37.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.992+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf8010460 con 0x7fcc04102f10 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.992+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0419d700 con 0x7fcc04102f10 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.993+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc041009c0 con 0x7fcc04102f10 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.993+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fcbf8005350 con 0x7fcc04102f10 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.994+0000 7fcc017fa640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 0x7fcbec06f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.994+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcbf808bc70 con 0x7fcc04102f10 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.994+0000 7fcc03fff640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 0x7fcbec06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:37.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.995+0000 7fcc03fff640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 0x7fcbec06f160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4006010 tx=0x7fcbf40314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:38.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:37.999+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcbf8056070 con 0x7fcc04102f10 2024-09-21T04:13:38.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.235+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc04108790 con 0x7fcc04102f10 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.236+0000 7fcc017fa640 1 -- 172.21.15.110:0/2471937248 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7fcbf8017070 con 0x7fcc04102f10 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:13:38.239 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:13:38.240 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:13:38.241 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:13:38.242 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:13:38.243 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:13:38.243 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:13:38.243 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:13:38.243 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:13:38.243 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 msgr2=0x7fcbec06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 0x7fcbec06f160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4006010 tx=0x7fcbf40314e0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 msgr2=0x7fcc040fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d6c0 tx=0x7fcbf800db80 comp rx=0 tx=0).stop 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 shutdown_connections 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fcbec06cca0 0x7fcbec06f160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc04102f10 0x7fcc040fe590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 --2- 172.21.15.110:0/2471937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc041005e0 0x7fcc040fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 >> 172.21.15.110:0/2471937248 conn(0x7fcc040fa1e0 msgr2=0x7fcc040fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 shutdown_connections 2024-09-21T04:13:38.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.238+0000 7fcc0a4eb640 1 -- 172.21.15.110:0/2471937248 wait complete. 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- 172.21.15.110:0/2682287207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0980101280 msgr2=0x7f0980101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2682287207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0980101280 0x7f0980101680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0974009880 tx=0x7f097402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- 172.21.15.110:0/2682287207 shutdown_connections 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2682287207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09800ff7c0 0x7f09800ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2682287207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0980101280 0x7f0980101680 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- 172.21.15.110:0/2682287207 >> 172.21.15.110:0/2682287207 conn(0x7f09800fb330 msgr2=0x7f09800fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- 172.21.15.110:0/2682287207 shutdown_connections 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- 172.21.15.110:0/2682287207 wait complete. 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 Processor -- start 2024-09-21T04:13:38.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.333+0000 7f0986bfd640 1 -- start start 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0986bfd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0986bfd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0980101280 0x7f09800728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0986bfd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980072de0 con 0x7f09800ff7c0 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0986bfd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980072f50 con 0x7f0980101280 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47294/0 (socket says 172.21.15.110:47294) 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 -- 172.21.15.110:0/2046964762 learned_addr learned my addr 172.21.15.110:0/2046964762 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f097ffff640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0980101280 0x7f09800728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 -- 172.21.15.110:0/2046964762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0980101280 msgr2=0x7f09800728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0980101280 0x7f09800728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 -- 172.21.15.110:0/2046964762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0974009560 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.334+0000 7f0984999640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0974006010 tx=0x7f09740055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.335+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0974004aa0 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.335+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0974005d50 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.335+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800731d0 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.335+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09740039b0 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.335+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0980195390 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.336+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0980109200 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.336+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f0974031c00 con 0x7f09800ff7c0 2024-09-21T04:13:38.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.337+0000 7f097dffb640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 0x7f097006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.337+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f09740ad1d0 con 0x7f09800ff7c0 2024-09-21T04:13:38.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.337+0000 7f097ffff640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 0x7f097006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.338+0000 7f097ffff640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 0x7f097006f110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0980073fd0 tx=0x7f096c018040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:38.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.341+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0974077670 con 0x7f09800ff7c0 2024-09-21T04:13:38.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.549+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09801089e0 con 0x7f097006cc50 2024-09-21T04:13:38.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:38 smithi110 ceph-mon[27723]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:38 smithi110 ceph-mon[27723]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:38 smithi110 ceph-mon[27723]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.552 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:38 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2561228914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:13:38.553 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:38 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2471937248' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.550+0000 7f097dffb640 1 -- 172.21.15.110:0/2046964762 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f09801089e0 con 0x7f097006cc50 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:13:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "", 2024-09-21T04:13:38.554 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 image", 2024-09-21T04:13:38.554 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:13:38.554 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 msgr2=0x7f097006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 0x7f097006f110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0980073fd0 tx=0x7f096c018040 comp rx=0 tx=0).stop 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 msgr2=0x7f0980074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0974006010 tx=0x7f09740055a0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 shutdown_connections 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0980101280 0x7f09800728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f097006cc50 0x7f097006f110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 --2- 172.21.15.110:0/2046964762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09800ff7c0 0x7f0980074250 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 >> 172.21.15.110:0/2046964762 conn(0x7f09800fb330 msgr2=0x7f0980104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:38.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 shutdown_connections 2024-09-21T04:13:38.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.553+0000 7f0986bfd640 1 -- 172.21.15.110:0/2046964762 wait complete. 2024-09-21T04:13:38.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 -- 172.21.15.110:0/3141654825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 msgr2=0x7eff08105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 --2- 172.21.15.110:0/3141654825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08105380 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efef8009880 tx=0x7efef802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 -- 172.21.15.110:0/3141654825 shutdown_connections 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 --2- 172.21.15.110:0/3141654825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08105380 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 --2- 172.21.15.110:0/3141654825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.644+0000 7eff10334640 1 -- 172.21.15.110:0/3141654825 >> 172.21.15.110:0/3141654825 conn(0x7eff080fa1c0 msgr2=0x7eff080fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 -- 172.21.15.110:0/3141654825 shutdown_connections 2024-09-21T04:13:38.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 -- 172.21.15.110:0/3141654825 wait complete. 2024-09-21T04:13:38.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 Processor -- start 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 -- start start 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.645+0000 7eff10334640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff10334640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08196120 con 0x7eff08102f90 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff10334640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08196290 con 0x7eff08100660 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0d8cf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0d8cf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47306/0 (socket says 172.21.15.110:47306) 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:60620/0 (socket says 172.21.15.110:60620) 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0d8cf640 1 -- 172.21.15.110:0/4204874313 learned_addr learned my addr 172.21.15.110:0/4204874313 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 -- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 msgr2=0x7eff08195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 -- 172.21.15.110:0/4204874313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009560 con 0x7eff08100660 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0d8cf640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7eff0e0d0640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e970 tx=0x7eff0400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.646+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff04009620 con 0x7eff08100660 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.647+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0819ad30 con 0x7eff08100660 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.647+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff040044f0 con 0x7eff08100660 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.647+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff040049d0 con 0x7eff08100660 2024-09-21T04:13:38.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.647+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0819b2a0 con 0x7eff08100660 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.648+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff08108730 con 0x7eff08100660 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.648+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7eff0400cc90 con 0x7eff08100660 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.649+0000 7efeff7fe640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 0x7efef406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.649+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7eff04014070 con 0x7eff08100660 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.649+0000 7eff0d8cf640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 0x7efef406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.649+0000 7eff0d8cf640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 0x7efef406f110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff08196b30 tx=0x7efef80314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:38.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.657+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7eff04056940 con 0x7eff08100660 2024-09-21T04:13:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:38 smithi148 ceph-mon[34957]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:38 smithi148 ceph-mon[34957]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:38 smithi148 ceph-mon[34957]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:38 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2561228914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:13:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:38 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2471937248' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:13:38.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.930+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7eff081088f0 con 0x7eff08100660 2024-09-21T04:13:38.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.930+0000 7efeff7fe640 1 -- 172.21.15.110:0/4204874313 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7eff04059f60 con 0x7eff08100660 2024-09-21T04:13:38.933 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 msgr2=0x7efef406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 0x7efef406f110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff08196b30 tx=0x7efef80314e0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 msgr2=0x7eff08195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e970 tx=0x7eff0400ee30 comp rx=0 tx=0).stop 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 shutdown_connections 2024-09-21T04:13:38.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7efef406cc50 0x7efef406f110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff08102f90 0x7eff08195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 --2- 172.21.15.110:0/4204874313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff08100660 0x7eff08195610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:38.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 >> 172.21.15.110:0/4204874313 conn(0x7eff080fa1c0 msgr2=0x7eff080fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:38.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 shutdown_connections 2024-09-21T04:13:38.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:13:38.932+0000 7eff10334640 1 -- 172.21.15.110:0/4204874313 wait complete. 2024-09-21T04:13:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:39 smithi110 ceph-mon[27723]: pgmap v224: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:39 smithi110 ceph-mon[27723]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:39 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4204874313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:13:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:39 smithi148 ceph-mon[34957]: pgmap v224: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:39 smithi148 ceph-mon[34957]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:13:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:39 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4204874313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:13:41.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:41 smithi110 ceph-mon[27723]: pgmap v225: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:41 smithi148 ceph-mon[34957]: pgmap v225: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:13:43.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:43 smithi110 ceph-mon[27723]: pgmap v226: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:43.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:43 smithi148 ceph-mon[34957]: pgmap v226: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:45.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:45 smithi110 ceph-mon[27723]: pgmap v227: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:45 smithi148 ceph-mon[34957]: pgmap v227: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:47.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:47 smithi110 ceph-mon[27723]: pgmap v228: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:47 smithi148 ceph-mon[34957]: pgmap v228: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:13:49.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:49 smithi110 ceph-mon[27723]: pgmap v229: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:49 smithi148 ceph-mon[34957]: pgmap v229: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:51.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:51 smithi110 ceph-mon[27723]: pgmap v230: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:51.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:51 smithi148 ceph-mon[34957]: pgmap v230: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:53.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:53 smithi110 ceph-mon[27723]: pgmap v231: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:53.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:53 smithi148 ceph-mon[34957]: pgmap v231: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:55.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:55 smithi110 ceph-mon[27723]: pgmap v232: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:55 smithi148 ceph-mon[34957]: pgmap v232: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:57.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:57 smithi110 ceph-mon[27723]: pgmap v233: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:57 smithi148 ceph-mon[34957]: pgmap v233: 65 pgs: 65 active+clean; 584 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:13:59.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:13:59 smithi110 ceph-mon[27723]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-21T04:13:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:13:59 smithi148 ceph-mon[34957]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-21T04:14:01.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:01 smithi148 ceph-mon[34957]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-21T04:14:01.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:01 smithi110 ceph-mon[27723]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-21T04:14:03.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:03 smithi110 ceph-mon[27723]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-21T04:14:03.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:03 smithi148 ceph-mon[34957]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-21T04:14:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:05.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:05 smithi110 ceph-mon[27723]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:05.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:05 smithi148 ceph-mon[34957]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:07.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:07 smithi110 ceph-mon[27723]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:07.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:07 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:07.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:07 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:14:07.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:07 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:07 smithi148 ceph-mon[34957]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:07 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:07 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:14:07.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:07 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:08.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:08 smithi110 ceph-mon[27723]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (unknown) 2024-09-21T04:14:08.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:08 smithi110 ceph-mon[27723]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:14:08.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:08 smithi110 ceph-mon[27723]: Upgrade: Need to upgrade myself (mgr.smithi110.axmqzb) 2024-09-21T04:14:08.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:08 smithi110 ceph-mon[27723]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 on smithi148 2024-09-21T04:14:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:08 smithi148 ceph-mon[34957]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (unknown) 2024-09-21T04:14:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:08 smithi148 ceph-mon[34957]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:14:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:08 smithi148 ceph-mon[34957]: Upgrade: Need to upgrade myself (mgr.smithi110.axmqzb) 2024-09-21T04:14:08.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:08 smithi148 ceph-mon[34957]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 on smithi148 2024-09-21T04:14:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2638165395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c68100d70 msgr2=0x7f0c681011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2638165395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c68100d70 0x7f0c681011f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009880 tx=0x7f0c5002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2638165395 shutdown_connections 2024-09-21T04:14:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2638165395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c68100d70 0x7f0c681011f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2638165395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c680ffb70 0x7f0c680fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2638165395 >> 172.21.15.110:0/2638165395 conn(0x7f0c680fb2e0 msgr2=0x7f0c680fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2638165395 shutdown_connections 2024-09-21T04:14:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.062+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2638165395 wait complete. 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.064+0000 7f0c6ceb9640 1 Processor -- start 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.064+0000 7f0c6ceb9640 1 -- start start 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.064+0000 7f0c6ceb9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6ceb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68100d70 0x7f0c68195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6ceb9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c68196170 con 0x7f0c680ffb70 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6ceb9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681962e0 con 0x7f0c68100d70 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68100d70 0x7f0c68195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43886/0 (socket says 172.21.15.110:43886) 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 -- 172.21.15.110:0/2155053198 learned_addr learned my addr 172.21.15.110:0/2155053198 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 -- 172.21.15.110:0/2155053198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68100d70 msgr2=0x7f0c68195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68100d70 0x7f0c68195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.065+0000 7f0c6659c640 1 -- 172.21.15.110:0/2155053198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009560 con 0x7f0c680ffb70 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c6659c640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800d900 tx=0x7f0c5800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c58009c00 con 0x7f0c680ffb70 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c58009d60 con 0x7f0c680ffb70 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c68072740 con 0x7f0c680ffb70 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c5800f7b0 con 0x7f0c680ffb70 2024-09-21T04:14:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.066+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c68072c60 con 0x7f0c680ffb70 2024-09-21T04:14:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.067+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c680fff70 con 0x7f0c680ffb70 2024-09-21T04:14:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.068+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f0c58010460 con 0x7f0c680ffb70 2024-09-21T04:14:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.069+0000 7f0c5f7fe640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 0x7f0c5406f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.069+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0c5808c530 con 0x7f0c680ffb70 2024-09-21T04:14:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.069+0000 7f0c5ffff640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 0x7f0c5406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.070+0000 7f0c5ffff640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 0x7f0c5406f160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c500314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.077+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0c58056880 con 0x7f0c680ffb70 2024-09-21T04:14:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:09 smithi110 ceph-mon[27723]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:09.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.446+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c6802c4f0 con 0x7f0c5406cca0 2024-09-21T04:14:09.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.448+0000 7f0c5f7fe640 1 -- 172.21.15.110:0/2155053198 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f0c6802c4f0 con 0x7f0c5406cca0 2024-09-21T04:14:09.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 msgr2=0x7f0c5406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 0x7f0c5406f160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c500314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 msgr2=0x7f0c68195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800d900 tx=0x7f0c5800ddc0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 shutdown_connections 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68100d70 0x7f0c68195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f0c5406cca0 0x7f0c5406f160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 --2- 172.21.15.110:0/2155053198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680ffb70 0x7f0c68195660 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 >> 172.21.15.110:0/2155053198 conn(0x7f0c680fb2e0 msgr2=0x7f0c681044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 shutdown_connections 2024-09-21T04:14:09.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.451+0000 7f0c6ceb9640 1 -- 172.21.15.110:0/2155053198 wait complete. 2024-09-21T04:14:09.466 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:14:09.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 -- 172.21.15.110:0/3868709554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800fea30 msgr2=0x7fe3800fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 --2- 172.21.15.110:0/3868709554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800fea30 0x7fe3800fee30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009880 tx=0x7fe37c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 -- 172.21.15.110:0/3868709554 shutdown_connections 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 --2- 172.21.15.110:0/3868709554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800ffc30 0x7fe3801000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 --2- 172.21.15.110:0/3868709554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800fea30 0x7fe3800fee30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 -- 172.21.15.110:0/3868709554 >> 172.21.15.110:0/3868709554 conn(0x7fe3800fa1c0 msgr2=0x7fe3800fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.573+0000 7fe387cce640 1 -- 172.21.15.110:0/3868709554 shutdown_connections 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 -- 172.21.15.110:0/3868709554 wait complete. 2024-09-21T04:14:09.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 Processor -- start 2024-09-21T04:14:09.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 -- start start 2024-09-21T04:14:09.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380196110 con 0x7fe3800ffc30 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.574+0000 7fe387cce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380196280 con 0x7fe3800fea30 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385a6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385a6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:46396/0 (socket says 172.21.15.110:46396) 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43908/0 (socket says 172.21.15.110:43908) 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385a6a640 1 -- 172.21.15.110:0/1540851209 learned_addr learned my addr 172.21.15.110:0/1540851209 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 -- 172.21.15.110:0/1540851209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 msgr2=0x7fe380195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 -- 172.21.15.110:0/1540851209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009560 con 0x7fe3800ffc30 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385a6a640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:14:09.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.575+0000 7fe385269640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e9a0 tx=0x7fe37000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.576+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe370009640 con 0x7fe3800ffc30 2024-09-21T04:14:09.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.576+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe37000cc90 con 0x7fe3800ffc30 2024-09-21T04:14:09.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.576+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe380100eb0 con 0x7fe3800ffc30 2024-09-21T04:14:09.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.576+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe380101420 con 0x7fe3800ffc30 2024-09-21T04:14:09.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.577+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe370018810 con 0x7fe3800ffc30 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.577+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364005280 con 0x7fe3800ffc30 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.578+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe370018a30 con 0x7fe3800ffc30 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.578+0000 7fe376ffd640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 0x7fe36c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.578+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe370014070 con 0x7fe3800ffc30 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.578+0000 7fe385a6a640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 0x7fe36c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.579+0000 7fe385a6a640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 0x7fe36c06f110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe37c000c00 tx=0x7fe37c0314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.586+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe370052e20 con 0x7fe3800ffc30 2024-09-21T04:14:09.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:09 smithi148 ceph-mon[34957]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-21T04:14:09.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.776+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe364000bf0 con 0x7fe36c06cc50 2024-09-21T04:14:09.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.778+0000 7fe376ffd640 1 -- 172.21.15.110:0/1540851209 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fe364000bf0 con 0x7fe36c06cc50 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 msgr2=0x7fe36c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 0x7fe36c06f110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe37c000c00 tx=0x7fe37c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 msgr2=0x7fe380195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e9a0 tx=0x7fe37000ee60 comp rx=0 tx=0).stop 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 shutdown_connections 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe36c06cc50 0x7fe36c06f110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3800ffc30 0x7fe380195b40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 --2- 172.21.15.110:0/1540851209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3800fea30 0x7fe380195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 >> 172.21.15.110:0/1540851209 conn(0x7fe3800fa1c0 msgr2=0x7fe3800fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:09.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 shutdown_connections 2024-09-21T04:14:09.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.780+0000 7fe387cce640 1 -- 172.21.15.110:0/1540851209 wait complete. 2024-09-21T04:14:09.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2679913950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180ffc50 msgr2=0x7f8b181000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2679913950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180ffc50 0x7f8b181000d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009880 tx=0x7f8b0802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2679913950 shutdown_connections 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2679913950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180ffc50 0x7f8b181000d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2679913950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180fea50 0x7f8b180fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.904+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2679913950 >> 172.21.15.110:0/2679913950 conn(0x7f8b180fa1c0 msgr2=0x7f8b180fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.905+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2679913950 shutdown_connections 2024-09-21T04:14:09.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.905+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2679913950 wait complete. 2024-09-21T04:14:09.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.905+0000 7f8b1f9f6640 1 Processor -- start 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.905+0000 7f8b1f9f6640 1 -- start start 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1f9f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 0x7f8b18068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1f9f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 0x7f8b1819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1f9f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1819a8f0 con 0x7f8b180fea50 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1f9f6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1819aa60 con 0x7f8b180ffc50 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1cf91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 0x7f8b1819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1d792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 0x7f8b18068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1cf91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 0x7f8b1819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:46408/0 (socket says 172.21.15.110:46408) 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1cf91640 1 -- 172.21.15.110:0/2814565806 learned_addr learned my addr 172.21.15.110:0/2814565806 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:09.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1d792640 1 -- 172.21.15.110:0/2814565806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 msgr2=0x7f8b1819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1d792640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 0x7f8b1819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.906+0000 7f8b1d792640 1 -- 172.21.15.110:0/2814565806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08009560 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b1d792640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 0x7f8b18068fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b180ffab0 tx=0x7f8b1400dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b14009a30 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b18194150 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b181946a0 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b14009b90 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.907+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1400f680 con 0x7f8b180fea50 2024-09-21T04:14:09.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.908+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc005280 con 0x7f8b180fea50 2024-09-21T04:14:09.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.910+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8b14009d00 con 0x7f8b180fea50 2024-09-21T04:14:09.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.910+0000 7f8b0e7fc640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 0x7f8b0406f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:09.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.910+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8b1408c010 con 0x7f8b180fea50 2024-09-21T04:14:09.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.910+0000 7f8b1cf91640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 0x7f8b0406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:09.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.911+0000 7f8b1cf91640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 0x7f8b0406f160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8b18068d20 tx=0x7f8b080314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:09.919+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8b140bfe20 con 0x7f8b180fea50 2024-09-21T04:14:10.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.272+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8afc000bf0 con 0x7f8b0406cca0 2024-09-21T04:14:10.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.283+0000 7f8b0e7fc640 1 -- 172.21.15.110:0/2814565806 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f8afc000bf0 con 0x7f8b0406cca0 2024-09-21T04:14:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:14:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (5m) 62s ago 7m 25.3M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (7m) 62s ago 7m 7524k - 18.2.4-871-g6a1bdf85 0adab276c97e b0186c860d81 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (6m) 68s ago 6m 7247k - 18.2.4-871-g6a1bdf85 0adab276c97e 6e1e8fb80a91 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (7m) 62s ago 7m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (6m) 68s ago 6m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 62s ago 7m 84.8M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (80s) 62s ago 80s 16.3M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (87s) 62s ago 87s 11.8M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (78s) 68s ago 77s 13.6M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (85s) 68s ago 84s 15.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (8m) 62s ago 8m 486M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (6m) 68s ago 6m 440M - 18.2.4-871-g6a1bdf85 0adab276c97e 4f74b6e9b876 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (8m) 62s ago 8m 47.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:14:10.287 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (6m) 68s ago 6m 36.0M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (7m) 62s ago 7m 16.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 68s ago 6m 15.7M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 62s ago 5m 57.7M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 62s ago 4m 54.6M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 62s ago 4m 53.1M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (3m) 68s ago 3m 53.1M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (3m) 68s ago 3m 49.9M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (2m) 68s ago 2m 53.7M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:14:10.288 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 62s ago 6m 59.4M - 2.43.0 a07b618ecd1d 2d686ff7a6ad 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.285+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 msgr2=0x7f8b0406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.285+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 0x7f8b0406f160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8b18068d20 tx=0x7f8b080314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.285+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 msgr2=0x7f8b18068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.285+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 0x7f8b18068fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b180ffab0 tx=0x7f8b1400dd60 comp rx=0 tx=0).stop 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 shutdown_connections 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b180ffc50 0x7f8b1819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8b0406cca0 0x7f8b0406f160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 --2- 172.21.15.110:0/2814565806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b180fea50 0x7f8b18068fa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 >> 172.21.15.110:0/2814565806 conn(0x7f8b180fa1c0 msgr2=0x7f8b18103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 shutdown_connections 2024-09-21T04:14:10.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.286+0000 7f8b1f9f6640 1 -- 172.21.15.110:0/2814565806 wait complete. 2024-09-21T04:14:10.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 -- 172.21.15.110:0/4086883779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f201005c0 msgr2=0x7f9f201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 --2- 172.21.15.110:0/4086883779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f201005c0 0x7f9f201029b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9f100098b0 tx=0x7f9f1002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 -- 172.21.15.110:0/4086883779 shutdown_connections 2024-09-21T04:14:10.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 --2- 172.21.15.110:0/4086883779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f20102ef0 0x7f9f201052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 --2- 172.21.15.110:0/4086883779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f201005c0 0x7f9f201029b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 -- 172.21.15.110:0/4086883779 >> 172.21.15.110:0/4086883779 conn(0x7f9f200fa1c0 msgr2=0x7f9f200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:10.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 -- 172.21.15.110:0/4086883779 shutdown_connections 2024-09-21T04:14:10.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.369+0000 7f9f2691c640 1 -- 172.21.15.110:0/4086883779 wait complete. 2024-09-21T04:14:10.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.370+0000 7f9f2691c640 1 Processor -- start 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.370+0000 7f9f2691c640 1 -- start start 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.370+0000 7f9f2691c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f201005c0 0x7f9f200fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.370+0000 7f9f2691c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f2691c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200feab0 con 0x7f9f20102ef0 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f2691c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200fec20 con 0x7f9f201005c0 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43940/0 (socket says 172.21.15.110:43940) 2024-09-21T04:14:10.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 -- 172.21.15.110:0/1202821482 learned_addr learned my addr 172.21.15.110:0/1202821482 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:10.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1ffff640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f201005c0 0x7f9f200fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 -- 172.21.15.110:0/1202821482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f201005c0 msgr2=0x7f9f200fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f201005c0 0x7f9f200fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 -- 172.21.15.110:0/1202821482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10009560 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1f7fe640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d6f0 tx=0x7f9f1400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14009a30 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f14009b90 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.371+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f200fef00 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.372+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1400f680 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.372+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2019d6c0 con 0x7f9f20102ef0 2024-09-21T04:14:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.373+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f201009a0 con 0x7f9f20102ef0 2024-09-21T04:14:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.373+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f9f14009d00 con 0x7f9f20102ef0 2024-09-21T04:14:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.374+0000 7f9f1d7fa640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 0x7f9f0806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.374+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9f1408c0c0 con 0x7f9f20102ef0 2024-09-21T04:14:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.374+0000 7f9f1ffff640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 0x7f9f0806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.375+0000 7f9f1ffff640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 0x7f9f0806f110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006010 tx=0x7f9f100314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:10.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.378+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9f14056430 con 0x7f9f20102ef0 2024-09-21T04:14:10.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:10 smithi110 ceph-mon[27723]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:10.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:10 smithi148 ceph-mon[34957]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:10.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.757+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9f201088f0 con 0x7f9f20102ef0 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.758+0000 7f9f1d7fa640 1 -- 172.21.15.110:0/1202821482 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f9f14059a50 con 0x7f9f20102ef0 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:14:10.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 14 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:14:10.762 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 msgr2=0x7f9f0806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 0x7f9f0806f110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006010 tx=0x7f9f100314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 msgr2=0x7f9f200fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400d6f0 tx=0x7f9f1400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 shutdown_connections 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f9f0806cc50 0x7f9f0806f110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f20102ef0 0x7f9f200fe570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 --2- 172.21.15.110:0/1202821482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f201005c0 0x7f9f200fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 >> 172.21.15.110:0/1202821482 conn(0x7f9f200fa1c0 msgr2=0x7f9f200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:10.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 shutdown_connections 2024-09-21T04:14:10.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.761+0000 7f9f2691c640 1 -- 172.21.15.110:0/1202821482 wait complete. 2024-09-21T04:14:10.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 -- 172.21.15.110:0/1969256794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c102ef0 msgr2=0x7f4b3c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 --2- 172.21.15.110:0/1969256794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c1052e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0098b0 tx=0x7f4b2c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 -- 172.21.15.110:0/1969256794 shutdown_connections 2024-09-21T04:14:10.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 --2- 172.21.15.110:0/1969256794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c1052e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 --2- 172.21.15.110:0/1969256794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 -- 172.21.15.110:0/1969256794 >> 172.21.15.110:0/1969256794 conn(0x7f4b3c0fa1c0 msgr2=0x7f4b3c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:10.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 -- 172.21.15.110:0/1969256794 shutdown_connections 2024-09-21T04:14:10.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.845+0000 7f4b4421a640 1 -- 172.21.15.110:0/1969256794 wait complete. 2024-09-21T04:14:10.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 Processor -- start 2024-09-21T04:14:10.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 -- start start 2024-09-21T04:14:10.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c0fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0feab0 con 0x7f4b3c1005c0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b4421a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0fec20 con 0x7f4b3c102ef0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b41fb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b41fb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c0fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43960/0 (socket says 172.21.15.110:43960) 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b417b5640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c0fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.846+0000 7f4b41fb6640 1 -- 172.21.15.110:0/2203083989 learned_addr learned my addr 172.21.15.110:0/2203083989 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b417b5640 1 -- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 msgr2=0x7f4b3c0fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b417b5640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c0fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b417b5640 1 -- 172.21.15.110:0/2203083989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009560 con 0x7f4b3c102ef0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b417b5640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c0fe570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c02a9f0 tx=0x7f4b2c0055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c004aa0 con 0x7f4b3c102ef0 2024-09-21T04:14:10.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c0feea0 con 0x7f4b3c102ef0 2024-09-21T04:14:10.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b2c004c00 con 0x7f4b3c102ef0 2024-09-21T04:14:10.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c003a80 con 0x7f4b3c102ef0 2024-09-21T04:14:10.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.847+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c19d6a0 con 0x7f4b3c102ef0 2024-09-21T04:14:10.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.849+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c1009a0 con 0x7f4b3c102ef0 2024-09-21T04:14:10.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.850+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f4b2c031c90 con 0x7f4b3c102ef0 2024-09-21T04:14:10.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.850+0000 7f4b32ffd640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 0x7f4b2806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:10.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.850+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4b2c0ad3a0 con 0x7f4b3c102ef0 2024-09-21T04:14:10.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.850+0000 7f4b41fb6640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 0x7f4b2806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:10.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.851+0000 7f4b41fb6640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 0x7f4b2806f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b3800b480 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:10.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:10.859+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4b2c077760 con 0x7f4b3c102ef0 2024-09-21T04:14:11.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.180+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b3c108720 con 0x7f4b3c102ef0 2024-09-21T04:14:11.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.181+0000 7f4b32ffd640 1 -- 172.21.15.110:0/2203083989 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f4b2c07ad80 con 0x7f4b3c102ef0 2024-09-21T04:14:11.269 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:14:11.269 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:14:11.270 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:14:11.271 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:14:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:11.273 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:11.273 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:11.273 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:11.273 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 msgr2=0x7f4b2806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 0x7f4b2806f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b3800b480 comp rx=0 tx=0).stop 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 msgr2=0x7f4b3c0fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c0fe570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c02a9f0 tx=0x7f4b2c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 shutdown_connections 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c0fe570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f4b2806cc50 0x7f4b2806f110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 --2- 172.21.15.110:0/2203083989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b3c1005c0 0x7f4b3c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 >> 172.21.15.110:0/2203083989 conn(0x7f4b3c0fa1c0 msgr2=0x7f4b3c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 shutdown_connections 2024-09-21T04:14:11.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.184+0000 7f4b4421a640 1 -- 172.21.15.110:0/2203083989 wait complete. 2024-09-21T04:14:11.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 -- 172.21.15.110:0/2928435441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80072e00 msgr2=0x7f8d80073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 --2- 172.21.15.110:0/2928435441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80072e00 0x7f8d80073260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009880 tx=0x7f8d7402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 -- 172.21.15.110:0/2928435441 shutdown_connections 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 --2- 172.21.15.110:0/2928435441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80072e00 0x7f8d80073260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 --2- 172.21.15.110:0/2928435441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80074460 0x7f8d800728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 -- 172.21.15.110:0/2928435441 >> 172.21.15.110:0/2928435441 conn(0x7f8d800fb290 msgr2=0x7f8d800fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 -- 172.21.15.110:0/2928435441 shutdown_connections 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.271+0000 7f8d85852640 1 -- 172.21.15.110:0/2928435441 wait complete. 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 Processor -- start 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 -- start start 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80072e00 0x7f8d801977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801982d0 con 0x7f8d80074460 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d7e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d7e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43986/0 (socket says 172.21.15.110:43986) 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d7e7fc640 1 -- 172.21.15.110:0/2669656663 learned_addr learned my addr 172.21.15.110:0/2669656663 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.272+0000 7f8d85852640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80198440 con 0x7f8d80072e00 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d7effd640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80072e00 0x7f8d801977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d7e7fc640 1 -- 172.21.15.110:0/2669656663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80072e00 msgr2=0x7f8d801977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d7e7fc640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80072e00 0x7f8d801977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d7e7fc640 1 -- 172.21.15.110:0/2669656663 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009560 con 0x7f8d80074460 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d7e7fc640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d740055a0 tx=0x7f8d740055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:11.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d74004b20 con 0x7f8d80074460 2024-09-21T04:14:11.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d74004c80 con 0x7f8d80074460 2024-09-21T04:14:11.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d74003bc0 con 0x7f8d80074460 2024-09-21T04:14:11.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.273+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8019ce30 con 0x7f8d80074460 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.274+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8019d370 con 0x7f8d80074460 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.275+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80073260 con 0x7f8d80074460 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.275+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8d74031640 con 0x7f8d80074460 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.276+0000 7f8d84850640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 0x7f8d6806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.276+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8d740ad210 con 0x7f8d80074460 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.276+0000 7f8d7effd640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 0x7f8d6806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.277+0000 7f8d7effd640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 0x7f8d6806f110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d70005fd0 tx=0x7f8d70005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:11.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.280+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8d74077580 con 0x7f8d80074460 2024-09-21T04:14:11.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.486+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d8002c4f0 con 0x7f8d6806cc50 2024-09-21T04:14:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:11 smithi110 ceph-mon[27723]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:11 smithi110 ceph-mon[27723]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:14:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:11 smithi110 ceph-mon[27723]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:11.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:11 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1202821482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:14:11.492 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:11 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2203083989' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "0/23 daemons upgraded", 2024-09-21T04:14:11.492 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 image on host smithi148", 2024-09-21T04:14:11.493 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:14:11.493 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:14:11.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.487+0000 7f8d84850640 1 -- 172.21.15.110:0/2669656663 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f8d8002c4f0 con 0x7f8d6806cc50 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 msgr2=0x7f8d6806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 0x7f8d6806f110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d70005fd0 tx=0x7f8d70005850 comp rx=0 tx=0).stop 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 msgr2=0x7f8d80197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d740055a0 tx=0x7f8d740055d0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 shutdown_connections 2024-09-21T04:14:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8d6806cc50 0x7f8d6806f110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d80074460 0x7f8d80197d00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 --2- 172.21.15.110:0/2669656663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d80072e00 0x7f8d801977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 >> 172.21.15.110:0/2669656663 conn(0x7f8d800fb290 msgr2=0x7f8d80105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 shutdown_connections 2024-09-21T04:14:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.491+0000 7f8d85852640 1 -- 172.21.15.110:0/2669656663 wait complete. 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 -- 172.21.15.110:0/4287975154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d00ffb40 msgr2=0x7f05d00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 --2- 172.21.15.110:0/4287975154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d00ffb40 0x7f05d00fff40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009880 tx=0x7f05c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 -- 172.21.15.110:0/4287975154 shutdown_connections 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 --2- 172.21.15.110:0/4287975154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 --2- 172.21.15.110:0/4287975154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d00ffb40 0x7f05d00fff40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 -- 172.21.15.110:0/4287975154 >> 172.21.15.110:0/4287975154 conn(0x7f05d00fb2f0 msgr2=0x7f05d00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 -- 172.21.15.110:0/4287975154 shutdown_connections 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.575+0000 7f05d57e2640 1 -- 172.21.15.110:0/4287975154 wait complete. 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 Processor -- start 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 -- start start 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d0195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d01961f0 con 0x7f05d0195d70 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05d57e2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0196360 con 0x7f05d0100d40 2024-09-21T04:14:11.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05ce7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05ce7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44000/0 (socket says 172.21.15.110:44000) 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05ce7fc640 1 -- 172.21.15.110:0/3249636610 learned_addr learned my addr 172.21.15.110:0/3249636610 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.576+0000 7f05ceffd640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d0195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05ce7fc640 1 -- 172.21.15.110:0/3249636610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 msgr2=0x7f05d0195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05ce7fc640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d0195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05ce7fc640 1 -- 172.21.15.110:0/3249636610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009560 con 0x7f05d0195d70 2024-09-21T04:14:11.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05ceffd640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d0195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:14:11.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05ce7fc640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05b800e970 tx=0x7f05b800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:11.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b800cc90 con 0x7f05d0195d70 2024-09-21T04:14:11.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d019b380 con 0x7f05d0195d70 2024-09-21T04:14:11.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05b800cdf0 con 0x7f05d0195d70 2024-09-21T04:14:11.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.577+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b8010430 con 0x7f05d0195d70 2024-09-21T04:14:11.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.578+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d019b8a0 con 0x7f05d0195d70 2024-09-21T04:14:11.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.579+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f05b8010590 con 0x7f05d0195d70 2024-09-21T04:14:11.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.579+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d0108750 con 0x7f05d0195d70 2024-09-21T04:14:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.579+0000 7f05c7fff640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 0x7f05bc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.580+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f05b8014070 con 0x7f05d0195d70 2024-09-21T04:14:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.580+0000 7f05ceffd640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 0x7f05bc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:11.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.580+0000 7f05ceffd640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 0x7f05bc06f110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05c0008000 tx=0x7f05c00314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:11.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:11.588+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f05b8091060 con 0x7f05d0195d70 2024-09-21T04:14:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:11 smithi148 ceph-mon[34957]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:11 smithi148 ceph-mon[34957]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-21T04:14:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:11 smithi148 ceph-mon[34957]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:11 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1202821482' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:14:11.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:11 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2203083989' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:14:12.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.056+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f05d0108930 con 0x7f05d0195d70 2024-09-21T04:14:12.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.057+0000 7f05c7fff640 1 -- 172.21.15.110:0/3249636610 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f05b8056350 con 0x7f05d0195d70 2024-09-21T04:14:12.059 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:14:12.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 msgr2=0x7f05bc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:12.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 0x7f05bc06f110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05c0008000 tx=0x7f05c00314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:12.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 msgr2=0x7f05d019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:12.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05b800e970 tx=0x7f05b800ed70 comp rx=0 tx=0).stop 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 shutdown_connections 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f05bc06cc50 0x7f05bc06f110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d0195d70 0x7f05d019ade0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 --2- 172.21.15.110:0/3249636610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0100d40 0x7f05d0195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.059+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 >> 172.21.15.110:0/3249636610 conn(0x7f05d00fb2f0 msgr2=0x7f05d0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:12.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.060+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 shutdown_connections 2024-09-21T04:14:12.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:12.060+0000 7f05d57e2640 1 -- 172.21.15.110:0/3249636610 wait complete. 2024-09-21T04:14:12.324 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:12 smithi110 ceph-mon[27723]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:12.324 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:12 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3249636610' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:14:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:12 smithi148 ceph-mon[34957]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:12 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3249636610' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:14:13.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:13 smithi110 ceph-mon[27723]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:14:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:13 smithi148 ceph-mon[34957]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:14:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:15 smithi110 ceph-mon[27723]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:14:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:15 smithi148 ceph-mon[34957]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:14:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:17 smithi110 ceph-mon[27723]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:17 smithi148 ceph-mon[34957]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:18.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:18 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:18.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:18 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:19.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:19 smithi110 ceph-mon[27723]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:19.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:19 smithi148 ceph-mon[34957]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:21 smithi110 ceph-mon[27723]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:21.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:21 smithi148 ceph-mon[34957]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:23.280 INFO:tasks.workunit.client.1.smithi148.stderr:Updating files: 89% (11319/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T04:14:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:23 smithi110 ceph-mon[27723]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:23.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:23 smithi148 ceph-mon[34957]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:23.915 INFO:tasks.workunit.client.1.smithi148.stderr:Note: switching to 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'. 2024-09-21T04:14:23.915 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.915 INFO:tasks.workunit.client.1.smithi148.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T04:14:23.915 INFO:tasks.workunit.client.1.smithi148.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: git switch -c 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:Or undo this operation with: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: git switch - 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-09-21T04:14:23.916 INFO:tasks.workunit.client.1.smithi148.stderr:HEAD is now at f36283e7c91 ceph-volume: fix OSD lvm/tpm2 activation 2024-09-21T04:14:23.921 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-21T04:14:23.984 INFO:tasks.workunit.client.1.smithi148.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T04:14:23.986 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-21T04:14:23.986 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T04:14:24.674 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T04:14:24.740 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T04:14:24.782 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-21T04:14:24.783 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-21T04:14:24.784 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T04:14:24.825 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-21T04:14:24.829 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:14:24.829 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-21T04:14:24.885 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-21T04:14:24.887 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-21T04:14:24.887 DEBUG:teuthology.orchestra.run.smithi148:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-21T04:14:24.947 INFO:tasks.workunit.client.1.smithi148.stderr:+ mkdir -p fsstress 2024-09-21T04:14:24.949 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd fsstress 2024-09-21T04:14:24.949 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:14:24.949 INFO:tasks.workunit.client.1.smithi148.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-21T04:14:25.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:25 smithi110 ceph-mon[27723]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:25.466 INFO:tasks.workunit.client.1.smithi148.stderr:+ tar xzf ltp-full.tgz 2024-09-21T04:14:25.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:25 smithi148 ceph-mon[34957]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:26.557 INFO:tasks.workunit.client.0.smithi110.stderr:Updating files: 92% (11726/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:Note: switching to 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'. 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T04:14:27.416 INFO:tasks.workunit.client.0.smithi110.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: git switch -c 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr:Or undo this operation with: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: git switch - 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-21T04:14:27.417 INFO:tasks.workunit.client.0.smithi110.stderr:HEAD is now at f36283e7c91 ceph-volume: fix OSD lvm/tpm2 activation 2024-09-21T04:14:27.422 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T04:14:27.444 INFO:tasks.workunit.client.0.smithi110.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T04:14:27.446 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:14:27.446 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T04:14:27.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:27 smithi110 ceph-mon[27723]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:27 smithi148 ceph-mon[34957]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 211 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:14:28.082 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T04:14:28.156 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T04:14:28.196 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:14:28.198 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:14:28.198 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T04:14:28.238 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:14:28.242 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:14:28.242 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T04:14:28.298 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-21T04:14:28.299 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-21T04:14:28.300 DEBUG:teuthology.orchestra.run.smithi110:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-21T04:14:28.363 INFO:tasks.workunit.client.0.smithi110.stderr:+ mkdir -p fsstress 2024-09-21T04:14:28.365 INFO:tasks.workunit.client.0.smithi110.stderr:+ pushd fsstress 2024-09-21T04:14:28.365 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:14:28.366 INFO:tasks.workunit.client.0.smithi110.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-21T04:14:28.996 INFO:tasks.workunit.client.0.smithi110.stderr:+ tar xzf ltp-full.tgz 2024-09-21T04:14:29.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:29 smithi110 ceph-mon[27723]: pgmap v249: 65 pgs: 65 active+clean; 4.6 MiB data, 215 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-09-21T04:14:29.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:29 smithi148 ceph-mon[34957]: pgmap v249: 65 pgs: 65 active+clean; 4.6 MiB data, 215 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-09-21T04:14:31.652 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:31 smithi148 ceph-mon[34957]: pgmap v250: 65 pgs: 65 active+clean; 8.6 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s wr, 0 op/s 2024-09-21T04:14:31.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:31 smithi110 ceph-mon[27723]: pgmap v250: 65 pgs: 65 active+clean; 8.6 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s wr, 0 op/s 2024-09-21T04:14:33.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:33 smithi110 ceph-mon[27723]: pgmap v251: 65 pgs: 65 active+clean; 30 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s wr, 204 op/s 2024-09-21T04:14:33.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:33 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:33 smithi148 ceph-mon[34957]: pgmap v251: 65 pgs: 65 active+clean; 30 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s wr, 204 op/s 2024-09-21T04:14:33.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:33 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:35 smithi110 ceph-mon[27723]: pgmap v252: 65 pgs: 65 active+clean; 51 MiB data, 381 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 309 op/s 2024-09-21T04:14:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:35 smithi148 ceph-mon[34957]: pgmap v252: 65 pgs: 65 active+clean; 51 MiB data, 381 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 309 op/s 2024-09-21T04:14:37.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:37 smithi110 ceph-mon[27723]: pgmap v253: 65 pgs: 65 active+clean; 57 MiB data, 417 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 344 op/s 2024-09-21T04:14:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:37 smithi148 ceph-mon[34957]: pgmap v253: 65 pgs: 65 active+clean; 57 MiB data, 417 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 344 op/s 2024-09-21T04:14:39.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:39 smithi110 ceph-mon[27723]: pgmap v254: 65 pgs: 65 active+clean; 82 MiB data, 483 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 434 op/s 2024-09-21T04:14:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:39 smithi148 ceph-mon[34957]: pgmap v254: 65 pgs: 65 active+clean; 82 MiB data, 483 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 434 op/s 2024-09-21T04:14:41.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:41 smithi110 ceph-mon[27723]: pgmap v255: 65 pgs: 65 active+clean; 83 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s wr, 455 op/s 2024-09-21T04:14:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:41 smithi148 ceph-mon[34957]: pgmap v255: 65 pgs: 65 active+clean; 83 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s wr, 455 op/s 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 -- 172.21.15.110:0/123214136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 msgr2=0x7f823c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 --2- 172.21.15.110:0/123214136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c102a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8238009880 tx=0x7f823802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 -- 172.21.15.110:0/123214136 shutdown_connections 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 --2- 172.21.15.110:0/123214136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 --2- 172.21.15.110:0/123214136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c102a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 -- 172.21.15.110:0/123214136 >> 172.21.15.110:0/123214136 conn(0x7f823c0fa1c0 msgr2=0x7f823c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 -- 172.21.15.110:0/123214136 shutdown_connections 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.160+0000 7f8244187640 1 -- 172.21.15.110:0/123214136 wait complete. 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 Processor -- start 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 -- start start 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1960d0 con 0x7f823c100620 2024-09-21T04:14:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8244187640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c196240 con 0x7f823c102f50 2024-09-21T04:14:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.161+0000 7f8241f23640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241722640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33310/0 (socket says 172.21.15.110:33310) 2024-09-21T04:14:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 -- 172.21.15.110:0/757606758 learned_addr learned my addr 172.21.15.110:0/757606758 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 -- 172.21.15.110:0/757606758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 msgr2=0x7f823c195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 -- 172.21.15.110:0/757606758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238009560 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241722640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8241f23640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8238005520 tx=0x7f8238005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8238004aa0 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c19ace0 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8238004c00 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8238003b10 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.162+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c0fe660 con 0x7f823c100620 2024-09-21T04:14:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.164+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c108700 con 0x7f823c100620 2024-09-21T04:14:42.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.164+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f8238003d40 con 0x7f823c100620 2024-09-21T04:14:42.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.165+0000 7f8232ffd640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 0x7f8228073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.165+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f82380ad380 con 0x7f823c100620 2024-09-21T04:14:42.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.165+0000 7f8241722640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 0x7f8228073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.165+0000 7f8241722640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 0x7f8228073520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f823c196ae0 tx=0x7f822c00b390 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.172+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f82380777a0 con 0x7f823c100620 2024-09-21T04:14:42.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.371+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f823c1039b0 con 0x7f8228071060 2024-09-21T04:14:42.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.373+0000 7f8232ffd640 1 -- 172.21.15.110:0/757606758 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f823c1039b0 con 0x7f8228071060 2024-09-21T04:14:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 msgr2=0x7f8228073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 0x7f8228073520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f823c196ae0 tx=0x7f822c00b390 comp rx=0 tx=0).stop 2024-09-21T04:14:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 msgr2=0x7f823c1955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8238005520 tx=0x7f8238005550 comp rx=0 tx=0).stop 2024-09-21T04:14:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 shutdown_connections 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f823c102f50 0x7f823c195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f8228071060 0x7f8228073520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 --2- 172.21.15.110:0/757606758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823c100620 0x7f823c1955c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 >> 172.21.15.110:0/757606758 conn(0x7f823c0fa1c0 msgr2=0x7f823c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 shutdown_connections 2024-09-21T04:14:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.375+0000 7f8244187640 1 -- 172.21.15.110:0/757606758 wait complete. 2024-09-21T04:14:42.386 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:14:42.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 -- 172.21.15.110:0/1810364868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef80ffb40 msgr2=0x7feef80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 --2- 172.21.15.110:0/1810364868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef80ffb40 0x7feef80fff40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feee8009880 tx=0x7feee802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 -- 172.21.15.110:0/1810364868 shutdown_connections 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 --2- 172.21.15.110:0/1810364868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef8100d40 0x7feef81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 --2- 172.21.15.110:0/1810364868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef80ffb40 0x7feef80fff40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 -- 172.21.15.110:0/1810364868 >> 172.21.15.110:0/1810364868 conn(0x7feef80fb2f0 msgr2=0x7feef80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 -- 172.21.15.110:0/1810364868 shutdown_connections 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.461+0000 7feefddbf640 1 -- 172.21.15.110:0/1810364868 wait complete. 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 Processor -- start 2024-09-21T04:14:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 -- start start 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef80ffb40 0x7feef8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef8072de0 con 0x7feef8100d40 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feefddbf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef8072f50 con 0x7feef80ffb40 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feef6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feef6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33338/0 (socket says 172.21.15.110:33338) 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feef6ffd640 1 -- 172.21.15.110:0/1808346308 learned_addr learned my addr 172.21.15.110:0/1808346308 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:42.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.462+0000 7feef77fe640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef80ffb40 0x7feef8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef6ffd640 1 -- 172.21.15.110:0/1808346308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef80ffb40 msgr2=0x7feef8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef6ffd640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef80ffb40 0x7feef8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef6ffd640 1 -- 172.21.15.110:0/1808346308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009560 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef6ffd640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d8d0 tx=0x7feeec00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec009c00 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef8073230 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec009d60 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef819d650 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.463+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec0049d0 con 0x7feef8100d40 2024-09-21T04:14:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.464+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee4005280 con 0x7feef8100d40 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.465+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7feeec004020 con 0x7feef8100d40 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.465+0000 7feef4ff9640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 0x7feee006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.465+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7feeec08bf80 con 0x7feef8100d40 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.465+0000 7feef77fe640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 0x7feee006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.466+0000 7feef77fe640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 0x7feee006f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feee8009230 tx=0x7feee80314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.470+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feeec0562f0 con 0x7feef8100d40 2024-09-21T04:14:42.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.669+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feee4000bf0 con 0x7feee006cc50 2024-09-21T04:14:42.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.671+0000 7feef4ff9640 1 -- 172.21.15.110:0/1808346308 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7feee4000bf0 con 0x7feee006cc50 2024-09-21T04:14:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 msgr2=0x7feee006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 0x7feee006f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feee8009230 tx=0x7feee80314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 msgr2=0x7feef80728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d8d0 tx=0x7feeec00dd90 comp rx=0 tx=0).stop 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 shutdown_connections 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7feee006cc50 0x7feee006f110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef8100d40 0x7feef80728a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 --2- 172.21.15.110:0/1808346308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef80ffb40 0x7feef8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 >> 172.21.15.110:0/1808346308 conn(0x7feef80fb2f0 msgr2=0x7feef8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 shutdown_connections 2024-09-21T04:14:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.673+0000 7feefddbf640 1 -- 172.21.15.110:0/1808346308 wait complete. 2024-09-21T04:14:42.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 -- 172.21.15.110:0/3042545470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fe430 msgr2=0x7f03a40fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3042545470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fe430 0x7f03a40fe830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009880 tx=0x7f03a002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 -- 172.21.15.110:0/3042545470 shutdown_connections 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3042545470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fed70 0x7f03a40ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3042545470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fe430 0x7f03a40fe830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 -- 172.21.15.110:0/3042545470 >> 172.21.15.110:0/3042545470 conn(0x7f03a40fa1c0 msgr2=0x7f03a40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 -- 172.21.15.110:0/3042545470 shutdown_connections 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.768+0000 7f03abf0b640 1 -- 172.21.15.110:0/3042545470 wait complete. 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 Processor -- start 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 -- start start 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fe430 0x7f03a41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41982f0 con 0x7f03a40fed70 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03abf0b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4198460 con 0x7f03a40fe430 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03a94a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03a94a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33354/0 (socket says 172.21.15.110:33354) 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03a94a6640 1 -- 172.21.15.110:0/3476253908 learned_addr learned my addr 172.21.15.110:0/3476253908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.769+0000 7f03a9ca7640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fe430 0x7f03a41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.770+0000 7f03a94a6640 1 -- 172.21.15.110:0/3476253908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fe430 msgr2=0x7f03a41977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.770+0000 7f03a94a6640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fe430 0x7f03a41977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.770+0000 7f03a94a6640 1 -- 172.21.15.110:0/3476253908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0009560 con 0x7f03a40fed70 2024-09-21T04:14:42.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.770+0000 7f03a94a6640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f039400e9a0 tx=0x7f039400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.770+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039400cd50 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.771+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f039400ceb0 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.771+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0394010660 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.771+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a419cf00 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.771+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a419d440 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.772+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a40fe830 con 0x7f03a40fed70 2024-09-21T04:14:42.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.773+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f03940048a0 con 0x7f03a40fed70 2024-09-21T04:14:42.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.773+0000 7f039affd640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 0x7f039006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.773+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0394014070 con 0x7f03a40fed70 2024-09-21T04:14:42.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.773+0000 7f03a9ca7640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 0x7f039006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.774+0000 7f03a9ca7640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 0x7f039006f110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03a0000c00 tx=0x7f03a00314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.780+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0394056b00 con 0x7f03a40fed70 2024-09-21T04:14:42.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.983+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a4108180 con 0x7f039006cc50 2024-09-21T04:14:42.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.993+0000 7f039affd640 1 -- 172.21.15.110:0/3476253908 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f03a4108180 con 0x7f039006cc50 2024-09-21T04:14:42.996 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:14:42.996 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (6m) 94s ago 7m 25.3M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:14:42.996 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (8m) 94s ago 8m 7524k - 18.2.4-871-g6a1bdf85 0adab276c97e b0186c860d81 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (7m) 101s ago 7m 7247k - 18.2.4-871-g6a1bdf85 0adab276c97e 6e1e8fb80a91 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 94s ago 8m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (7m) 101s ago 7m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 94s ago 7m 84.8M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (113s) 94s ago 112s 16.3M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (2m) 94s ago 119s 11.8M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (110s) 101s ago 110s 13.6M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (117s) 101s ago 117s 15.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (9m) 94s ago 9m 486M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (7m) 101s ago 7m 440M - 18.2.4-871-g6a1bdf85 0adab276c97e 4f74b6e9b876 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 94s ago 9m 47.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (6m) 101s ago 6m 36.0M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (8m) 94s ago 8m 16.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:14:42.997 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 101s ago 7m 15.7M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 94s ago 5m 57.7M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 94s ago 5m 54.6M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 94s ago 4m 53.1M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (4m) 101s ago 4m 53.1M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (3m) 101s ago 3m 49.9M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (3m) 101s ago 3m 53.7M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:14:42.998 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 94s ago 7m 59.4M - 2.43.0 a07b618ecd1d 2d686ff7a6ad 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.995+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 msgr2=0x7f039006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 0x7f039006f110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03a0000c00 tx=0x7f03a00314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 msgr2=0x7f03a4197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f039400e9a0 tx=0x7f039400eda0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 shutdown_connections 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f039006cc50 0x7f039006f110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a40fed70 0x7f03a4197d20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 --2- 172.21.15.110:0/3476253908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03a40fe430 0x7f03a41977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 >> 172.21.15.110:0/3476253908 conn(0x7f03a40fa1c0 msgr2=0x7f03a4106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 shutdown_connections 2024-09-21T04:14:42.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:42.996+0000 7f03abf0b640 1 -- 172.21.15.110:0/3476253908 wait complete. 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 -- 172.21.15.110:0/4092041287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0102f70 msgr2=0x7f36d0105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 --2- 172.21.15.110:0/4092041287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0102f70 0x7f36d0105360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009880 tx=0x7f36c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 -- 172.21.15.110:0/4092041287 shutdown_connections 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 --2- 172.21.15.110:0/4092041287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0102f70 0x7f36d0105360 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 --2- 172.21.15.110:0/4092041287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0100640 0x7f36d0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 -- 172.21.15.110:0/4092041287 >> 172.21.15.110:0/4092041287 conn(0x7f36d00fa1c0 msgr2=0x7f36d00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 -- 172.21.15.110:0/4092041287 shutdown_connections 2024-09-21T04:14:43.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.095+0000 7f36d8611640 1 -- 172.21.15.110:0/4092041287 wait complete. 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 Processor -- start 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 -- start start 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102f70 0x7f36d0195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0196140 con 0x7f36d0100640 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d8611640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d01962b0 con 0x7f36d0102f70 2024-09-21T04:14:43.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d63ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d63ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33372/0 (socket says 172.21.15.110:33372) 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d63ad640 1 -- 172.21.15.110:0/3838315404 learned_addr learned my addr 172.21.15.110:0/3838315404 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.096+0000 7f36d5bac640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102f70 0x7f36d0195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d63ad640 1 -- 172.21.15.110:0/3838315404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102f70 msgr2=0x7f36d0195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d63ad640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102f70 0x7f36d0195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d63ad640 1 -- 172.21.15.110:0/3838315404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009560 con 0x7f36d0100640 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d63ad640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00d900 tx=0x7f36cc00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36cc009c00 con 0x7f36d0100640 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36cc00b660 con 0x7f36d0100640 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d019ad00 con 0x7f36d0100640 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36cc00f680 con 0x7f36d0100640 2024-09-21T04:14:43.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.097+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d019b270 con 0x7f36d0100640 2024-09-21T04:14:43.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.098+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d0108710 con 0x7f36d0100640 2024-09-21T04:14:43.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.099+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f36cc010ab0 con 0x7f36d0100640 2024-09-21T04:14:43.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.099+0000 7f36c77fe640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 0x7f36bc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.100+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f36cc08bfb0 con 0x7f36d0100640 2024-09-21T04:14:43.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.100+0000 7f36d5bac640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 0x7f36bc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.100+0000 7f36d5bac640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 0x7f36bc06f110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.107+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f36cc0563a0 con 0x7f36d0100640 2024-09-21T04:14:43.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.377+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36d019b4e0 con 0x7f36d0100640 2024-09-21T04:14:43.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.378+0000 7f36c77fe640 1 -- 172.21.15.110:0/3838315404 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f36cc0599c0 con 0x7f36d0100640 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:14:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:14:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:14:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:14:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 14 2024-09-21T04:14:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:14:43.382 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 msgr2=0x7f36bc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 0x7f36bc06f110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0000bc0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 msgr2=0x7f36d0195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00d900 tx=0x7f36cc00ddc0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 shutdown_connections 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36d0102f70 0x7f36d0195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f36bc06cc50 0x7f36bc06f110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 --2- 172.21.15.110:0/3838315404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36d0100640 0x7f36d0195630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 >> 172.21.15.110:0/3838315404 conn(0x7f36d00fa1c0 msgr2=0x7f36d00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 shutdown_connections 2024-09-21T04:14:43.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.380+0000 7f36d8611640 1 -- 172.21.15.110:0/3838315404 wait complete. 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: pgmap v256: 65 pgs: 65 active+clean; 95 MiB data, 532 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s wr, 457 op/s 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:14:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:43 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 -- 172.21.15.110:0/612562473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430100630 msgr2=0x7f2430102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 --2- 172.21.15.110:0/612562473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430100630 0x7f2430102a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f242c0098b0 tx=0x7f242c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 -- 172.21.15.110:0/612562473 shutdown_connections 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 --2- 172.21.15.110:0/612562473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430102f60 0x7f2430105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 --2- 172.21.15.110:0/612562473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430100630 0x7f2430102a20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 -- 172.21.15.110:0/612562473 >> 172.21.15.110:0/612562473 conn(0x7f24300fa230 msgr2=0x7f24300fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 -- 172.21.15.110:0/612562473 shutdown_connections 2024-09-21T04:14:43.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.483+0000 7f24380b0640 1 -- 172.21.15.110:0/612562473 wait complete. 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 Processor -- start 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 -- start start 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430100630 0x7f2430195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430196150 con 0x7f2430102f60 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f24380b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24301962c0 con 0x7f2430100630 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f243564b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f243564b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33382/0 (socket says 172.21.15.110:33382) 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f243564b640 1 -- 172.21.15.110:0/2619281781 learned_addr learned my addr 172.21.15.110:0/2619281781 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f2435e4c640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430100630 0x7f2430195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f243564b640 1 -- 172.21.15.110:0/2619281781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430100630 msgr2=0x7f2430195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.484+0000 7f243564b640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430100630 0x7f2430195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f243564b640 1 -- 172.21.15.110:0/2619281781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c009560 con 0x7f2430102f60 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f243564b640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f242000d720 tx=0x7f242000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24200099f0 con 0x7f2430102f60 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243019ad60 con 0x7f2430102f60 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243019b330 con 0x7f2430102f60 2024-09-21T04:14:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.485+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2420009b50 con 0x7f2430102f60 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.486+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242000f5b0 con 0x7f2430102f60 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.487+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f2420009cc0 con 0x7f2430102f60 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.487+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24301087c0 con 0x7f2430102f60 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.487+0000 7f2426ffd640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 0x7f241c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.487+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f242008c370 con 0x7f2430102f60 2024-09-21T04:14:43.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.487+0000 7f2435e4c640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 0x7f241c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.488+0000 7f2435e4c640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 0x7f241c06f110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f242c006010 tx=0x7f242c0314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.492+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f24200566c0 con 0x7f2430102f60 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: pgmap v256: 65 pgs: 65 active+clean; 95 MiB data, 532 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s wr, 457 op/s 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:14:43.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:43 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:14:43.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.774+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f243019b570 con 0x7f2430102f60 2024-09-21T04:14:43.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.775+0000 7f2426ffd640 1 -- 172.21.15.110:0/2619281781 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f2420059ce0 con 0x7f2430102f60 2024-09-21T04:14:43.778 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:14:43.778 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:14:43.778 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:14:43.778 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:14:43.779 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:14:43.780 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:43.781 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 msgr2=0x7f241c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 0x7f241c06f110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f242c006010 tx=0x7f242c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 msgr2=0x7f2430195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f242000d720 tx=0x7f242000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 shutdown_connections 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f241c06cc50 0x7f241c06f110 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2430102f60 0x7f2430195b80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 --2- 172.21.15.110:0/2619281781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2430100630 0x7f2430195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 >> 172.21.15.110:0/2619281781 conn(0x7f24300fa230 msgr2=0x7f24300fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 shutdown_connections 2024-09-21T04:14:43.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.778+0000 7f24380b0640 1 -- 172.21.15.110:0/2619281781 wait complete. 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 -- 172.21.15.110:0/409228247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e8101280 msgr2=0x7f63e8101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 --2- 172.21.15.110:0/409228247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e8101280 0x7f63e8101680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009880 tx=0x7f63d802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 -- 172.21.15.110:0/409228247 shutdown_connections 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 --2- 172.21.15.110:0/409228247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e80ff7c0 0x7f63e80ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 --2- 172.21.15.110:0/409228247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e8101280 0x7f63e8101680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 -- 172.21.15.110:0/409228247 >> 172.21.15.110:0/409228247 conn(0x7f63e80fb330 msgr2=0x7f63e80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 -- 172.21.15.110:0/409228247 shutdown_connections 2024-09-21T04:14:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.877+0000 7f63ee230640 1 -- 172.21.15.110:0/409228247 wait complete. 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 Processor -- start 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 -- start start 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 0x7f63e80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8072de0 con 0x7f63e80ff7c0 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.878+0000 7f63ee230640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8072f50 con 0x7f63e8101280 2024-09-21T04:14:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 0x7f63e80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33396/0 (socket says 172.21.15.110:33396) 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e77fe640 1 -- 172.21.15.110:0/4063278952 learned_addr learned my addr 172.21.15.110:0/4063278952 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e6ffd640 1 -- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 msgr2=0x7f63e8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e6ffd640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e6ffd640 1 -- 172.21.15.110:0/4063278952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009560 con 0x7f63e8101280 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e77fe640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e6ffd640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 0x7f63e80728a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00e970 tx=0x7f63dc00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63dc009620 con 0x7f63e8101280 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e8073230 con 0x7f63e8101280 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63dc0044f0 con 0x7f63e8101280 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.879+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63dc0049d0 con 0x7f63e8101280 2024-09-21T04:14:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.880+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e819d770 con 0x7f63e8101280 2024-09-21T04:14:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.881+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e80ffc40 con 0x7f63e8101280 2024-09-21T04:14:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.881+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7f63dc00cc90 con 0x7f63e8101280 2024-09-21T04:14:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.882+0000 7f63e4ff9640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 0x7f63d006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.882+0000 7f63e77fe640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 0x7f63d006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.882+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f63dc014070 con 0x7f63e8101280 2024-09-21T04:14:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.882+0000 7f63e77fe640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 0x7f63d006f110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d80314e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:43.886+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f63dc056950 con 0x7f63e8101280 2024-09-21T04:14:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.168+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 --> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63e80608f0 con 0x7f63d006cc50 2024-09-21T04:14:44.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.171+0000 7f63e4ff9640 1 -- 172.21.15.110:0/4063278952 <== mgr.14213 v2:172.21.15.110:6800/1362067387 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f63e80608f0 con 0x7f63d006cc50 2024-09-21T04:14:44.174 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:14:44.174 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:14:44.174 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:14:44.174 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:14:44.175 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:14:44.175 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "0/23 daemons upgraded", 2024-09-21T04:14:44.175 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mgr daemons", 2024-09-21T04:14:44.175 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:14:44.175 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:14:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 msgr2=0x7f63d006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 0x7f63d006f110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d80314e0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 msgr2=0x7f63e80728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 0x7f63e80728a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00e970 tx=0x7f63dc00ee30 comp rx=0 tx=0).stop 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 shutdown_connections 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63e8101280 0x7f63e80728a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7f63d006cc50 0x7f63d006f110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.176+0000 7f63ee230640 1 --2- 172.21.15.110:0/4063278952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e80ff7c0 0x7f63e8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.177+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 >> 172.21.15.110:0/4063278952 conn(0x7f63e80fb330 msgr2=0x7f63e8104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.177+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 shutdown_connections 2024-09-21T04:14:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.177+0000 7f63ee230640 1 -- 172.21.15.110:0/4063278952 wait complete. 2024-09-21T04:14:44.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.331+0000 7fe8057b5640 1 -- 172.21.15.110:0/3192452587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe800100d00 msgr2=0x7fe800103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.331+0000 7fe8057b5640 1 --2- 172.21.15.110:0/3192452587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe800100d00 0x7fe800103170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00097e0 tx=0x7fe7f002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.333+0000 7fe8057b5640 1 -- 172.21.15.110:0/3192452587 shutdown_connections 2024-09-21T04:14:44.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.333+0000 7fe8057b5640 1 --2- 172.21.15.110:0/3192452587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe800100d00 0x7fe800103170 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.333+0000 7fe8057b5640 1 --2- 172.21.15.110:0/3192452587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8000fe3b0 0x7fe8001007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.333+0000 7fe8057b5640 1 -- 172.21.15.110:0/3192452587 >> 172.21.15.110:0/3192452587 conn(0x7fe8000fa1c0 msgr2=0x7fe8000fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:44.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.335+0000 7fe8057b5640 1 -- 172.21.15.110:0/3192452587 shutdown_connections 2024-09-21T04:14:44.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.335+0000 7fe8057b5640 1 -- 172.21.15.110:0/3192452587 wait complete. 2024-09-21T04:14:44.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.336+0000 7fe8057b5640 1 Processor -- start 2024-09-21T04:14:44.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe8057b5640 1 -- start start 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe8057b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8000fe3b0 0x7fe800071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe8057b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe8057b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000720b0 con 0x7fe8000fe3b0 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe8057b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000721f0 con 0x7fe800100d00 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe7fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe7fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:46578/0 (socket says 172.21.15.110:46578) 2024-09-21T04:14:44.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.337+0000 7fe7fe7fc640 1 -- 172.21.15.110:0/2322056258 learned_addr learned my addr 172.21.15.110:0/2322056258 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:14:44.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.338+0000 7fe7fe7fc640 1 -- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8000fe3b0 msgr2=0x7fe800071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.338+0000 7fe7fe7fc640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8000fe3b0 0x7fe800071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.338+0000 7fe7fe7fc640 1 -- 172.21.15.110:0/2322056258 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0009490 con 0x7fe800100d00 2024-09-21T04:14:44.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.338+0000 7fe7fe7fc640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0030c30 tx=0x7fe7f0030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:44.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.339+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f00039d0 con 0x7fe800100d00 2024-09-21T04:14:44.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.339+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8001a9ac0 con 0x7fe800100d00 2024-09-21T04:14:44.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.339+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8001a9fb0 con 0x7fe800100d00 2024-09-21T04:14:44.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.339+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7f0030ec0 con 0x7fe800100d00 2024-09-21T04:14:44.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.339+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f0045460 con 0x7fe800100d00 2024-09-21T04:14:44.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.341+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec005280 con 0x7fe800100d00 2024-09-21T04:14:44.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.343+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 90852+0+0 (secure 0 0 0) 0x7fe7f0031400 con 0x7fe800100d00 2024-09-21T04:14:44.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.343+0000 7fe7e7fff640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 0x7fe7e806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:44.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.344+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe7f00ade90 con 0x7fe800100d00 2024-09-21T04:14:44.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.344+0000 7fe7feffd640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 0x7fe7e806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:44.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.348+0000 7fe7feffd640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 0x7fe7e806f280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4005fd0 tx=0x7fe7f4005a00 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:44.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.350+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe7f0078260 con 0x7fe800100d00 2024-09-21T04:14:44.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:44.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: Upgrade: Updating mgr.smithi148.zjrahi 2024-09-21T04:14:44.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:14:44.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:44.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3838315404' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:14:44.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:44 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2619281781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: Upgrade: Updating mgr.smithi148.zjrahi 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3838315404' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:14:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:44 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2619281781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:14:44.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.833+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7ec0050f0 con 0x7fe800100d00 2024-09-21T04:14:44.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.835+0000 7fe7e7fff640 1 -- 172.21.15.110:0/2322056258 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe7f007b880 con 0x7fe800100d00 2024-09-21T04:14:44.838 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:14:44.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.841+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 msgr2=0x7fe7e806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.841+0000 7fe8057b5640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 0x7fe7e806f280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4005fd0 tx=0x7fe7f4005a00 comp rx=0 tx=0).stop 2024-09-21T04:14:44.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.841+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 msgr2=0x7fe800071a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:44.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.841+0000 7fe8057b5640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0030c30 tx=0x7fe7f0030c60 comp rx=0 tx=0).stop 2024-09-21T04:14:44.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.843+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 shutdown_connections 2024-09-21T04:14:44.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.843+0000 7fe8057b5640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe800100d00 0x7fe800071a90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.843+0000 7fe8057b5640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:6800/1362067387,v1:172.21.15.110:6801/1362067387] conn(0x7fe7e806cdc0 0x7fe7e806f280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.843+0000 7fe8057b5640 1 --2- 172.21.15.110:0/2322056258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8000fe3b0 0x7fe800071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:44.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.843+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 >> 172.21.15.110:0/2322056258 conn(0x7fe8000fa1c0 msgr2=0x7fe8000fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:44.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.844+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 shutdown_connections 2024-09-21T04:14:44.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:14:44.844+0000 7fe8057b5640 1 -- 172.21.15.110:0/2322056258 wait complete. 2024-09-21T04:14:45.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:45 smithi110 ceph-mon[27723]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:45 smithi110 ceph-mon[27723]: pgmap v257: 65 pgs: 65 active+clean; 99 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 5.9 MiB/s wr, 258 op/s 2024-09-21T04:14:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:45 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2322056258' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:14:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:45 smithi148 ceph-mon[34957]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:45 smithi148 ceph-mon[34957]: pgmap v257: 65 pgs: 65 active+clean; 99 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 5.9 MiB/s wr, 258 op/s 2024-09-21T04:14:45.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:45 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2322056258' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:14:47.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:47 smithi110 ceph-mon[27723]: pgmap v258: 65 pgs: 65 active+clean; 112 MiB data, 571 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 319 op/s 2024-09-21T04:14:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:47 smithi148 ceph-mon[34957]: pgmap v258: 65 pgs: 65 active+clean; 112 MiB data, 571 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 319 op/s 2024-09-21T04:14:48.410 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:48 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:48.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:48 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:14:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:49 smithi110 ceph-mon[27723]: pgmap v259: 65 pgs: 65 active+clean; 140 MiB data, 698 MiB used, 536 GiB / 536 GiB avail; 7.4 MiB/s wr, 734 op/s 2024-09-21T04:14:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:49 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:14:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:49 smithi148 ceph-mon[34957]: pgmap v259: 65 pgs: 65 active+clean; 140 MiB data, 698 MiB used, 536 GiB / 536 GiB avail; 7.4 MiB/s wr, 734 op/s 2024-09-21T04:14:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:49 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:14:51.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:51 smithi110 ceph-mon[27723]: pgmap v260: 65 pgs: 65 active+clean; 149 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 772 op/s 2024-09-21T04:14:51.673 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:51 smithi148 ceph-mon[34957]: pgmap v260: 65 pgs: 65 active+clean; 149 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 772 op/s 2024-09-21T04:14:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:53 smithi110 ceph-mon[27723]: pgmap v261: 65 pgs: 65 active+clean; 167 MiB data, 790 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s wr, 796 op/s 2024-09-21T04:14:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:53 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:53 smithi148 ceph-mon[34957]: pgmap v261: 65 pgs: 65 active+clean; 167 MiB data, 790 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s wr, 796 op/s 2024-09-21T04:14:53.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:53.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:53 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:14:54.652 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-21T04:14:54.653 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:14:54.653 INFO:tasks.workunit.client.1.smithi148.stderr:+ make 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: pgmap v262: 65 pgs: 65 active+clean; 171 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s wr, 795 op/s 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:14:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:55 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:14:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: pgmap v262: 65 pgs: 65 active+clean; 171 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s wr, 795 op/s 2024-09-21T04:14:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:14:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:14:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:14:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:14:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:14:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:55 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:14:55.765 INFO:tasks.workunit.client.1.smithi148.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-21T04:14:56.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:56 smithi110 ceph-mon[27723]: mgrmap e20: smithi110.axmqzb(active, since 7m), standbys: smithi148.zjrahi 2024-09-21T04:14:56.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:56 smithi148 ceph-mon[34957]: mgrmap e20: smithi110.axmqzb(active, since 7m), standbys: smithi148.zjrahi 2024-09-21T04:14:57.561 INFO:tasks.workunit.client.1.smithi148.stderr:++ readlink -f fsstress 2024-09-21T04:14:57.562 INFO:tasks.workunit.client.1.smithi148.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-21T04:14:57.562 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-09-21T04:14:57.562 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:14:57.563 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-09-21T04:14:57.563 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:14:57.563 INFO:tasks.workunit.client.1.smithi148.stderr:++ mktemp -d -p . 2024-09-21T04:14:57.564 INFO:tasks.workunit.client.1.smithi148.stderr:+ T=./tmp.9ZUkyyQeE2 2024-09-21T04:14:57.564 INFO:tasks.workunit.client.1.smithi148.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.9ZUkyyQeE2 -l 1 -n 1000 -p 10 -v 2024-09-21T04:14:57.566 INFO:tasks.workunit.client.1.smithi148.stdout:seed = 1726404162 2024-09-21T04:14:57.570 INFO:tasks.workunit.client.1.smithi148.stdout:0/0: creat f0 x:0 0 0 2024-09-21T04:14:57.570 INFO:tasks.workunit.client.1.smithi148.stdout:0/1: dread - f0 zero size 2024-09-21T04:14:57.571 INFO:tasks.workunit.client.1.smithi148.stdout:2/0: getdents . 0 2024-09-21T04:14:57.571 INFO:tasks.workunit.client.1.smithi148.stdout:2/1: fdatasync - no filename 2024-09-21T04:14:57.571 INFO:tasks.workunit.client.1.smithi148.stdout:1/0: creat f0 x:0 0 0 2024-09-21T04:14:57.572 INFO:tasks.workunit.client.1.smithi148.stdout:0/2: unlink f0 0 2024-09-21T04:14:57.572 INFO:tasks.workunit.client.1.smithi148.stdout:0/3: write - no filename 2024-09-21T04:14:57.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/4: rename - no filename 2024-09-21T04:14:57.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/5: dwrite - no filename 2024-09-21T04:14:57.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/6: rmdir - no directory 2024-09-21T04:14:57.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/0: dwrite - no filename 2024-09-21T04:14:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/2: mkdir d0 0 2024-09-21T04:14:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/3: readlink - no filename 2024-09-21T04:14:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/4: getdents d0 0 2024-09-21T04:14:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/5: link - no file 2024-09-21T04:14:57.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/6: rename d0 to d0/d1 22 2024-09-21T04:14:57.575 INFO:tasks.workunit.client.1.smithi148.stdout:2/7: readlink - no filename 2024-09-21T04:14:57.576 INFO:tasks.workunit.client.1.smithi148.stdout:0/7: symlink l1 0 2024-09-21T04:14:57.577 INFO:tasks.workunit.client.1.smithi148.stdout:4/0: dwrite - no filename 2024-09-21T04:14:57.577 INFO:tasks.workunit.client.1.smithi148.stdout:3/1: symlink l0 0 2024-09-21T04:14:57.577 INFO:tasks.workunit.client.1.smithi148.stdout:3/2: chown l0 4480889 1 2024-09-21T04:14:57.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/8: mkdir d0/d2 0 2024-09-21T04:14:57.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/9: dread - no filename 2024-09-21T04:14:57.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/10: write - no filename 2024-09-21T04:14:57.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/11: dwrite - no filename 2024-09-21T04:14:57.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/12: truncate - no filename 2024-09-21T04:14:57.579 INFO:tasks.workunit.client.1.smithi148.stdout:0/8: mkdir d2 0 2024-09-21T04:14:57.579 INFO:tasks.workunit.client.1.smithi148.stdout:0/9: truncate - no filename 2024-09-21T04:14:57.579 INFO:tasks.workunit.client.1.smithi148.stdout:0/10: chown d2 827489 1 2024-09-21T04:14:57.579 INFO:tasks.workunit.client.1.smithi148.stdout:0/11: rename d2 to d2/d3 22 2024-09-21T04:14:57.580 INFO:tasks.workunit.client.1.smithi148.stdout:4/1: mknod c0 0 2024-09-21T04:14:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:3/3: creat f1 x:0 0 0 2024-09-21T04:14:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:3/4: chown l0 1268 1 2024-09-21T04:14:57.581 INFO:tasks.workunit.client.1.smithi148.stdout:3/5: fdatasync f1 0 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/13: mknod d0/c3 0 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/14: dwrite - no filename 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/15: write - no filename 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/16: fsync - no filename 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/17: chown d0 2916 1 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/18: chown d0/c3 0 1 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/19: write - no filename 2024-09-21T04:14:57.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/20: getdents d0/d2 0 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/21: chown d0 57542676 1 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:2/22: dwrite - no filename 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/12: rmdir d2 0 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/13: dread - no filename 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/14: readlink l1 0 2024-09-21T04:14:57.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/15: readlink l1 0 2024-09-21T04:14:57.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/16: write - no filename 2024-09-21T04:14:57.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/17: rmdir - no directory 2024-09-21T04:14:57.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/18: dread - no filename 2024-09-21T04:14:57.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/19: dread - no filename 2024-09-21T04:14:57.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/0: rename - no filename 2024-09-21T04:14:57.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/1: chown . 17749659 1 2024-09-21T04:14:57.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/2: dwrite - no filename 2024-09-21T04:14:57.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/3: dread - no filename 2024-09-21T04:14:57.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/0: creat f0 x:0 0 0 2024-09-21T04:14:57.587 INFO:tasks.workunit.client.1.smithi148.stdout:4/2: creat f1 x:0 0 0 2024-09-21T04:14:57.587 INFO:tasks.workunit.client.1.smithi148.stdout:4/3: chown c0 0 1 2024-09-21T04:14:57.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/6: symlink l2 0 2024-09-21T04:14:57.589 INFO:tasks.workunit.client.1.smithi148.stdout:2/23: creat d0/f4 x:0 0 0 2024-09-21T04:14:57.591 INFO:tasks.workunit.client.1.smithi148.stdout:0/20: unlink l1 0 2024-09-21T04:14:57.591 INFO:tasks.workunit.client.1.smithi148.stdout:0/21: dread - no filename 2024-09-21T04:14:57.591 INFO:tasks.workunit.client.1.smithi148.stdout:0/22: unlink - no file 2024-09-21T04:14:57.591 INFO:tasks.workunit.client.1.smithi148.stdout:0/23: creat f4 x:0 0 0 2024-09-21T04:14:57.593 INFO:tasks.workunit.client.1.smithi148.stdout:7/0: rename - no filename 2024-09-21T04:14:57.594 INFO:tasks.workunit.client.1.smithi148.stdout:6/4: mkdir d0 0 2024-09-21T04:14:57.595 INFO:tasks.workunit.client.1.smithi148.stdout:6/5: fdatasync - no filename 2024-09-21T04:14:57.595 INFO:tasks.workunit.client.1.smithi148.stdout:6/6: write - no filename 2024-09-21T04:14:57.595 INFO:tasks.workunit.client.1.smithi148.stdout:6/7: dread - no filename 2024-09-21T04:14:57.598 INFO:tasks.workunit.client.1.smithi148.stdout:4/4: symlink l2 0 2024-09-21T04:14:57.599 INFO:tasks.workunit.client.1.smithi148.stdout:4/5: rmdir - no directory 2024-09-21T04:14:57.599 INFO:tasks.workunit.client.1.smithi148.stdout:4/6: creat f3 x:0 0 0 2024-09-21T04:14:57.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/7: mkdir d3 0 2024-09-21T04:14:57.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/8: getdents d3 0 2024-09-21T04:14:57.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/9: write f1 [241043,50783] 0 2024-09-21T04:14:57.601 INFO:tasks.workunit.client.1.smithi148.stdout:3/10: write f1 [146792,118] 0 2024-09-21T04:14:57.603 INFO:tasks.workunit.client.1.smithi148.stdout:0/24: mkdir d5 0 2024-09-21T04:14:57.603 INFO:tasks.workunit.client.1.smithi148.stdout:0/25: truncate f4 998303 0 2024-09-21T04:14:57.604 INFO:tasks.workunit.client.1.smithi148.stdout:8/0: chown . 122892285 1 2024-09-21T04:14:57.604 INFO:tasks.workunit.client.1.smithi148.stdout:8/1: dread - no filename 2024-09-21T04:14:57.604 INFO:tasks.workunit.client.1.smithi148.stdout:8/2: stat - no entries 2024-09-21T04:14:57.605 INFO:tasks.workunit.client.1.smithi148.stdout:8/3: link - no file 2024-09-21T04:14:57.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/1: creat f0 x:0 0 0 2024-09-21T04:14:57.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/2: write f0 [430156,119520] 0 2024-09-21T04:14:57.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/24: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:14:57.608 INFO:tasks.workunit.client.1.smithi148.stdout:6/8: creat d0/f1 x:0 0 0 2024-09-21T04:14:57.608 INFO:tasks.workunit.client.1.smithi148.stdout:6/9: truncate d0/f1 314268 0 2024-09-21T04:14:57.610 INFO:tasks.workunit.client.1.smithi148.stdout:4/7: mkdir d4 0 2024-09-21T04:14:57.616 INFO:tasks.workunit.client.1.smithi148.stdout:8/4: creat f0 x:0 0 0 2024-09-21T04:14:57.616 INFO:tasks.workunit.client.1.smithi148.stdout:8/5: dread - f0 zero size 2024-09-21T04:14:57.617 INFO:tasks.workunit.client.1.smithi148.stdout:7/3: mknod c1 0 2024-09-21T04:14:57.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/26: dwrite f4 [0,4194304] 0 2024-09-21T04:14:57.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/10: mknod d0/c2 0 2024-09-21T04:14:57.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/11: fdatasync d0/f1 0 2024-09-21T04:14:57.623 INFO:tasks.workunit.client.1.smithi148.stdout:4/8: mknod d4/c5 0 2024-09-21T04:14:57.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/0: creat f0 x:0 0 0 2024-09-21T04:14:57.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/1: creat f1 x:0 0 0 2024-09-21T04:14:57.631 INFO:tasks.workunit.client.1.smithi148.stdout:0/27: mkdir d5/d6 0 2024-09-21T04:14:57.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/6: dwrite f0 [0,4194304] 0 2024-09-21T04:14:57.634 INFO:tasks.workunit.client.1.smithi148.stdout:4/9: creat d4/f6 x:0 0 0 2024-09-21T04:14:57.638 INFO:tasks.workunit.client.1.smithi148.stdout:0/28: mkdir d5/d6/d7 0 2024-09-21T04:14:57.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:57 smithi110 ceph-mon[27723]: pgmap v263: 65 pgs: 65 active+clean; 171 MiB data, 818 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 790 op/s 2024-09-21T04:14:57.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/2: dwrite f0 [0,4194304] 0 2024-09-21T04:14:57.680 INFO:tasks.workunit.client.1.smithi148.stdout:9/3: chown f0 18 1 2024-09-21T04:14:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:57 smithi148 ceph-mon[34957]: pgmap v263: 65 pgs: 65 active+clean; 171 MiB data, 818 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 790 op/s 2024-09-21T04:14:57.738 INFO:tasks.workunit.client.1.smithi148.stdout:8/7: dread f0 [0,4194304] 0 2024-09-21T04:14:57.882 INFO:tasks.workunit.client.1.smithi148.stdout:8/8: mknod c1 0 2024-09-21T04:14:58.027 INFO:tasks.workunit.client.1.smithi148.stdout:1/1: fsync f0 0 2024-09-21T04:14:58.027 INFO:tasks.workunit.client.1.smithi148.stdout:1/2: stat f0 0 2024-09-21T04:14:58.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/11: fsync f1 0 2024-09-21T04:14:58.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/12: write f1 [415529,117990] 0 2024-09-21T04:14:58.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/13: getdents d3 0 2024-09-21T04:14:58.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/1: fsync f0 0 2024-09-21T04:14:58.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/2: write f0 [715853,58792] 0 2024-09-21T04:14:58.063 INFO:tasks.workunit.client.1.smithi148.stdout:2/25: rmdir d0 39 2024-09-21T04:14:58.063 INFO:tasks.workunit.client.1.smithi148.stdout:2/26: fdatasync d0/f4 0 2024-09-21T04:14:58.085 INFO:tasks.workunit.client.1.smithi148.stdout:9/4: dwrite f1 [0,4194304] 0 2024-09-21T04:14:58.091 INFO:tasks.workunit.client.1.smithi148.stdout:7/4: getdents . 0 2024-09-21T04:14:58.091 INFO:tasks.workunit.client.1.smithi148.stdout:7/5: truncate f0 1318080 0 2024-09-21T04:14:58.091 INFO:tasks.workunit.client.1.smithi148.stdout:7/6: rmdir - no directory 2024-09-21T04:14:58.093 INFO:tasks.workunit.client.1.smithi148.stdout:6/12: getdents d0 0 2024-09-21T04:14:58.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/13: dread d0/f1 [0,4194304] 0 2024-09-21T04:14:58.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/14: write d0/f1 [421525,62051] 0 2024-09-21T04:14:58.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/15: truncate d0/f1 713214 0 2024-09-21T04:14:58.231 INFO:tasks.workunit.client.1.smithi148.stdout:3/14: dread f1 [0,4194304] 0 2024-09-21T04:14:58.459 INFO:tasks.workunit.client.1.smithi148.stdout:0/29: dwrite f4 [4194304,4194304] 0 2024-09-21T04:14:59.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/9: dwrite f0 [4194304,4194304] 0 2024-09-21T04:14:59.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/10: dread f0 [0,4194304] 0 2024-09-21T04:14:59.025 INFO:tasks.workunit.client.1.smithi148.stdout:9/5: write f0 [4621682,21881] 0 2024-09-21T04:14:59.036 INFO:tasks.workunit.client.1.smithi148.stdout:9/6: write f0 [1205015,8867] 0 2024-09-21T04:14:59.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/7: rmdir - no directory 2024-09-21T04:14:59.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/8: creat f2 x:0 0 0 2024-09-21T04:14:59.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/9: dwrite f1 [0,4194304] 0 2024-09-21T04:14:59.056 INFO:tasks.workunit.client.1.smithi148.stdout:9/10: truncate f2 76104 0 2024-09-21T04:14:59.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/3: rename f0 to f1 0 2024-09-21T04:14:59.247 INFO:tasks.workunit.client.1.smithi148.stdout:5/3: symlink l1 0 2024-09-21T04:14:59.247 INFO:tasks.workunit.client.1.smithi148.stdout:5/4: creat f2 x:0 0 0 2024-09-21T04:14:59.248 INFO:tasks.workunit.client.1.smithi148.stdout:2/27: mkdir d0/d5 0 2024-09-21T04:14:59.258 INFO:tasks.workunit.client.1.smithi148.stdout:2/28: dread d0/f4 [0,4194304] 0 2024-09-21T04:14:59.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/7: creat f2 x:0 0 0 2024-09-21T04:14:59.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/8: write f2 [819179,17233] 0 2024-09-21T04:14:59.278 INFO:tasks.workunit.client.1.smithi148.stdout:2/29: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:14:59.284 INFO:tasks.workunit.client.1.smithi148.stdout:7/9: dwrite f2 [0,4194304] 0 2024-09-21T04:14:59.329 INFO:tasks.workunit.client.1.smithi148.stdout:3/15: creat d3/f4 x:0 0 0 2024-09-21T04:14:59.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/30: creat d5/f8 x:0 0 0 2024-09-21T04:14:59.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/31: rename d5 to d5/d6/d9 22 2024-09-21T04:14:59.346 INFO:tasks.workunit.client.1.smithi148.stdout:3/16: dwrite f1 [0,4194304] 0 2024-09-21T04:14:59.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/5: mkdir d3 0 2024-09-21T04:14:59.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/10: creat f3 x:0 0 0 2024-09-21T04:14:59.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/11: chown f3 105 1 2024-09-21T04:14:59.391 INFO:tasks.workunit.client.1.smithi148.stdout:0/32: mknod d5/d6/d7/ca 0 2024-09-21T04:14:59.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/11: getdents . 0 2024-09-21T04:14:59.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/11: link f2 f3 0 2024-09-21T04:14:59.399 INFO:tasks.workunit.client.1.smithi148.stdout:5/6: creat d3/f4 x:0 0 0 2024-09-21T04:14:59.399 INFO:tasks.workunit.client.1.smithi148.stdout:8/12: dread f0 [0,4194304] 0 2024-09-21T04:14:59.400 INFO:tasks.workunit.client.1.smithi148.stdout:8/13: stat c1 0 2024-09-21T04:14:59.402 INFO:tasks.workunit.client.1.smithi148.stdout:7/12: write f0 [1505980,108265] 0 2024-09-21T04:14:59.402 INFO:tasks.workunit.client.1.smithi148.stdout:7/13: read - f3 zero size 2024-09-21T04:14:59.403 INFO:tasks.workunit.client.1.smithi148.stdout:9/12: mknod c4 0 2024-09-21T04:14:59.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/7: link d3/f4 d3/f5 0 2024-09-21T04:14:59.407 INFO:tasks.workunit.client.1.smithi148.stdout:8/14: creat f2 x:0 0 0 2024-09-21T04:14:59.407 INFO:tasks.workunit.client.1.smithi148.stdout:8/15: fdatasync f0 0 2024-09-21T04:14:59.407 INFO:tasks.workunit.client.1.smithi148.stdout:0/33: dwrite d5/f8 [0,4194304] 0 2024-09-21T04:14:59.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/30: rmdir d0/d2 0 2024-09-21T04:14:59.411 INFO:tasks.workunit.client.1.smithi148.stdout:9/13: mkdir d5 0 2024-09-21T04:14:59.411 INFO:tasks.workunit.client.1.smithi148.stdout:9/14: getdents d5 0 2024-09-21T04:14:59.417 INFO:tasks.workunit.client.1.smithi148.stdout:0/34: dread d5/f8 [0,4194304] 0 2024-09-21T04:14:59.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/31: write d0/f4 [3382179,108056] 0 2024-09-21T04:14:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/32: fdatasync d0/f4 0 2024-09-21T04:14:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/33: write d0/f4 [1815358,9675] 0 2024-09-21T04:14:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/34: chown d0/f4 937061054 1 2024-09-21T04:14:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/35: mknod d5/d6/d7/cb 0 2024-09-21T04:14:59.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/36: chown d5/d6/d7/ca 0 1 2024-09-21T04:14:59.424 INFO:tasks.workunit.client.1.smithi148.stdout:2/35: rmdir d0/d5 0 2024-09-21T04:14:59.424 INFO:tasks.workunit.client.1.smithi148.stdout:2/36: chown d0/f4 4434 1 2024-09-21T04:14:59.426 INFO:tasks.workunit.client.1.smithi148.stdout:2/37: creat d0/f6 x:0 0 0 2024-09-21T04:14:59.426 INFO:tasks.workunit.client.1.smithi148.stdout:2/38: write d0/f6 [863599,93046] 0 2024-09-21T04:14:59.429 INFO:tasks.workunit.client.1.smithi148.stdout:9/15: dwrite f1 [0,4194304] 0 2024-09-21T04:14:59.429 INFO:tasks.workunit.client.1.smithi148.stdout:2/39: rename d0/c3 to d0/c7 0 2024-09-21T04:14:59.429 INFO:tasks.workunit.client.1.smithi148.stdout:2/40: stat d0/f6 0 2024-09-21T04:14:59.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/37: dread f4 [4194304,4194304] 0 2024-09-21T04:14:59.432 INFO:tasks.workunit.client.1.smithi148.stdout:9/16: rename c4 to d5/c6 0 2024-09-21T04:14:59.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/38: link d5/f8 d5/fc 0 2024-09-21T04:14:59.435 INFO:tasks.workunit.client.1.smithi148.stdout:0/39: readlink - no filename 2024-09-21T04:14:59.438 INFO:tasks.workunit.client.1.smithi148.stdout:0/40: dread d5/fc [0,4194304] 0 2024-09-21T04:14:59.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/17: symlink d5/l7 0 2024-09-21T04:14:59.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/18: write f3 [800474,93912] 0 2024-09-21T04:14:59.442 INFO:tasks.workunit.client.1.smithi148.stdout:9/19: creat d5/f8 x:0 0 0 2024-09-21T04:14:59.443 INFO:tasks.workunit.client.1.smithi148.stdout:9/20: write d5/f8 [582405,86735] 0 2024-09-21T04:14:59.443 INFO:tasks.workunit.client.1.smithi148.stdout:9/21: fdatasync f1 0 2024-09-21T04:14:59.443 INFO:tasks.workunit.client.1.smithi148.stdout:9/22: chown f0 41042 1 2024-09-21T04:14:59.443 INFO:tasks.workunit.client.1.smithi148.stdout:9/23: creat d5/f9 x:0 0 0 2024-09-21T04:14:59.455 INFO:tasks.workunit.client.1.smithi148.stdout:0/41: dwrite f4 [0,4194304] 0 2024-09-21T04:14:59.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:14:59 smithi110 ceph-mon[27723]: pgmap v264: 65 pgs: 65 active+clean; 184 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 634 op/s 2024-09-21T04:14:59.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:14:59 smithi148 ceph-mon[34957]: pgmap v264: 65 pgs: 65 active+clean; 184 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 634 op/s 2024-09-21T04:15:00.193 INFO:tasks.workunit.client.1.smithi148.stdout:1/4: unlink f1 0 2024-09-21T04:15:00.194 INFO:tasks.workunit.client.1.smithi148.stdout:1/5: dread - no filename 2024-09-21T04:15:00.194 INFO:tasks.workunit.client.1.smithi148.stdout:1/6: stat - no entries 2024-09-21T04:15:00.194 INFO:tasks.workunit.client.1.smithi148.stdout:1/7: stat - no entries 2024-09-21T04:15:00.195 INFO:tasks.workunit.client.1.smithi148.stdout:1/8: mknod c2 0 2024-09-21T04:15:00.197 INFO:tasks.workunit.client.1.smithi148.stdout:1/9: symlink l3 0 2024-09-21T04:15:00.197 INFO:tasks.workunit.client.1.smithi148.stdout:1/10: creat f4 x:0 0 0 2024-09-21T04:15:00.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/16: truncate d0/f1 685149 0 2024-09-21T04:15:00.344 INFO:tasks.workunit.client.1.smithi148.stdout:6/17: rmdir d0 39 2024-09-21T04:15:00.345 INFO:tasks.workunit.client.1.smithi148.stdout:6/18: chown d0/c2 3 1 2024-09-21T04:15:00.361 INFO:tasks.workunit.client.1.smithi148.stdout:7/14: truncate f2 1820547 0 2024-09-21T04:15:00.366 INFO:tasks.workunit.client.1.smithi148.stdout:7/15: rename f3 to f4 0 2024-09-21T04:15:00.366 INFO:tasks.workunit.client.1.smithi148.stdout:7/16: dread - f4 zero size 2024-09-21T04:15:00.366 INFO:tasks.workunit.client.1.smithi148.stdout:7/17: dread - f4 zero size 2024-09-21T04:15:00.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/41: fsync d0/f6 0 2024-09-21T04:15:00.376 INFO:tasks.workunit.client.1.smithi148.stdout:0/42: dwrite d5/fc [4194304,4194304] 0 2024-09-21T04:15:00.385 INFO:tasks.workunit.client.1.smithi148.stdout:2/42: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:00.386 INFO:tasks.workunit.client.1.smithi148.stdout:9/24: rmdir d5 39 2024-09-21T04:15:00.395 INFO:tasks.workunit.client.1.smithi148.stdout:9/25: dread f2 [0,4194304] 0 2024-09-21T04:15:00.399 INFO:tasks.workunit.client.1.smithi148.stdout:9/26: mknod d5/ca 0 2024-09-21T04:15:00.399 INFO:tasks.workunit.client.1.smithi148.stdout:9/27: stat f1 0 2024-09-21T04:15:00.401 INFO:tasks.workunit.client.1.smithi148.stdout:0/43: dwrite f4 [8388608,4194304] 0 2024-09-21T04:15:00.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/43: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:15:00.409 INFO:tasks.workunit.client.1.smithi148.stdout:0/44: dread d5/fc [4194304,4194304] 0 2024-09-21T04:15:00.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/45: rename d5/d6/d7/cb to d5/d6/d7/cd 0 2024-09-21T04:15:00.416 INFO:tasks.workunit.client.1.smithi148.stdout:0/46: dread d5/f8 [0,4194304] 0 2024-09-21T04:15:00.418 INFO:tasks.workunit.client.1.smithi148.stdout:9/28: dwrite f3 [0,4194304] 0 2024-09-21T04:15:00.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/44: dread d0/f4 [0,4194304] 0 2024-09-21T04:15:00.419 INFO:tasks.workunit.client.1.smithi148.stdout:0/47: getdents d5/d6 0 2024-09-21T04:15:00.419 INFO:tasks.workunit.client.1.smithi148.stdout:0/48: creat d5/fe x:0 0 0 2024-09-21T04:15:00.422 INFO:tasks.workunit.client.1.smithi148.stdout:0/49: dread d5/f8 [0,4194304] 0 2024-09-21T04:15:00.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/50: chown d5/fc 82533 1 2024-09-21T04:15:00.423 INFO:tasks.workunit.client.1.smithi148.stdout:2/45: symlink d0/l8 0 2024-09-21T04:15:00.428 INFO:tasks.workunit.client.1.smithi148.stdout:2/46: mkdir d0/d9 0 2024-09-21T04:15:00.431 INFO:tasks.workunit.client.1.smithi148.stdout:2/47: symlink d0/la 0 2024-09-21T04:15:00.433 INFO:tasks.workunit.client.1.smithi148.stdout:9/29: dwrite f3 [0,4194304] 0 2024-09-21T04:15:00.436 INFO:tasks.workunit.client.1.smithi148.stdout:9/30: symlink d5/lb 0 2024-09-21T04:15:00.439 INFO:tasks.workunit.client.1.smithi148.stdout:9/31: rename f2 to d5/fc 0 2024-09-21T04:15:00.439 INFO:tasks.workunit.client.1.smithi148.stdout:9/32: chown d5/ca 576708793 1 2024-09-21T04:15:00.448 INFO:tasks.workunit.client.1.smithi148.stdout:0/51: dwrite d5/fe [0,4194304] 0 2024-09-21T04:15:00.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/48: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:00.451 INFO:tasks.workunit.client.1.smithi148.stdout:0/52: rmdir d5/d6 39 2024-09-21T04:15:00.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/49: link d0/f6 d0/d9/fb 0 2024-09-21T04:15:00.454 INFO:tasks.workunit.client.1.smithi148.stdout:0/53: creat d5/d6/d7/ff x:0 0 0 2024-09-21T04:15:00.456 INFO:tasks.workunit.client.1.smithi148.stdout:2/50: mkdir d0/d9/dc 0 2024-09-21T04:15:00.456 INFO:tasks.workunit.client.1.smithi148.stdout:2/51: readlink d0/la 0 2024-09-21T04:15:00.457 INFO:tasks.workunit.client.1.smithi148.stdout:0/54: read d5/f8 [5546285,116537] 0 2024-09-21T04:15:00.469 INFO:tasks.workunit.client.1.smithi148.stdout:2/52: dwrite d0/f4 [4194304,4194304] 0 2024-09-21T04:15:00.474 INFO:tasks.workunit.client.1.smithi148.stdout:2/53: read d0/f4 [3161521,37881] 0 2024-09-21T04:15:00.475 INFO:tasks.workunit.client.1.smithi148.stdout:2/54: chown d0/la 242063 1 2024-09-21T04:15:00.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/55: write d0/d9/fb [3364450,88394] 0 2024-09-21T04:15:00.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/56: write d0/f4 [2286361,42440] 0 2024-09-21T04:15:00.486 INFO:tasks.workunit.client.1.smithi148.stdout:2/57: write d0/f4 [5603275,12806] 0 2024-09-21T04:15:00.488 INFO:tasks.workunit.client.1.smithi148.stdout:2/58: creat d0/d9/dc/fd x:0 0 0 2024-09-21T04:15:00.491 INFO:tasks.workunit.client.1.smithi148.stdout:2/59: creat d0/d9/fe x:0 0 0 2024-09-21T04:15:00.491 INFO:tasks.workunit.client.1.smithi148.stdout:2/60: read d0/f6 [627193,60235] 0 2024-09-21T04:15:00.493 INFO:tasks.workunit.client.1.smithi148.stdout:2/61: rename d0/d9/fb to d0/ff 0 2024-09-21T04:15:00.764 INFO:tasks.workunit.client.1.smithi148.stdout:1/11: getdents . 0 2024-09-21T04:15:00.764 INFO:tasks.workunit.client.1.smithi148.stdout:1/12: chown l3 11642026 1 2024-09-21T04:15:00.764 INFO:tasks.workunit.client.1.smithi148.stdout:1/13: chown c2 471848364 1 2024-09-21T04:15:00.765 INFO:tasks.workunit.client.1.smithi148.stdout:1/14: mknod c5 0 2024-09-21T04:15:00.779 INFO:tasks.workunit.client.1.smithi148.stdout:1/15: dwrite f4 [0,4194304] 0 2024-09-21T04:15:00.828 INFO:tasks.workunit.client.1.smithi148.stdout:7/18: write f2 [2378094,24179] 0 2024-09-21T04:15:00.828 INFO:tasks.workunit.client.1.smithi148.stdout:7/19: dread - f4 zero size 2024-09-21T04:15:00.831 INFO:tasks.workunit.client.1.smithi148.stdout:7/20: getdents . 0 2024-09-21T04:15:00.831 INFO:tasks.workunit.client.1.smithi148.stdout:7/21: chown f4 4159 1 2024-09-21T04:15:00.837 INFO:tasks.workunit.client.1.smithi148.stdout:6/19: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:15:00.839 INFO:tasks.workunit.client.1.smithi148.stdout:6/20: mknod d0/c3 0 2024-09-21T04:15:00.850 INFO:tasks.workunit.client.1.smithi148.stdout:9/33: getdents d5 0 2024-09-21T04:15:00.850 INFO:tasks.workunit.client.1.smithi148.stdout:6/21: read d0/f1 [2347132,9367] 0 2024-09-21T04:15:00.852 INFO:tasks.workunit.client.1.smithi148.stdout:6/22: rename d0/c3 to d0/c4 0 2024-09-21T04:15:00.852 INFO:tasks.workunit.client.1.smithi148.stdout:6/23: chown d0/f1 11797 1 2024-09-21T04:15:00.852 INFO:tasks.workunit.client.1.smithi148.stdout:6/24: fdatasync d0/f1 0 2024-09-21T04:15:00.852 INFO:tasks.workunit.client.1.smithi148.stdout:6/25: stat d0 0 2024-09-21T04:15:00.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/26: mkdir d0/d5 0 2024-09-21T04:15:00.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/27: stat d0/c2 0 2024-09-21T04:15:00.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/22: dwrite f2 [0,4194304] 0 2024-09-21T04:15:00.856 INFO:tasks.workunit.client.1.smithi148.stdout:7/23: mkdir d5 0 2024-09-21T04:15:00.858 INFO:tasks.workunit.client.1.smithi148.stdout:0/55: truncate d5/fe 613384 0 2024-09-21T04:15:00.858 INFO:tasks.workunit.client.1.smithi148.stdout:0/56: chown d5/d6/d7/ca 29921 1 2024-09-21T04:15:00.864 INFO:tasks.workunit.client.1.smithi148.stdout:7/24: write f2 [1862870,127836] 0 2024-09-21T04:15:00.864 INFO:tasks.workunit.client.1.smithi148.stdout:7/25: dread - f4 zero size 2024-09-21T04:15:00.865 INFO:tasks.workunit.client.1.smithi148.stdout:7/26: creat d5/f6 x:0 0 0 2024-09-21T04:15:00.866 INFO:tasks.workunit.client.1.smithi148.stdout:7/27: creat d5/f7 x:0 0 0 2024-09-21T04:15:00.872 INFO:tasks.workunit.client.1.smithi148.stdout:2/62: getdents d0/d9/dc 0 2024-09-21T04:15:00.874 INFO:tasks.workunit.client.1.smithi148.stdout:2/63: unlink d0/c7 0 2024-09-21T04:15:00.874 INFO:tasks.workunit.client.1.smithi148.stdout:2/64: readlink d0/l8 0 2024-09-21T04:15:00.875 INFO:tasks.workunit.client.1.smithi148.stdout:2/65: link d0/la d0/d9/l10 0 2024-09-21T04:15:00.876 INFO:tasks.workunit.client.1.smithi148.stdout:2/66: fsync d0/f6 0 2024-09-21T04:15:00.876 INFO:tasks.workunit.client.1.smithi148.stdout:2/67: fdatasync d0/d9/dc/fd 0 2024-09-21T04:15:00.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/68: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:00.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/69: creat d0/f11 x:0 0 0 2024-09-21T04:15:00.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/16: write f4 [4209631,44500] 0 2024-09-21T04:15:00.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/70: rmdir d0 39 2024-09-21T04:15:00.944 INFO:tasks.workunit.client.1.smithi148.stdout:1/17: dwrite f4 [0,4194304] 0 2024-09-21T04:15:00.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/71: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:00.961 INFO:tasks.workunit.client.1.smithi148.stdout:2/72: dread - d0/d9/dc/fd zero size 2024-09-21T04:15:00.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/18: dwrite f4 [0,4194304] 0 2024-09-21T04:15:00.968 INFO:tasks.workunit.client.1.smithi148.stdout:1/19: creat f6 x:0 0 0 2024-09-21T04:15:00.970 INFO:tasks.workunit.client.1.smithi148.stdout:2/73: dread d0/f6 [0,4194304] 0 2024-09-21T04:15:00.977 INFO:tasks.workunit.client.1.smithi148.stdout:1/20: write f4 [19505,114050] 0 2024-09-21T04:15:00.980 INFO:tasks.workunit.client.1.smithi148.stdout:1/21: mknod c7 0 2024-09-21T04:15:00.982 INFO:tasks.workunit.client.1.smithi148.stdout:1/22: mknod c8 0 2024-09-21T04:15:00.989 INFO:tasks.workunit.client.1.smithi148.stdout:1/23: dread f4 [0,4194304] 0 2024-09-21T04:15:00.991 INFO:tasks.workunit.client.1.smithi148.stdout:1/24: rename l3 to l9 0 2024-09-21T04:15:00.991 INFO:tasks.workunit.client.1.smithi148.stdout:1/25: stat c8 0 2024-09-21T04:15:00.991 INFO:tasks.workunit.client.1.smithi148.stdout:1/26: readlink l9 0 2024-09-21T04:15:00.992 INFO:tasks.workunit.client.1.smithi148.stdout:1/27: unlink c7 0 2024-09-21T04:15:00.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/28: dread f4 [0,4194304] 0 2024-09-21T04:15:00.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/29: fdatasync f6 0 2024-09-21T04:15:00.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/30: symlink la 0 2024-09-21T04:15:01.014 INFO:tasks.workunit.client.1.smithi148.stdout:1/31: dwrite f6 [0,4194304] 0 2024-09-21T04:15:01.018 INFO:tasks.workunit.client.1.smithi148.stdout:1/32: dread f4 [0,4194304] 0 2024-09-21T04:15:01.019 INFO:tasks.workunit.client.1.smithi148.stdout:1/33: mkdir db 0 2024-09-21T04:15:01.021 INFO:tasks.workunit.client.1.smithi148.stdout:1/34: symlink db/lc 0 2024-09-21T04:15:01.024 INFO:tasks.workunit.client.1.smithi148.stdout:1/35: mkdir db/dd 0 2024-09-21T04:15:01.043 INFO:tasks.workunit.client.1.smithi148.stdout:1/36: dwrite f6 [0,4194304] 0 2024-09-21T04:15:01.062 INFO:tasks.workunit.client.0.smithi110.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-21T04:15:01.063 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:15:01.063 INFO:tasks.workunit.client.0.smithi110.stderr:+ make 2024-09-21T04:15:01.097 INFO:tasks.workunit.client.1.smithi148.stdout:1/37: dwrite f6 [4194304,4194304] 0 2024-09-21T04:15:01.106 INFO:tasks.workunit.client.1.smithi148.stdout:1/38: dread f6 [0,4194304] 0 2024-09-21T04:15:01.106 INFO:tasks.workunit.client.1.smithi148.stdout:1/39: chown db/dd 53 1 2024-09-21T04:15:01.108 INFO:tasks.workunit.client.0.smithi110.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-21T04:15:01.109 INFO:tasks.workunit.client.1.smithi148.stdout:1/40: truncate f6 5684655 0 2024-09-21T04:15:01.109 INFO:tasks.workunit.client.1.smithi148.stdout:1/41: rename db to db/dd/de 22 2024-09-21T04:15:01.110 INFO:tasks.workunit.client.1.smithi148.stdout:1/42: creat db/ff x:0 0 0 2024-09-21T04:15:01.127 INFO:tasks.workunit.client.1.smithi148.stdout:1/43: dwrite f6 [0,4194304] 0 2024-09-21T04:15:01.128 INFO:tasks.workunit.client.1.smithi148.stdout:1/44: symlink db/dd/l10 0 2024-09-21T04:15:01.131 INFO:tasks.workunit.client.1.smithi148.stdout:1/45: truncate f6 5633976 0 2024-09-21T04:15:01.132 INFO:tasks.workunit.client.1.smithi148.stdout:1/46: mknod db/dd/c11 0 2024-09-21T04:15:01.132 INFO:tasks.workunit.client.1.smithi148.stdout:1/47: fsync db/ff 0 2024-09-21T04:15:01.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/48: creat db/dd/f12 x:0 0 0 2024-09-21T04:15:01.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/49: dread - db/ff zero size 2024-09-21T04:15:01.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/50: fsync db/ff 0 2024-09-21T04:15:01.156 INFO:tasks.workunit.client.1.smithi148.stdout:1/51: dwrite db/ff [0,4194304] 0 2024-09-21T04:15:01.156 INFO:tasks.workunit.client.1.smithi148.stdout:1/52: dread - db/dd/f12 zero size 2024-09-21T04:15:01.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/53: dread f6 [4194304,4194304] 0 2024-09-21T04:15:01.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/54: creat db/dd/f13 x:0 0 0 2024-09-21T04:15:01.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/55: stat f4 0 2024-09-21T04:15:01.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/56: stat f6 0 2024-09-21T04:15:01.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/57: write f6 [6643464,129191] 0 2024-09-21T04:15:01.160 INFO:tasks.workunit.client.1.smithi148.stdout:1/58: mknod db/dd/c14 0 2024-09-21T04:15:01.661 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:01 smithi148 ceph-mon[34957]: pgmap v265: 65 pgs: 65 active+clean; 191 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 217 op/s 2024-09-21T04:15:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:01 smithi110 ceph-mon[27723]: pgmap v265: 65 pgs: 65 active+clean; 191 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s wr, 217 op/s 2024-09-21T04:15:02.966 INFO:tasks.workunit.client.0.smithi110.stderr:++ readlink -f fsstress 2024-09-21T04:15:02.967 INFO:tasks.workunit.client.0.smithi110.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-21T04:15:02.967 INFO:tasks.workunit.client.0.smithi110.stderr:+ popd 2024-09-21T04:15:02.967 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:15:02.968 INFO:tasks.workunit.client.0.smithi110.stderr:+ popd 2024-09-21T04:15:02.968 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:15:02.968 INFO:tasks.workunit.client.0.smithi110.stderr:++ mktemp -d -p . 2024-09-21T04:15:02.969 INFO:tasks.workunit.client.0.smithi110.stderr:+ T=./tmp.zpC9udqq9t 2024-09-21T04:15:02.969 INFO:tasks.workunit.client.0.smithi110.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.zpC9udqq9t -l 1 -n 1000 -p 10 -v 2024-09-21T04:15:02.970 INFO:tasks.workunit.client.0.smithi110.stdout:seed = 1725989224 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/0: truncate - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/1: dwrite - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/2: write - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/3: stat - no entries 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/4: write - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/5: dread - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/6: dwrite - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/7: dread - no filename 2024-09-21T04:15:02.972 INFO:tasks.workunit.client.0.smithi110.stdout:1/0: getdents . 0 2024-09-21T04:15:02.973 INFO:tasks.workunit.client.0.smithi110.stdout:1/1: fdatasync - no filename 2024-09-21T04:15:02.973 INFO:tasks.workunit.client.0.smithi110.stdout:0/8: creat f0 x:0 0 0 2024-09-21T04:15:02.973 INFO:tasks.workunit.client.0.smithi110.stdout:0/9: dread - f0 zero size 2024-09-21T04:15:02.973 INFO:tasks.workunit.client.0.smithi110.stdout:0/10: creat f1 x:0 0 0 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:0/11: dread - f0 zero size 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:0/12: creat f2 x:0 0 0 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:2/0: dread - no filename 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:2/1: write - no filename 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:2/2: write - no filename 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/2: creat f0 x:0 0 0 2024-09-21T04:15:02.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/3: write f0 [909201,8407] 0 2024-09-21T04:15:02.975 INFO:tasks.workunit.client.0.smithi110.stdout:3/0: rename - no filename 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:3/1: link - no file 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:3/2: chown . 24 1 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:3/3: read - no filename 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:3/4: dwrite - no filename 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/3: mknod c0 0 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/4: chown c0 3 1 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/5: chown c0 31 1 2024-09-21T04:15:02.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/4: rename f0 to f1 0 2024-09-21T04:15:02.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/0: read - no filename 2024-09-21T04:15:02.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/1: chown . 0 1 2024-09-21T04:15:02.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/2: chown . 58 1 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:3/5: mkdir d0 0 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:3/6: dwrite - no filename 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:3/7: read - no filename 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:3/8: dread - no filename 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:2/6: rename c0 to c1 0 2024-09-21T04:15:02.978 INFO:tasks.workunit.client.0.smithi110.stdout:2/7: write - no filename 2024-09-21T04:15:02.979 INFO:tasks.workunit.client.0.smithi110.stdout:2/8: truncate - no filename 2024-09-21T04:15:02.979 INFO:tasks.workunit.client.0.smithi110.stdout:2/9: chown c1 16630458 1 2024-09-21T04:15:02.979 INFO:tasks.workunit.client.0.smithi110.stdout:1/5: mkdir d2 0 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/0: chown . 18 1 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/1: rename - no filename 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/2: write - no filename 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/3: getdents . 0 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/4: link - no file 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/5: dwrite - no filename 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:1/6: write f1 [846505,89928] 0 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/3: mknod c0 0 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/4: truncate - no filename 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/5: dread - no filename 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/6: stat c0 0 2024-09-21T04:15:02.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/7: chown c0 5192 1 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:3/9: symlink d0/l1 0 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:2/10: mkdir d2 0 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:6/0: fsync - no filename 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:6/1: rename - no filename 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:6/2: rename - no filename 2024-09-21T04:15:02.981 INFO:tasks.workunit.client.0.smithi110.stdout:6/3: dread - no filename 2024-09-21T04:15:02.982 INFO:tasks.workunit.client.0.smithi110.stdout:5/6: mknod c0 0 2024-09-21T04:15:02.982 INFO:tasks.workunit.client.0.smithi110.stdout:5/7: truncate - no filename 2024-09-21T04:15:02.982 INFO:tasks.workunit.client.0.smithi110.stdout:1/7: symlink d2/l3 0 2024-09-21T04:15:02.982 INFO:tasks.workunit.client.0.smithi110.stdout:1/8: chown f1 3100998 1 2024-09-21T04:15:02.983 INFO:tasks.workunit.client.0.smithi110.stdout:4/8: creat f1 x:0 0 0 2024-09-21T04:15:02.984 INFO:tasks.workunit.client.0.smithi110.stdout:3/10: creat d0/f2 x:0 0 0 2024-09-21T04:15:02.984 INFO:tasks.workunit.client.0.smithi110.stdout:3/11: write d0/f2 [311789,2037] 0 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/12: chown d0 7 1 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/13: write d0/f2 [801086,1563] 0 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/14: write d0/f2 [1454552,7108] 0 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/15: rename d0 to d0/d3 22 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/11: mkdir d2/d3 0 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/12: write - no filename 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/13: fdatasync - no filename 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/14: chown c1 99876999 1 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/15: fdatasync - no filename 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/16: read - no filename 2024-09-21T04:15:02.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/16: write d0/f2 [1411292,6570] 0 2024-09-21T04:15:02.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/4: mknod c0 0 2024-09-21T04:15:02.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/5: dwrite - no filename 2024-09-21T04:15:02.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/6: write - no filename 2024-09-21T04:15:02.986 INFO:tasks.workunit.client.0.smithi110.stdout:5/8: mkdir d1 0 2024-09-21T04:15:02.986 INFO:tasks.workunit.client.0.smithi110.stdout:4/9: symlink l2 0 2024-09-21T04:15:02.987 INFO:tasks.workunit.client.0.smithi110.stdout:2/17: creat d2/f4 x:0 0 0 2024-09-21T04:15:02.987 INFO:tasks.workunit.client.0.smithi110.stdout:2/18: write d2/f4 [526734,3099] 0 2024-09-21T04:15:02.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/0: truncate - no filename 2024-09-21T04:15:02.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/1: chown . 48387901 1 2024-09-21T04:15:02.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/2: chown . 118587 1 2024-09-21T04:15:02.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/3: truncate - no filename 2024-09-21T04:15:02.989 INFO:tasks.workunit.client.0.smithi110.stdout:8/4: rmdir - no directory 2024-09-21T04:15:02.989 INFO:tasks.workunit.client.0.smithi110.stdout:7/0: symlink l0 0 2024-09-21T04:15:02.989 INFO:tasks.workunit.client.0.smithi110.stdout:6/7: rename c0 to c1 0 2024-09-21T04:15:02.989 INFO:tasks.workunit.client.0.smithi110.stdout:6/8: dwrite - no filename 2024-09-21T04:15:02.990 INFO:tasks.workunit.client.0.smithi110.stdout:5/9: rename c0 to d1/c2 0 2024-09-21T04:15:02.991 INFO:tasks.workunit.client.0.smithi110.stdout:5/10: dread - no filename 2024-09-21T04:15:02.991 INFO:tasks.workunit.client.0.smithi110.stdout:4/10: rename f1 to f3 0 2024-09-21T04:15:02.991 INFO:tasks.workunit.client.0.smithi110.stdout:4/11: creat f4 x:0 0 0 2024-09-21T04:15:02.992 INFO:tasks.workunit.client.0.smithi110.stdout:4/12: chown c0 263 1 2024-09-21T04:15:02.992 INFO:tasks.workunit.client.0.smithi110.stdout:4/13: creat f5 x:0 0 0 2024-09-21T04:15:02.992 INFO:tasks.workunit.client.0.smithi110.stdout:2/19: mknod d2/c5 0 2024-09-21T04:15:02.992 INFO:tasks.workunit.client.0.smithi110.stdout:1/9: dread f1 [0,4194304] 0 2024-09-21T04:15:02.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/5: mkdir d0 0 2024-09-21T04:15:02.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/6: getdents d0 0 2024-09-21T04:15:02.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/7: dread - no filename 2024-09-21T04:15:02.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/8: rename d0 to d0/d1 22 2024-09-21T04:15:02.995 INFO:tasks.workunit.client.0.smithi110.stdout:7/1: symlink l1 0 2024-09-21T04:15:02.996 INFO:tasks.workunit.client.0.smithi110.stdout:5/11: mkdir d1/d3 0 2024-09-21T04:15:02.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/9: creat f2 x:0 0 0 2024-09-21T04:15:02.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/10: rmdir - no directory 2024-09-21T04:15:02.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/11: write f2 [506722,84078] 0 2024-09-21T04:15:02.997 INFO:tasks.workunit.client.0.smithi110.stdout:0/13: dwrite f1 [0,4194304] 0 2024-09-21T04:15:02.998 INFO:tasks.workunit.client.0.smithi110.stdout:0/14: dread - f2 zero size 2024-09-21T04:15:02.999 INFO:tasks.workunit.client.0.smithi110.stdout:1/10: creat d2/f4 x:0 0 0 2024-09-21T04:15:03.000 INFO:tasks.workunit.client.0.smithi110.stdout:9/0: creat f0 x:0 0 0 2024-09-21T04:15:03.001 INFO:tasks.workunit.client.0.smithi110.stdout:8/9: creat d0/f2 x:0 0 0 2024-09-21T04:15:03.006 INFO:tasks.workunit.client.0.smithi110.stdout:6/12: rename f2 to f3 0 2024-09-21T04:15:03.007 INFO:tasks.workunit.client.0.smithi110.stdout:1/11: unlink d2/l3 0 2024-09-21T04:15:03.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/10: rename d0/f2 to d0/f3 0 2024-09-21T04:15:03.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/11: creat d0/f4 x:0 0 0 2024-09-21T04:15:03.008 INFO:tasks.workunit.client.0.smithi110.stdout:1/12: mknod d2/c5 0 2024-09-21T04:15:03.040 INFO:tasks.workunit.client.0.smithi110.stdout:7/2: fsync - no filename 2024-09-21T04:15:03.046 INFO:tasks.workunit.client.0.smithi110.stdout:7/3: unlink l0 0 2024-09-21T04:15:03.046 INFO:tasks.workunit.client.0.smithi110.stdout:7/4: dread - no filename 2024-09-21T04:15:03.046 INFO:tasks.workunit.client.0.smithi110.stdout:7/5: dwrite - no filename 2024-09-21T04:15:03.049 INFO:tasks.workunit.client.0.smithi110.stdout:9/1: unlink f0 0 2024-09-21T04:15:03.049 INFO:tasks.workunit.client.0.smithi110.stdout:9/2: dread - no filename 2024-09-21T04:15:03.049 INFO:tasks.workunit.client.0.smithi110.stdout:9/3: creat f1 x:0 0 0 2024-09-21T04:15:03.049 INFO:tasks.workunit.client.0.smithi110.stdout:9/4: chown f1 1 1 2024-09-21T04:15:03.059 INFO:tasks.workunit.client.0.smithi110.stdout:0/15: write f1 [2703699,123452] 0 2024-09-21T04:15:03.059 INFO:tasks.workunit.client.0.smithi110.stdout:0/16: write f2 [174103,41644] 0 2024-09-21T04:15:03.064 INFO:tasks.workunit.client.0.smithi110.stdout:1/13: link f1 d2/f6 0 2024-09-21T04:15:03.064 INFO:tasks.workunit.client.0.smithi110.stdout:1/14: creat d2/f7 x:0 0 0 2024-09-21T04:15:03.065 INFO:tasks.workunit.client.0.smithi110.stdout:9/5: dwrite f1 [0,4194304] 0 2024-09-21T04:15:03.074 INFO:tasks.workunit.client.0.smithi110.stdout:3/17: getdents d0 0 2024-09-21T04:15:03.074 INFO:tasks.workunit.client.0.smithi110.stdout:3/18: truncate d0/f2 1712576 0 2024-09-21T04:15:03.077 INFO:tasks.workunit.client.0.smithi110.stdout:0/17: getdents . 0 2024-09-21T04:15:03.077 INFO:tasks.workunit.client.0.smithi110.stdout:0/18: readlink - no filename 2024-09-21T04:15:03.078 INFO:tasks.workunit.client.0.smithi110.stdout:3/19: creat d0/f4 x:0 0 0 2024-09-21T04:15:03.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/6: rename f1 to f2 0 2024-09-21T04:15:03.080 INFO:tasks.workunit.client.0.smithi110.stdout:3/20: mknod d0/c5 0 2024-09-21T04:15:03.080 INFO:tasks.workunit.client.0.smithi110.stdout:3/21: creat d0/f6 x:0 0 0 2024-09-21T04:15:03.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/12: link d1/c2 d1/c4 0 2024-09-21T04:15:03.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/13: truncate - no filename 2024-09-21T04:15:03.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/14: write - no filename 2024-09-21T04:15:03.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/15: readlink - no filename 2024-09-21T04:15:03.083 INFO:tasks.workunit.client.0.smithi110.stdout:3/22: rename d0/f2 to d0/f7 0 2024-09-21T04:15:03.084 INFO:tasks.workunit.client.0.smithi110.stdout:5/16: mknod d1/d3/c5 0 2024-09-21T04:15:03.084 INFO:tasks.workunit.client.0.smithi110.stdout:5/17: readlink - no filename 2024-09-21T04:15:03.085 INFO:tasks.workunit.client.0.smithi110.stdout:3/23: rename d0/c5 to d0/c8 0 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/24: dread - d0/f6 zero size 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/25: dread - d0/f4 zero size 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/26: creat d0/f9 x:0 0 0 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/27: creat d0/fa x:0 0 0 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/28: read - d0/f9 zero size 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/29: creat d0/fb x:0 0 0 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/30: chown d0/c8 402 1 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/31: stat d0/f4 0 2024-09-21T04:15:03.086 INFO:tasks.workunit.client.0.smithi110.stdout:1/15: dwrite d2/f4 [0,4194304] 0 2024-09-21T04:15:03.087 INFO:tasks.workunit.client.0.smithi110.stdout:5/18: creat d1/f6 x:0 0 0 2024-09-21T04:15:03.087 INFO:tasks.workunit.client.0.smithi110.stdout:0/19: dread f2 [0,4194304] 0 2024-09-21T04:15:03.090 INFO:tasks.workunit.client.0.smithi110.stdout:4/14: getdents . 0 2024-09-21T04:15:03.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/16: rename d2/f7 to d2/f8 0 2024-09-21T04:15:03.093 INFO:tasks.workunit.client.0.smithi110.stdout:5/19: mkdir d1/d7 0 2024-09-21T04:15:03.094 INFO:tasks.workunit.client.0.smithi110.stdout:3/32: unlink d0/fb 0 2024-09-21T04:15:03.095 INFO:tasks.workunit.client.0.smithi110.stdout:4/15: symlink l6 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/20: write f1 [5096930,115267] 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/21: write f0 [88292,61702] 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/22: write f0 [958189,36196] 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/23: write f1 [443995,94732] 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/24: getdents . 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/25: write f1 [1028273,129850] 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:5/20: mkdir d1/d7/d8 0 2024-09-21T04:15:03.097 INFO:tasks.workunit.client.0.smithi110.stdout:5/21: write d1/f6 [472763,46397] 0 2024-09-21T04:15:03.099 INFO:tasks.workunit.client.0.smithi110.stdout:8/12: unlink d0/f3 0 2024-09-21T04:15:03.099 INFO:tasks.workunit.client.0.smithi110.stdout:8/13: write d0/f4 [201139,52095] 0 2024-09-21T04:15:03.100 INFO:tasks.workunit.client.0.smithi110.stdout:0/26: symlink l3 0 2024-09-21T04:15:03.101 INFO:tasks.workunit.client.0.smithi110.stdout:5/22: creat d1/d7/d8/f9 x:0 0 0 2024-09-21T04:15:03.102 INFO:tasks.workunit.client.0.smithi110.stdout:4/16: mknod c7 0 2024-09-21T04:15:03.102 INFO:tasks.workunit.client.0.smithi110.stdout:8/14: dread d0/f4 [0,4194304] 0 2024-09-21T04:15:03.103 INFO:tasks.workunit.client.0.smithi110.stdout:0/27: mkdir d4 0 2024-09-21T04:15:03.104 INFO:tasks.workunit.client.0.smithi110.stdout:5/23: unlink d1/d7/d8/f9 0 2024-09-21T04:15:03.104 INFO:tasks.workunit.client.0.smithi110.stdout:5/24: chown d1/d7 121 1 2024-09-21T04:15:03.105 INFO:tasks.workunit.client.0.smithi110.stdout:5/25: truncate d1/f6 797871 0 2024-09-21T04:15:03.105 INFO:tasks.workunit.client.0.smithi110.stdout:4/17: creat f8 x:0 0 0 2024-09-21T04:15:03.106 INFO:tasks.workunit.client.0.smithi110.stdout:0/28: dread f0 [0,4194304] 0 2024-09-21T04:15:03.106 INFO:tasks.workunit.client.0.smithi110.stdout:0/29: getdents d4 0 2024-09-21T04:15:03.106 INFO:tasks.workunit.client.0.smithi110.stdout:0/30: fsync f0 0 2024-09-21T04:15:03.107 INFO:tasks.workunit.client.0.smithi110.stdout:8/15: rename d0/f4 to d0/f5 0 2024-09-21T04:15:03.107 INFO:tasks.workunit.client.0.smithi110.stdout:8/16: write d0/f5 [1080479,49317] 0 2024-09-21T04:15:03.108 INFO:tasks.workunit.client.0.smithi110.stdout:5/26: dread d1/f6 [0,4194304] 0 2024-09-21T04:15:03.108 INFO:tasks.workunit.client.0.smithi110.stdout:5/27: creat d1/d7/d8/fa x:0 0 0 2024-09-21T04:15:03.108 INFO:tasks.workunit.client.0.smithi110.stdout:5/28: dread - d1/d7/d8/fa zero size 2024-09-21T04:15:03.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/31: creat d4/f5 x:0 0 0 2024-09-21T04:15:03.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/32: stat f0 0 2024-09-21T04:15:03.110 INFO:tasks.workunit.client.0.smithi110.stdout:8/17: symlink d0/l6 0 2024-09-21T04:15:03.111 INFO:tasks.workunit.client.0.smithi110.stdout:5/29: stat d1/c4 0 2024-09-21T04:15:03.111 INFO:tasks.workunit.client.0.smithi110.stdout:5/30: truncate d1/d7/d8/fa 574087 0 2024-09-21T04:15:03.111 INFO:tasks.workunit.client.0.smithi110.stdout:5/31: write d1/d7/d8/fa [1521683,98438] 0 2024-09-21T04:15:03.113 INFO:tasks.workunit.client.0.smithi110.stdout:8/18: mkdir d0/d7 0 2024-09-21T04:15:03.123 INFO:tasks.workunit.client.0.smithi110.stdout:1/17: fsync f1 0 2024-09-21T04:15:03.124 INFO:tasks.workunit.client.0.smithi110.stdout:1/18: chown d2/f8 0 1 2024-09-21T04:15:03.129 INFO:tasks.workunit.client.0.smithi110.stdout:4/18: dwrite f4 [0,4194304] 0 2024-09-21T04:15:03.129 INFO:tasks.workunit.client.0.smithi110.stdout:5/32: rename d1/d3 to d1/d7/db 0 2024-09-21T04:15:03.130 INFO:tasks.workunit.client.0.smithi110.stdout:8/19: dwrite d0/f5 [0,4194304] 0 2024-09-21T04:15:03.134 INFO:tasks.workunit.client.0.smithi110.stdout:1/19: write d2/f4 [2536660,97114] 0 2024-09-21T04:15:03.134 INFO:tasks.workunit.client.0.smithi110.stdout:1/20: write f1 [229342,60891] 0 2024-09-21T04:15:03.134 INFO:tasks.workunit.client.0.smithi110.stdout:8/20: symlink d0/d7/l8 0 2024-09-21T04:15:03.135 INFO:tasks.workunit.client.0.smithi110.stdout:4/19: mknod c9 0 2024-09-21T04:15:03.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/21: mkdir d2/d9 0 2024-09-21T04:15:03.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/22: chown d2 8 1 2024-09-21T04:15:03.137 INFO:tasks.workunit.client.0.smithi110.stdout:8/21: symlink d0/d7/l9 0 2024-09-21T04:15:03.138 INFO:tasks.workunit.client.0.smithi110.stdout:4/20: readlink l2 0 2024-09-21T04:15:03.139 INFO:tasks.workunit.client.0.smithi110.stdout:8/22: creat d0/d7/fa x:0 0 0 2024-09-21T04:15:03.139 INFO:tasks.workunit.client.0.smithi110.stdout:8/23: creat d0/fb x:0 0 0 2024-09-21T04:15:03.140 INFO:tasks.workunit.client.0.smithi110.stdout:8/24: chown d0/f5 14832 1 2024-09-21T04:15:03.140 INFO:tasks.workunit.client.0.smithi110.stdout:8/25: truncate d0/f5 4213308 0 2024-09-21T04:15:03.143 INFO:tasks.workunit.client.0.smithi110.stdout:4/21: link f8 fa 0 2024-09-21T04:15:03.144 INFO:tasks.workunit.client.0.smithi110.stdout:4/22: creat fb x:0 0 0 2024-09-21T04:15:03.146 INFO:tasks.workunit.client.0.smithi110.stdout:4/23: mknod cc 0 2024-09-21T04:15:03.149 INFO:tasks.workunit.client.0.smithi110.stdout:0/33: rmdir d4 39 2024-09-21T04:15:03.150 INFO:tasks.workunit.client.0.smithi110.stdout:0/34: mknod d4/c6 0 2024-09-21T04:15:03.151 INFO:tasks.workunit.client.0.smithi110.stdout:0/35: mknod d4/c7 0 2024-09-21T04:15:03.152 INFO:tasks.workunit.client.0.smithi110.stdout:0/36: creat d4/f8 x:0 0 0 2024-09-21T04:15:03.153 INFO:tasks.workunit.client.0.smithi110.stdout:5/33: rename d1/d7/db to d1/dc 0 2024-09-21T04:15:03.153 INFO:tasks.workunit.client.0.smithi110.stdout:1/23: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:03.154 INFO:tasks.workunit.client.0.smithi110.stdout:0/37: rename d4/c6 to d4/c9 0 2024-09-21T04:15:03.155 INFO:tasks.workunit.client.0.smithi110.stdout:5/34: mknod d1/dc/cd 0 2024-09-21T04:15:03.161 INFO:tasks.workunit.client.0.smithi110.stdout:5/35: mkdir d1/d7/d8/de 0 2024-09-21T04:15:03.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/26: getdents d0/d7 0 2024-09-21T04:15:03.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/27: fdatasync d0/fb 0 2024-09-21T04:15:03.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/28: fsync d0/f5 0 2024-09-21T04:15:03.163 INFO:tasks.workunit.client.0.smithi110.stdout:1/24: getdents d2 0 2024-09-21T04:15:03.167 INFO:tasks.workunit.client.0.smithi110.stdout:8/29: link d0/l6 d0/d7/lc 0 2024-09-21T04:15:03.170 INFO:tasks.workunit.client.0.smithi110.stdout:8/30: rename d0/f5 to d0/d7/fd 0 2024-09-21T04:15:03.171 INFO:tasks.workunit.client.0.smithi110.stdout:0/38: rmdir d4 39 2024-09-21T04:15:03.171 INFO:tasks.workunit.client.0.smithi110.stdout:0/39: dread - d4/f5 zero size 2024-09-21T04:15:03.171 INFO:tasks.workunit.client.0.smithi110.stdout:0/40: chown d4 656815 1 2024-09-21T04:15:03.171 INFO:tasks.workunit.client.0.smithi110.stdout:0/41: write f0 [1559661,18004] 0 2024-09-21T04:15:03.171 INFO:tasks.workunit.client.0.smithi110.stdout:8/31: stat d0/l6 0 2024-09-21T04:15:03.173 INFO:tasks.workunit.client.0.smithi110.stdout:0/42: mknod d4/ca 0 2024-09-21T04:15:03.173 INFO:tasks.workunit.client.0.smithi110.stdout:0/43: fdatasync d4/f8 0 2024-09-21T04:15:03.174 INFO:tasks.workunit.client.0.smithi110.stdout:1/25: truncate d2/f6 935897 0 2024-09-21T04:15:03.175 INFO:tasks.workunit.client.0.smithi110.stdout:1/26: rmdir d2 39 2024-09-21T04:15:03.177 INFO:tasks.workunit.client.0.smithi110.stdout:0/44: dread f0 [0,4194304] 0 2024-09-21T04:15:03.179 INFO:tasks.workunit.client.0.smithi110.stdout:0/45: creat d4/fb x:0 0 0 2024-09-21T04:15:03.179 INFO:tasks.workunit.client.0.smithi110.stdout:0/46: chown f1 13101662 1 2024-09-21T04:15:03.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/32: dread d0/d7/fd [0,4194304] 0 2024-09-21T04:15:03.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/33: read - d0/d7/fa zero size 2024-09-21T04:15:03.181 INFO:tasks.workunit.client.0.smithi110.stdout:8/34: getdents d0/d7 0 2024-09-21T04:15:03.181 INFO:tasks.workunit.client.0.smithi110.stdout:0/47: dread f0 [0,4194304] 0 2024-09-21T04:15:03.181 INFO:tasks.workunit.client.0.smithi110.stdout:0/48: dread - d4/f8 zero size 2024-09-21T04:15:03.181 INFO:tasks.workunit.client.0.smithi110.stdout:1/27: dread d2/f4 [0,4194304] 0 2024-09-21T04:15:03.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/28: dread f1 [0,4194304] 0 2024-09-21T04:15:03.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/29: getdents d2 0 2024-09-21T04:15:03.185 INFO:tasks.workunit.client.0.smithi110.stdout:8/35: read d0/d7/fd [1781592,40918] 0 2024-09-21T04:15:03.186 INFO:tasks.workunit.client.0.smithi110.stdout:8/36: write d0/d7/fd [703596,107678] 0 2024-09-21T04:15:03.186 INFO:tasks.workunit.client.0.smithi110.stdout:8/37: chown d0/d7/fd 1307 1 2024-09-21T04:15:03.186 INFO:tasks.workunit.client.0.smithi110.stdout:1/30: dread f1 [0,4194304] 0 2024-09-21T04:15:03.187 INFO:tasks.workunit.client.0.smithi110.stdout:8/38: symlink d0/d7/le 0 2024-09-21T04:15:03.187 INFO:tasks.workunit.client.0.smithi110.stdout:8/39: fsync d0/fb 0 2024-09-21T04:15:03.187 INFO:tasks.workunit.client.0.smithi110.stdout:8/40: fdatasync d0/fb 0 2024-09-21T04:15:03.196 INFO:tasks.workunit.client.0.smithi110.stdout:8/41: truncate d0/d7/fd 2060936 0 2024-09-21T04:15:03.197 INFO:tasks.workunit.client.0.smithi110.stdout:1/31: write d2/f8 [1445020,80723] 0 2024-09-21T04:15:03.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/49: dwrite f1 [0,4194304] 0 2024-09-21T04:15:03.205 INFO:tasks.workunit.client.0.smithi110.stdout:0/50: truncate f1 469950 0 2024-09-21T04:15:03.211 INFO:tasks.workunit.client.0.smithi110.stdout:0/51: unlink f1 0 2024-09-21T04:15:03.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/52: dread f2 [0,4194304] 0 2024-09-21T04:15:03.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/53: dread - d4/fb zero size 2024-09-21T04:15:03.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/54: write d4/fb [576201,55710] 0 2024-09-21T04:15:03.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/55: write d4/f5 [1020593,17689] 0 2024-09-21T04:15:03.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/56: read d4/fb [298369,9542] 0 2024-09-21T04:15:03.213 INFO:tasks.workunit.client.0.smithi110.stdout:0/57: chown d4/c9 24 1 2024-09-21T04:15:03.213 INFO:tasks.workunit.client.0.smithi110.stdout:0/58: link d4/c9 d4/cc 0 2024-09-21T04:15:03.217 INFO:tasks.workunit.client.0.smithi110.stdout:8/42: dwrite d0/d7/fd [0,4194304] 0 2024-09-21T04:15:03.217 INFO:tasks.workunit.client.0.smithi110.stdout:8/43: dread - d0/fb zero size 2024-09-21T04:15:03.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/32: dwrite f1 [0,4194304] 0 2024-09-21T04:15:03.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/33: creat d2/fa x:0 0 0 2024-09-21T04:15:03.229 INFO:tasks.workunit.client.0.smithi110.stdout:1/34: getdents d2 0 2024-09-21T04:15:03.229 INFO:tasks.workunit.client.0.smithi110.stdout:1/35: readlink - no filename 2024-09-21T04:15:03.229 INFO:tasks.workunit.client.0.smithi110.stdout:1/36: write d2/f8 [2523304,116404] 0 2024-09-21T04:15:03.229 INFO:tasks.workunit.client.0.smithi110.stdout:1/37: chown f1 0 1 2024-09-21T04:15:03.229 INFO:tasks.workunit.client.0.smithi110.stdout:1/38: symlink d2/lb 0 2024-09-21T04:15:03.230 INFO:tasks.workunit.client.0.smithi110.stdout:1/39: unlink d2/fa 0 2024-09-21T04:15:03.240 INFO:tasks.workunit.client.0.smithi110.stdout:0/59: dwrite f2 [0,4194304] 0 2024-09-21T04:15:03.240 INFO:tasks.workunit.client.0.smithi110.stdout:0/60: fdatasync d4/f8 0 2024-09-21T04:15:03.241 INFO:tasks.workunit.client.0.smithi110.stdout:0/61: mkdir d4/dd 0 2024-09-21T04:15:03.242 INFO:tasks.workunit.client.0.smithi110.stdout:0/62: unlink d4/f8 0 2024-09-21T04:15:03.249 INFO:tasks.workunit.client.0.smithi110.stdout:1/40: dwrite d2/f6 [0,4194304] 0 2024-09-21T04:15:03.252 INFO:tasks.workunit.client.0.smithi110.stdout:1/41: dread d2/f4 [0,4194304] 0 2024-09-21T04:15:03.253 INFO:tasks.workunit.client.0.smithi110.stdout:1/42: creat d2/fc x:0 0 0 2024-09-21T04:15:03.254 INFO:tasks.workunit.client.0.smithi110.stdout:1/43: link d2/f6 d2/fd 0 2024-09-21T04:15:03.254 INFO:tasks.workunit.client.0.smithi110.stdout:1/44: creat d2/fe x:0 0 0 2024-09-21T04:15:03.254 INFO:tasks.workunit.client.0.smithi110.stdout:1/45: write d2/f8 [1570654,104226] 0 2024-09-21T04:15:03.255 INFO:tasks.workunit.client.0.smithi110.stdout:0/63: dwrite f0 [0,4194304] 0 2024-09-21T04:15:03.260 INFO:tasks.workunit.client.0.smithi110.stdout:0/64: dread d4/fb [0,4194304] 0 2024-09-21T04:15:03.261 INFO:tasks.workunit.client.0.smithi110.stdout:0/65: creat d4/dd/fe x:0 0 0 2024-09-21T04:15:03.261 INFO:tasks.workunit.client.0.smithi110.stdout:0/66: rename d4 to d4/df 22 2024-09-21T04:15:03.261 INFO:tasks.workunit.client.0.smithi110.stdout:0/67: write d4/f5 [1051567,58178] 0 2024-09-21T04:15:03.263 INFO:tasks.workunit.client.0.smithi110.stdout:0/68: fdatasync d4/f5 0 2024-09-21T04:15:03.264 INFO:tasks.workunit.client.0.smithi110.stdout:0/69: mkdir d4/dd/d10 0 2024-09-21T04:15:03.273 INFO:tasks.workunit.client.0.smithi110.stdout:1/46: dwrite d2/fc [0,4194304] 0 2024-09-21T04:15:03.275 INFO:tasks.workunit.client.0.smithi110.stdout:1/47: creat d2/d9/ff x:0 0 0 2024-09-21T04:15:03.277 INFO:tasks.workunit.client.0.smithi110.stdout:1/48: rename d2/f6 to d2/d9/f10 0 2024-09-21T04:15:03.277 INFO:tasks.workunit.client.0.smithi110.stdout:1/49: write f1 [4323331,85543] 0 2024-09-21T04:15:03.277 INFO:tasks.workunit.client.0.smithi110.stdout:1/50: write d2/d9/ff [68706,91091] 0 2024-09-21T04:15:03.278 INFO:tasks.workunit.client.0.smithi110.stdout:1/51: rename d2/c5 to d2/d9/c11 0 2024-09-21T04:15:03.281 INFO:tasks.workunit.client.0.smithi110.stdout:0/70: dwrite d4/dd/fe [0,4194304] 0 2024-09-21T04:15:03.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/52: rmdir d2 39 2024-09-21T04:15:03.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/53: symlink d2/l12 0 2024-09-21T04:15:03.287 INFO:tasks.workunit.client.0.smithi110.stdout:0/71: dread f0 [0,4194304] 0 2024-09-21T04:15:03.287 INFO:tasks.workunit.client.0.smithi110.stdout:0/72: readlink l3 0 2024-09-21T04:15:03.289 INFO:tasks.workunit.client.0.smithi110.stdout:0/73: link d4/f5 d4/f11 0 2024-09-21T04:15:03.304 INFO:tasks.workunit.client.0.smithi110.stdout:0/74: dwrite f0 [0,4194304] 0 2024-09-21T04:15:03.305 INFO:tasks.workunit.client.0.smithi110.stdout:0/75: mkdir d4/dd/d12 0 2024-09-21T04:15:03.305 INFO:tasks.workunit.client.0.smithi110.stdout:0/76: chown d4/dd/fe 443 1 2024-09-21T04:15:03.305 INFO:tasks.workunit.client.0.smithi110.stdout:0/77: creat d4/dd/d10/f13 x:0 0 0 2024-09-21T04:15:03.306 INFO:tasks.workunit.client.0.smithi110.stdout:0/78: mknod d4/c14 0 2024-09-21T04:15:03.306 INFO:tasks.workunit.client.0.smithi110.stdout:0/79: stat d4/fb 0 2024-09-21T04:15:03.309 INFO:tasks.workunit.client.0.smithi110.stdout:1/54: dwrite d2/fd [4194304,4194304] 0 2024-09-21T04:15:03.309 INFO:tasks.workunit.client.0.smithi110.stdout:1/55: write d2/fe [1034539,34138] 0 2024-09-21T04:15:03.312 INFO:tasks.workunit.client.0.smithi110.stdout:1/56: getdents d2 0 2024-09-21T04:15:03.315 INFO:tasks.workunit.client.0.smithi110.stdout:1/57: truncate d2/d9/f10 726608 0 2024-09-21T04:15:03.321 INFO:tasks.workunit.client.0.smithi110.stdout:0/80: dwrite f2 [4194304,4194304] 0 2024-09-21T04:15:03.322 INFO:tasks.workunit.client.0.smithi110.stdout:0/81: truncate d4/dd/d10/f13 426279 0 2024-09-21T04:15:03.329 INFO:tasks.workunit.client.0.smithi110.stdout:0/82: write f2 [2259563,45183] 0 2024-09-21T04:15:03.330 INFO:tasks.workunit.client.0.smithi110.stdout:0/83: truncate d4/dd/fe 718962 0 2024-09-21T04:15:03.331 INFO:tasks.workunit.client.0.smithi110.stdout:0/84: rename d4/c9 to d4/dd/d10/c15 0 2024-09-21T04:15:03.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/58: dwrite d2/fe [0,4194304] 0 2024-09-21T04:15:03.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/59: fsync d2/d9/f10 0 2024-09-21T04:15:03.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/60: write d2/d9/ff [598888,44605] 0 2024-09-21T04:15:03.336 INFO:tasks.workunit.client.0.smithi110.stdout:0/85: rmdir d4/dd/d10 39 2024-09-21T04:15:03.336 INFO:tasks.workunit.client.0.smithi110.stdout:0/86: symlink d4/dd/l16 0 2024-09-21T04:15:03.337 INFO:tasks.workunit.client.0.smithi110.stdout:0/87: chown d4/dd/l16 1837742 1 2024-09-21T04:15:03.350 INFO:tasks.workunit.client.0.smithi110.stdout:1/61: dwrite d2/d9/f10 [0,4194304] 0 2024-09-21T04:15:03.350 INFO:tasks.workunit.client.0.smithi110.stdout:1/62: rename d2 to d2/d9/d13 22 2024-09-21T04:15:03.362 INFO:tasks.workunit.client.0.smithi110.stdout:0/88: dwrite f2 [0,4194304] 0 2024-09-21T04:15:03.366 INFO:tasks.workunit.client.0.smithi110.stdout:1/63: dwrite d2/d9/ff [0,4194304] 0 2024-09-21T04:15:03.368 INFO:tasks.workunit.client.0.smithi110.stdout:1/64: symlink d2/d9/l14 0 2024-09-21T04:15:03.369 INFO:tasks.workunit.client.0.smithi110.stdout:1/65: unlink d2/fd 0 2024-09-21T04:15:03.369 INFO:tasks.workunit.client.0.smithi110.stdout:1/66: chown d2/lb 227155 1 2024-09-21T04:15:03.371 INFO:tasks.workunit.client.0.smithi110.stdout:1/67: link d2/d9/c11 d2/d9/c15 0 2024-09-21T04:15:03.371 INFO:tasks.workunit.client.0.smithi110.stdout:1/68: fdatasync d2/fe 0 2024-09-21T04:15:03.381 INFO:tasks.workunit.client.0.smithi110.stdout:0/89: dwrite d4/f5 [0,4194304] 0 2024-09-21T04:15:03.382 INFO:tasks.workunit.client.0.smithi110.stdout:0/90: symlink d4/dd/l17 0 2024-09-21T04:15:03.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/69: write f1 [1600016,22962] 0 2024-09-21T04:15:03.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/70: read f1 [1441308,102819] 0 2024-09-21T04:15:03.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/71: readlink d2/lb 0 2024-09-21T04:15:03.390 INFO:tasks.workunit.client.0.smithi110.stdout:1/72: creat d2/d9/f16 x:0 0 0 2024-09-21T04:15:03.391 INFO:tasks.workunit.client.0.smithi110.stdout:1/73: rename d2/d9/ff to d2/f17 0 2024-09-21T04:15:03.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/91: dwrite d4/f5 [0,4194304] 0 2024-09-21T04:15:03.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/92: readlink l3 0 2024-09-21T04:15:03.400 INFO:tasks.workunit.client.0.smithi110.stdout:1/74: getdents d2 0 2024-09-21T04:15:03.401 INFO:tasks.workunit.client.0.smithi110.stdout:0/93: mknod d4/dd/d12/c18 0 2024-09-21T04:15:03.401 INFO:tasks.workunit.client.0.smithi110.stdout:0/94: fsync d4/dd/d10/f13 0 2024-09-21T04:15:03.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/95: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:03.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/96: stat d4/c14 0 2024-09-21T04:15:03.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/97: chown d4/c7 2 1 2024-09-21T04:15:03.416 INFO:tasks.workunit.client.0.smithi110.stdout:1/75: dwrite d2/d9/f16 [0,4194304] 0 2024-09-21T04:15:03.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/98: mknod d4/c19 0 2024-09-21T04:15:03.438 INFO:tasks.workunit.client.0.smithi110.stdout:0/99: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:03.438 INFO:tasks.workunit.client.0.smithi110.stdout:0/100: stat d4/dd/fe 0 2024-09-21T04:15:03.438 INFO:tasks.workunit.client.0.smithi110.stdout:0/101: fdatasync d4/dd/d10/f13 0 2024-09-21T04:15:03.438 INFO:tasks.workunit.client.0.smithi110.stdout:0/102: chown d4/cc 0 1 2024-09-21T04:15:03.439 INFO:tasks.workunit.client.0.smithi110.stdout:0/103: rmdir d4/dd/d10 39 2024-09-21T04:15:03.440 INFO:tasks.workunit.client.0.smithi110.stdout:0/104: creat d4/dd/d10/f1a x:0 0 0 2024-09-21T04:15:03.440 INFO:tasks.workunit.client.0.smithi110.stdout:1/76: dwrite f1 [0,4194304] 0 2024-09-21T04:15:03.446 INFO:tasks.workunit.client.0.smithi110.stdout:1/77: dread d2/fe [0,4194304] 0 2024-09-21T04:15:03.448 INFO:tasks.workunit.client.0.smithi110.stdout:1/78: symlink d2/l18 0 2024-09-21T04:15:03.457 INFO:tasks.workunit.client.0.smithi110.stdout:0/105: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:03.457 INFO:tasks.workunit.client.0.smithi110.stdout:0/106: creat d4/dd/f1b x:0 0 0 2024-09-21T04:15:03.459 INFO:tasks.workunit.client.0.smithi110.stdout:0/107: getdents d4 0 2024-09-21T04:15:03.461 INFO:tasks.workunit.client.0.smithi110.stdout:0/108: mkdir d4/d1c 0 2024-09-21T04:15:03.466 INFO:tasks.workunit.client.0.smithi110.stdout:1/79: dread d2/f8 [0,4194304] 0 2024-09-21T04:15:03.466 INFO:tasks.workunit.client.0.smithi110.stdout:1/80: fdatasync d2/f17 0 2024-09-21T04:15:03.476 INFO:tasks.workunit.client.0.smithi110.stdout:1/81: write d2/fc [3610944,79068] 0 2024-09-21T04:15:03.477 INFO:tasks.workunit.client.0.smithi110.stdout:1/82: rmdir d2/d9 39 2024-09-21T04:15:03.480 INFO:tasks.workunit.client.0.smithi110.stdout:1/83: dread d2/f4 [0,4194304] 0 2024-09-21T04:15:03.480 INFO:tasks.workunit.client.0.smithi110.stdout:1/84: write d2/f8 [1972760,38531] 0 2024-09-21T04:15:03.481 INFO:tasks.workunit.client.0.smithi110.stdout:0/109: dwrite d4/dd/d10/f13 [0,4194304] 0 2024-09-21T04:15:03.481 INFO:tasks.workunit.client.0.smithi110.stdout:0/110: chown d4/dd/d10/f13 30 1 2024-09-21T04:15:03.488 INFO:tasks.workunit.client.0.smithi110.stdout:1/85: dread d2/fc [0,4194304] 0 2024-09-21T04:15:03.496 INFO:tasks.workunit.client.0.smithi110.stdout:1/86: dread d2/f8 [0,4194304] 0 2024-09-21T04:15:03.496 INFO:tasks.workunit.client.0.smithi110.stdout:0/111: dwrite f0 [0,4194304] 0 2024-09-21T04:15:03.497 INFO:tasks.workunit.client.0.smithi110.stdout:1/87: creat d2/f19 x:0 0 0 2024-09-21T04:15:03.497 INFO:tasks.workunit.client.0.smithi110.stdout:0/112: rmdir d4 39 2024-09-21T04:15:03.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/88: dread d2/f8 [0,4194304] 0 2024-09-21T04:15:03.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/89: write d2/f4 [2042701,3341] 0 2024-09-21T04:15:03.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/90: chown d2/fe 50122 1 2024-09-21T04:15:03.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/91: chown d2/d9/f10 962 1 2024-09-21T04:15:03.504 INFO:tasks.workunit.client.0.smithi110.stdout:1/92: stat d2/d9/c15 0 2024-09-21T04:15:03.507 INFO:tasks.workunit.client.0.smithi110.stdout:0/113: dread d4/fb [0,4194304] 0 2024-09-21T04:15:03.507 INFO:tasks.workunit.client.0.smithi110.stdout:0/114: rename d4/d1c to d4/d1c/d1d 22 2024-09-21T04:15:03.507 INFO:tasks.workunit.client.0.smithi110.stdout:1/93: rmdir d2 39 2024-09-21T04:15:03.509 INFO:tasks.workunit.client.0.smithi110.stdout:0/115: rename d4/dd/d12/c18 to d4/c1e 0 2024-09-21T04:15:03.513 INFO:tasks.workunit.client.0.smithi110.stdout:0/116: truncate d4/f5 433159 0 2024-09-21T04:15:03.514 INFO:tasks.workunit.client.0.smithi110.stdout:0/117: link d4/dd/d10/f13 d4/f1f 0 2024-09-21T04:15:03.521 INFO:tasks.workunit.client.0.smithi110.stdout:0/118: write f2 [988572,10105] 0 2024-09-21T04:15:03.521 INFO:tasks.workunit.client.0.smithi110.stdout:0/119: dread - d4/dd/d10/f1a zero size 2024-09-21T04:15:03.522 INFO:tasks.workunit.client.0.smithi110.stdout:0/120: mknod d4/dd/c20 0 2024-09-21T04:15:03.522 INFO:tasks.workunit.client.0.smithi110.stdout:0/121: chown f0 80797067 1 2024-09-21T04:15:03.523 INFO:tasks.workunit.client.0.smithi110.stdout:0/122: creat d4/f21 x:0 0 0 2024-09-21T04:15:03.524 INFO:tasks.workunit.client.0.smithi110.stdout:0/123: truncate d4/dd/fe 531647 0 2024-09-21T04:15:03.525 INFO:tasks.workunit.client.0.smithi110.stdout:1/94: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:03.526 INFO:tasks.workunit.client.0.smithi110.stdout:0/124: fdatasync f2 0 2024-09-21T04:15:03.526 INFO:tasks.workunit.client.0.smithi110.stdout:0/125: truncate d4/f21 808118 0 2024-09-21T04:15:03.526 INFO:tasks.workunit.client.0.smithi110.stdout:0/126: creat d4/dd/d10/f22 x:0 0 0 2024-09-21T04:15:03.539 INFO:tasks.workunit.client.0.smithi110.stdout:1/95: dwrite d2/f17 [0,4194304] 0 2024-09-21T04:15:03.541 INFO:tasks.workunit.client.0.smithi110.stdout:1/96: creat d2/d9/f1a x:0 0 0 2024-09-21T04:15:03.543 INFO:tasks.workunit.client.0.smithi110.stdout:1/97: unlink d2/d9/f16 0 2024-09-21T04:15:03.543 INFO:tasks.workunit.client.0.smithi110.stdout:1/98: creat d2/d9/f1b x:0 0 0 2024-09-21T04:15:03.545 INFO:tasks.workunit.client.0.smithi110.stdout:1/99: creat d2/f1c x:0 0 0 2024-09-21T04:15:03.550 INFO:tasks.workunit.client.0.smithi110.stdout:1/100: dread d2/f17 [0,4194304] 0 2024-09-21T04:15:03.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:03 smithi148 ceph-mon[34957]: pgmap v266: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 467 op/s 2024-09-21T04:15:03.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:03 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:03 smithi110 ceph-mon[27723]: pgmap v266: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 467 op/s 2024-09-21T04:15:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:03 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:05.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:05 smithi110 ceph-mon[27723]: pgmap v267: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 580 op/s 2024-09-21T04:15:05.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:05 smithi148 ceph-mon[34957]: pgmap v267: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 580 op/s 2024-09-21T04:15:06.243 INFO:tasks.workunit.client.0.smithi110.stdout:4/24: sync 2024-09-21T04:15:06.243 INFO:tasks.workunit.client.0.smithi110.stdout:8/44: sync 2024-09-21T04:15:06.243 INFO:tasks.workunit.client.0.smithi110.stdout:7/6: sync 2024-09-21T04:15:06.243 INFO:tasks.workunit.client.0.smithi110.stdout:0/127: sync 2024-09-21T04:15:06.243 INFO:tasks.workunit.client.0.smithi110.stdout:5/36: sync 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/13: sync 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:9/7: sync 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:4/25: dread - f8 zero size 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:5/37: chown d1/d7/d8/de 0 1 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:2/20: sync 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:1/101: sync 2024-09-21T04:15:06.244 INFO:tasks.workunit.client.0.smithi110.stdout:1/102: chown d2/fe 825 1 2024-09-21T04:15:06.245 INFO:tasks.workunit.client.0.smithi110.stdout:3/33: sync 2024-09-21T04:15:06.245 INFO:tasks.workunit.client.0.smithi110.stdout:8/45: mknod d0/d7/cf 0 2024-09-21T04:15:06.245 INFO:tasks.workunit.client.0.smithi110.stdout:8/46: write d0/fb [908471,35684] 0 2024-09-21T04:15:06.247 INFO:tasks.workunit.client.0.smithi110.stdout:6/14: dread f3 [0,4194304] 0 2024-09-21T04:15:06.247 INFO:tasks.workunit.client.0.smithi110.stdout:0/128: creat d4/dd/d12/f23 x:0 0 0 2024-09-21T04:15:06.247 INFO:tasks.workunit.client.0.smithi110.stdout:7/7: creat f2 x:0 0 0 2024-09-21T04:15:06.248 INFO:tasks.workunit.client.0.smithi110.stdout:1/103: symlink d2/d9/l1d 0 2024-09-21T04:15:06.250 INFO:tasks.workunit.client.0.smithi110.stdout:3/34: rename d0/l1 to d0/lc 0 2024-09-21T04:15:06.250 INFO:tasks.workunit.client.0.smithi110.stdout:3/35: chown d0/fa 2 1 2024-09-21T04:15:06.250 INFO:tasks.workunit.client.0.smithi110.stdout:3/36: fdatasync d0/f6 0 2024-09-21T04:15:06.250 INFO:tasks.workunit.client.0.smithi110.stdout:3/37: write d0/f9 [855961,120964] 0 2024-09-21T04:15:06.251 INFO:tasks.workunit.client.0.smithi110.stdout:8/47: symlink d0/d7/l10 0 2024-09-21T04:15:06.252 INFO:tasks.workunit.client.0.smithi110.stdout:4/26: symlink ld 0 2024-09-21T04:15:06.254 INFO:tasks.workunit.client.0.smithi110.stdout:7/8: rename l1 to l3 0 2024-09-21T04:15:06.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/104: rename d2/d9/c15 to d2/d9/c1e 0 2024-09-21T04:15:06.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/105: write d2/fc [2965780,104446] 0 2024-09-21T04:15:06.256 INFO:tasks.workunit.client.0.smithi110.stdout:3/38: symlink d0/ld 0 2024-09-21T04:15:06.257 INFO:tasks.workunit.client.0.smithi110.stdout:9/8: write f2 [1498487,107080] 0 2024-09-21T04:15:06.258 INFO:tasks.workunit.client.0.smithi110.stdout:0/129: dread d4/fb [0,4194304] 0 2024-09-21T04:15:06.260 INFO:tasks.workunit.client.0.smithi110.stdout:4/27: symlink le 0 2024-09-21T04:15:06.261 INFO:tasks.workunit.client.0.smithi110.stdout:0/130: truncate d4/dd/fe 278286 0 2024-09-21T04:15:06.264 INFO:tasks.workunit.client.0.smithi110.stdout:0/131: unlink d4/dd/d10/c15 0 2024-09-21T04:15:06.265 INFO:tasks.workunit.client.0.smithi110.stdout:4/28: rename c7 to cf 0 2024-09-21T04:15:06.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/29: symlink l10 0 2024-09-21T04:15:06.268 INFO:tasks.workunit.client.0.smithi110.stdout:6/15: dwrite f3 [0,4194304] 0 2024-09-21T04:15:06.269 INFO:tasks.workunit.client.0.smithi110.stdout:6/16: readlink - no filename 2024-09-21T04:15:06.269 INFO:tasks.workunit.client.0.smithi110.stdout:6/17: fdatasync f3 0 2024-09-21T04:15:06.270 INFO:tasks.workunit.client.0.smithi110.stdout:4/30: rename ld to l11 0 2024-09-21T04:15:06.271 INFO:tasks.workunit.client.0.smithi110.stdout:6/18: mkdir d4 0 2024-09-21T04:15:06.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/106: dread d2/f8 [0,4194304] 0 2024-09-21T04:15:06.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/107: creat d2/d9/f1f x:0 0 0 2024-09-21T04:15:06.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/108: write d2/d9/f1f [382,34380] 0 2024-09-21T04:15:06.272 INFO:tasks.workunit.client.0.smithi110.stdout:4/31: creat f12 x:0 0 0 2024-09-21T04:15:06.272 INFO:tasks.workunit.client.0.smithi110.stdout:5/38: dwrite d1/f6 [0,4194304] 0 2024-09-21T04:15:06.273 INFO:tasks.workunit.client.0.smithi110.stdout:9/9: dread f2 [0,4194304] 0 2024-09-21T04:15:06.273 INFO:tasks.workunit.client.0.smithi110.stdout:2/21: dwrite d2/f4 [0,4194304] 0 2024-09-21T04:15:06.273 INFO:tasks.workunit.client.0.smithi110.stdout:2/22: fdatasync d2/f4 0 2024-09-21T04:15:06.274 INFO:tasks.workunit.client.0.smithi110.stdout:4/32: fdatasync f5 0 2024-09-21T04:15:06.278 INFO:tasks.workunit.client.0.smithi110.stdout:5/39: unlink d1/dc/c5 0 2024-09-21T04:15:06.280 INFO:tasks.workunit.client.0.smithi110.stdout:1/109: dread d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:06.280 INFO:tasks.workunit.client.0.smithi110.stdout:1/110: fdatasync d2/d9/f1b 0 2024-09-21T04:15:06.281 INFO:tasks.workunit.client.0.smithi110.stdout:1/111: write d2/f4 [1315473,61928] 0 2024-09-21T04:15:06.281 INFO:tasks.workunit.client.0.smithi110.stdout:1/112: fdatasync d2/fe 0 2024-09-21T04:15:06.281 INFO:tasks.workunit.client.0.smithi110.stdout:1/113: dread - d2/d9/f1a zero size 2024-09-21T04:15:06.283 INFO:tasks.workunit.client.0.smithi110.stdout:6/19: write f3 [2970871,53755] 0 2024-09-21T04:15:06.283 INFO:tasks.workunit.client.0.smithi110.stdout:2/23: dread d2/f4 [0,4194304] 0 2024-09-21T04:15:06.283 INFO:tasks.workunit.client.0.smithi110.stdout:2/24: truncate d2/f4 4825601 0 2024-09-21T04:15:06.284 INFO:tasks.workunit.client.0.smithi110.stdout:7/9: rename l3 to l4 0 2024-09-21T04:15:06.285 INFO:tasks.workunit.client.0.smithi110.stdout:7/10: truncate f2 794909 0 2024-09-21T04:15:06.285 INFO:tasks.workunit.client.0.smithi110.stdout:8/48: dread d0/fb [0,4194304] 0 2024-09-21T04:15:06.285 INFO:tasks.workunit.client.0.smithi110.stdout:8/49: write d0/fb [1498925,29757] 0 2024-09-21T04:15:06.285 INFO:tasks.workunit.client.0.smithi110.stdout:8/50: stat d0/d7/l8 0 2024-09-21T04:15:06.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/51: write d0/fb [1021976,126166] 0 2024-09-21T04:15:06.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/52: creat d0/f11 x:0 0 0 2024-09-21T04:15:06.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/53: write d0/f11 [14967,116883] 0 2024-09-21T04:15:06.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/54: write d0/f11 [508572,61698] 0 2024-09-21T04:15:06.293 INFO:tasks.workunit.client.0.smithi110.stdout:4/33: rename c0 to c13 0 2024-09-21T04:15:06.295 INFO:tasks.workunit.client.0.smithi110.stdout:8/55: read d0/d7/fd [1593446,29535] 0 2024-09-21T04:15:06.295 INFO:tasks.workunit.client.0.smithi110.stdout:8/56: readlink d0/d7/lc 0 2024-09-21T04:15:06.295 INFO:tasks.workunit.client.0.smithi110.stdout:3/39: dwrite d0/f7 [0,4194304] 0 2024-09-21T04:15:06.295 INFO:tasks.workunit.client.0.smithi110.stdout:3/40: creat d0/fe x:0 0 0 2024-09-21T04:15:06.295 INFO:tasks.workunit.client.0.smithi110.stdout:3/41: chown d0/c8 71 1 2024-09-21T04:15:06.296 INFO:tasks.workunit.client.0.smithi110.stdout:4/34: creat f14 x:0 0 0 2024-09-21T04:15:06.296 INFO:tasks.workunit.client.0.smithi110.stdout:4/35: write f12 [153162,53344] 0 2024-09-21T04:15:06.298 INFO:tasks.workunit.client.0.smithi110.stdout:0/132: dwrite d4/f11 [0,4194304] 0 2024-09-21T04:15:06.298 INFO:tasks.workunit.client.0.smithi110.stdout:1/114: read f1 [2929182,31403] 0 2024-09-21T04:15:06.305 INFO:tasks.workunit.client.0.smithi110.stdout:9/10: dwrite f2 [0,4194304] 0 2024-09-21T04:15:06.305 INFO:tasks.workunit.client.0.smithi110.stdout:9/11: creat f3 x:0 0 0 2024-09-21T04:15:06.305 INFO:tasks.workunit.client.0.smithi110.stdout:5/40: dwrite d1/d7/d8/fa [0,4194304] 0 2024-09-21T04:15:06.307 INFO:tasks.workunit.client.0.smithi110.stdout:8/57: rename d0/d7/l9 to d0/l12 0 2024-09-21T04:15:06.311 INFO:tasks.workunit.client.0.smithi110.stdout:0/133: creat d4/d1c/f24 x:0 0 0 2024-09-21T04:15:06.314 INFO:tasks.workunit.client.0.smithi110.stdout:5/41: write d1/f6 [4738686,81951] 0 2024-09-21T04:15:06.319 INFO:tasks.workunit.client.0.smithi110.stdout:8/58: mkdir d0/d7/d13 0 2024-09-21T04:15:06.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/11: truncate f2 419350 0 2024-09-21T04:15:06.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/12: creat f5 x:0 0 0 2024-09-21T04:15:06.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/13: creat f6 x:0 0 0 2024-09-21T04:15:06.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/14: dread - f6 zero size 2024-09-21T04:15:06.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/15: dread - f5 zero size 2024-09-21T04:15:06.323 INFO:tasks.workunit.client.0.smithi110.stdout:0/134: creat d4/d1c/f25 x:0 0 0 2024-09-21T04:15:06.326 INFO:tasks.workunit.client.0.smithi110.stdout:4/36: link c13 c15 0 2024-09-21T04:15:06.326 INFO:tasks.workunit.client.0.smithi110.stdout:4/37: stat f4 0 2024-09-21T04:15:06.328 INFO:tasks.workunit.client.0.smithi110.stdout:7/16: rename l4 to l7 0 2024-09-21T04:15:06.329 INFO:tasks.workunit.client.0.smithi110.stdout:3/42: dwrite d0/f9 [0,4194304] 0 2024-09-21T04:15:06.330 INFO:tasks.workunit.client.0.smithi110.stdout:5/42: write d1/d7/d8/fa [3782134,15739] 0 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/115: dwrite d2/f19 [0,4194304] 0 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/116: truncate d2/d9/f1b 331479 0 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/117: chown d2/d9/f1a 2 1 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:6/20: dwrite f3 [4194304,4194304] 0 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:4/38: fsync f3 0 2024-09-21T04:15:06.331 INFO:tasks.workunit.client.0.smithi110.stdout:4/39: write fb [269739,29820] 0 2024-09-21T04:15:06.338 INFO:tasks.workunit.client.0.smithi110.stdout:5/43: creat d1/dc/ff x:0 0 0 2024-09-21T04:15:06.343 INFO:tasks.workunit.client.0.smithi110.stdout:6/21: creat d4/f5 x:0 0 0 2024-09-21T04:15:06.343 INFO:tasks.workunit.client.0.smithi110.stdout:6/22: creat d4/f6 x:0 0 0 2024-09-21T04:15:06.346 INFO:tasks.workunit.client.0.smithi110.stdout:1/118: rmdir d2/d9 39 2024-09-21T04:15:06.346 INFO:tasks.workunit.client.0.smithi110.stdout:1/119: stat d2/f4 0 2024-09-21T04:15:06.349 INFO:tasks.workunit.client.0.smithi110.stdout:4/40: symlink l16 0 2024-09-21T04:15:06.349 INFO:tasks.workunit.client.0.smithi110.stdout:4/41: truncate f8 998392 0 2024-09-21T04:15:06.349 INFO:tasks.workunit.client.0.smithi110.stdout:3/43: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:3/44: dread - d0/fe zero size 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:8/59: dwrite d0/fb [0,4194304] 0 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:8/60: fdatasync d0/d7/fa 0 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:4/42: read fa [247193,128023] 0 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:4/43: rmdir - no directory 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:1/120: symlink d2/d9/l20 0 2024-09-21T04:15:06.350 INFO:tasks.workunit.client.0.smithi110.stdout:1/121: chown d2/f4 1 1 2024-09-21T04:15:06.354 INFO:tasks.workunit.client.0.smithi110.stdout:3/45: symlink d0/lf 0 2024-09-21T04:15:06.354 INFO:tasks.workunit.client.0.smithi110.stdout:3/46: creat d0/f10 x:0 0 0 2024-09-21T04:15:06.354 INFO:tasks.workunit.client.0.smithi110.stdout:3/47: truncate d0/fa 255703 0 2024-09-21T04:15:06.354 INFO:tasks.workunit.client.0.smithi110.stdout:3/48: read - d0/f6 zero size 2024-09-21T04:15:06.354 INFO:tasks.workunit.client.0.smithi110.stdout:8/61: mknod d0/d7/d13/c14 0 2024-09-21T04:15:06.355 INFO:tasks.workunit.client.0.smithi110.stdout:1/122: creat d2/d9/f21 x:0 0 0 2024-09-21T04:15:06.356 INFO:tasks.workunit.client.0.smithi110.stdout:0/135: dwrite d4/dd/d10/f1a [0,4194304] 0 2024-09-21T04:15:06.358 INFO:tasks.workunit.client.0.smithi110.stdout:8/62: link d0/d7/l8 d0/l15 0 2024-09-21T04:15:06.358 INFO:tasks.workunit.client.0.smithi110.stdout:8/63: truncate d0/f11 920183 0 2024-09-21T04:15:06.358 INFO:tasks.workunit.client.0.smithi110.stdout:8/64: fsync d0/fb 0 2024-09-21T04:15:06.358 INFO:tasks.workunit.client.0.smithi110.stdout:1/123: mknod d2/d9/c22 0 2024-09-21T04:15:06.358 INFO:tasks.workunit.client.0.smithi110.stdout:1/124: chown d2/fc 1 1 2024-09-21T04:15:06.359 INFO:tasks.workunit.client.0.smithi110.stdout:1/125: readlink d2/l18 0 2024-09-21T04:15:06.360 INFO:tasks.workunit.client.0.smithi110.stdout:7/17: write f2 [292457,51997] 0 2024-09-21T04:15:06.360 INFO:tasks.workunit.client.0.smithi110.stdout:7/18: write f6 [647464,48325] 0 2024-09-21T04:15:06.360 INFO:tasks.workunit.client.0.smithi110.stdout:7/19: chown f6 15626246 1 2024-09-21T04:15:06.360 INFO:tasks.workunit.client.0.smithi110.stdout:7/20: dread - f5 zero size 2024-09-21T04:15:06.361 INFO:tasks.workunit.client.0.smithi110.stdout:0/136: symlink d4/dd/d10/l26 0 2024-09-21T04:15:06.362 INFO:tasks.workunit.client.0.smithi110.stdout:8/65: mkdir d0/d7/d13/d16 0 2024-09-21T04:15:06.363 INFO:tasks.workunit.client.0.smithi110.stdout:1/126: mkdir d2/d9/d23 0 2024-09-21T04:15:06.363 INFO:tasks.workunit.client.0.smithi110.stdout:1/127: write d2/d9/f10 [3825967,117721] 0 2024-09-21T04:15:06.364 INFO:tasks.workunit.client.0.smithi110.stdout:5/44: dwrite d1/dc/ff [0,4194304] 0 2024-09-21T04:15:06.364 INFO:tasks.workunit.client.0.smithi110.stdout:0/137: chown d4/f5 7850598 1 2024-09-21T04:15:06.364 INFO:tasks.workunit.client.0.smithi110.stdout:0/138: chown d4/d1c/f24 5 1 2024-09-21T04:15:06.365 INFO:tasks.workunit.client.0.smithi110.stdout:6/23: dwrite f3 [8388608,4194304] 0 2024-09-21T04:15:06.366 INFO:tasks.workunit.client.0.smithi110.stdout:8/66: link d0/fb d0/d7/d13/d16/f17 0 2024-09-21T04:15:06.366 INFO:tasks.workunit.client.0.smithi110.stdout:4/44: dread fb [0,4194304] 0 2024-09-21T04:15:06.366 INFO:tasks.workunit.client.0.smithi110.stdout:4/45: chown cc 564070882 1 2024-09-21T04:15:06.366 INFO:tasks.workunit.client.0.smithi110.stdout:4/46: chown l10 156696862 1 2024-09-21T04:15:06.366 INFO:tasks.workunit.client.0.smithi110.stdout:1/128: creat d2/d9/d23/f24 x:0 0 0 2024-09-21T04:15:06.368 INFO:tasks.workunit.client.0.smithi110.stdout:0/139: dread d4/dd/fe [0,4194304] 0 2024-09-21T04:15:06.368 INFO:tasks.workunit.client.0.smithi110.stdout:0/140: creat d4/dd/d12/f27 x:0 0 0 2024-09-21T04:15:06.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/67: unlink d0/d7/l8 0 2024-09-21T04:15:06.371 INFO:tasks.workunit.client.0.smithi110.stdout:1/129: mknod d2/d9/d23/c25 0 2024-09-21T04:15:06.372 INFO:tasks.workunit.client.0.smithi110.stdout:7/21: dread f6 [0,4194304] 0 2024-09-21T04:15:06.373 INFO:tasks.workunit.client.0.smithi110.stdout:0/141: rename f0 to d4/dd/d12/f28 0 2024-09-21T04:15:06.374 INFO:tasks.workunit.client.0.smithi110.stdout:4/47: mkdir d17 0 2024-09-21T04:15:06.375 INFO:tasks.workunit.client.0.smithi110.stdout:5/45: dread d1/d7/d8/fa [0,4194304] 0 2024-09-21T04:15:06.375 INFO:tasks.workunit.client.0.smithi110.stdout:5/46: chown d1/d7/d8/de 6934 1 2024-09-21T04:15:06.375 INFO:tasks.workunit.client.0.smithi110.stdout:1/130: mkdir d2/d9/d26 0 2024-09-21T04:15:06.376 INFO:tasks.workunit.client.0.smithi110.stdout:7/22: rename l7 to l8 0 2024-09-21T04:15:06.376 INFO:tasks.workunit.client.0.smithi110.stdout:7/23: readlink l8 0 2024-09-21T04:15:06.376 INFO:tasks.workunit.client.0.smithi110.stdout:7/24: fdatasync f6 0 2024-09-21T04:15:06.376 INFO:tasks.workunit.client.0.smithi110.stdout:7/25: truncate f2 618369 0 2024-09-21T04:15:06.377 INFO:tasks.workunit.client.0.smithi110.stdout:4/48: symlink d17/l18 0 2024-09-21T04:15:06.379 INFO:tasks.workunit.client.0.smithi110.stdout:1/131: mknod d2/d9/c27 0 2024-09-21T04:15:06.379 INFO:tasks.workunit.client.0.smithi110.stdout:7/26: symlink l9 0 2024-09-21T04:15:06.380 INFO:tasks.workunit.client.0.smithi110.stdout:3/49: getdents d0 0 2024-09-21T04:15:06.380 INFO:tasks.workunit.client.0.smithi110.stdout:3/50: write d0/f6 [533878,95127] 0 2024-09-21T04:15:06.381 INFO:tasks.workunit.client.0.smithi110.stdout:6/24: dread f3 [8388608,4194304] 0 2024-09-21T04:15:06.381 INFO:tasks.workunit.client.0.smithi110.stdout:6/25: creat d4/f7 x:0 0 0 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/49: symlink d17/l19 0 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/50: write f12 [403060,63123] 0 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/51: readlink le 0 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/52: readlink l11 0 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/53: dread - f14 zero size 2024-09-21T04:15:06.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/54: write f3 [75349,15383] 0 2024-09-21T04:15:06.386 INFO:tasks.workunit.client.0.smithi110.stdout:1/132: mknod d2/d9/d26/c28 0 2024-09-21T04:15:06.386 INFO:tasks.workunit.client.0.smithi110.stdout:1/133: write d2/d9/f21 [982126,106382] 0 2024-09-21T04:15:06.386 INFO:tasks.workunit.client.0.smithi110.stdout:1/134: chown d2/d9/f21 195077 1 2024-09-21T04:15:06.387 INFO:tasks.workunit.client.0.smithi110.stdout:7/27: unlink f6 0 2024-09-21T04:15:06.387 INFO:tasks.workunit.client.0.smithi110.stdout:7/28: chown l8 594053107 1 2024-09-21T04:15:06.388 INFO:tasks.workunit.client.0.smithi110.stdout:7/29: write f5 [970682,43545] 0 2024-09-21T04:15:06.391 INFO:tasks.workunit.client.0.smithi110.stdout:4/55: creat d17/f1a x:0 0 0 2024-09-21T04:15:06.392 INFO:tasks.workunit.client.0.smithi110.stdout:8/68: rename d0/d7/d13/d16 to d0/d7/d18 0 2024-09-21T04:15:06.395 INFO:tasks.workunit.client.0.smithi110.stdout:0/142: dwrite d4/dd/fe [0,4194304] 0 2024-09-21T04:15:06.396 INFO:tasks.workunit.client.0.smithi110.stdout:1/135: rename d2/f19 to d2/d9/d23/f29 0 2024-09-21T04:15:06.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/143: mkdir d4/dd/d10/d29 0 2024-09-21T04:15:06.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/144: truncate d4/dd/d10/f22 150302 0 2024-09-21T04:15:06.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/145: getdents d4/dd 0 2024-09-21T04:15:06.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/146: creat d4/dd/d10/f2a x:0 0 0 2024-09-21T04:15:06.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/147: write d4/dd/d10/f2a [961966,83554] 0 2024-09-21T04:15:06.400 INFO:tasks.workunit.client.0.smithi110.stdout:3/51: dwrite d0/f9 [0,4194304] 0 2024-09-21T04:15:06.405 INFO:tasks.workunit.client.0.smithi110.stdout:1/136: dread d2/d9/f21 [0,4194304] 0 2024-09-21T04:15:06.406 INFO:tasks.workunit.client.0.smithi110.stdout:0/148: dread d4/dd/d10/f1a [0,4194304] 0 2024-09-21T04:15:06.406 INFO:tasks.workunit.client.0.smithi110.stdout:0/149: chown d4/dd/d12/f28 117989 1 2024-09-21T04:15:06.408 INFO:tasks.workunit.client.0.smithi110.stdout:1/137: symlink d2/d9/d26/l2a 0 2024-09-21T04:15:06.408 INFO:tasks.workunit.client.0.smithi110.stdout:1/138: write d2/d9/f1a [429791,30748] 0 2024-09-21T04:15:06.408 INFO:tasks.workunit.client.0.smithi110.stdout:3/52: dread d0/f9 [0,4194304] 0 2024-09-21T04:15:06.409 INFO:tasks.workunit.client.0.smithi110.stdout:1/139: dread d2/d9/f1b [0,4194304] 0 2024-09-21T04:15:06.409 INFO:tasks.workunit.client.0.smithi110.stdout:1/140: creat d2/f2b x:0 0 0 2024-09-21T04:15:06.409 INFO:tasks.workunit.client.0.smithi110.stdout:1/141: dread - d2/d9/d23/f24 zero size 2024-09-21T04:15:06.410 INFO:tasks.workunit.client.0.smithi110.stdout:0/150: symlink d4/dd/d10/d29/l2b 0 2024-09-21T04:15:06.410 INFO:tasks.workunit.client.0.smithi110.stdout:0/151: fsync d4/dd/d12/f28 0 2024-09-21T04:15:06.412 INFO:tasks.workunit.client.0.smithi110.stdout:3/53: unlink d0/lf 0 2024-09-21T04:15:06.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/56: read f4 [2993554,69519] 0 2024-09-21T04:15:06.414 INFO:tasks.workunit.client.0.smithi110.stdout:8/69: dwrite d0/d7/d18/f17 [4194304,4194304] 0 2024-09-21T04:15:06.416 INFO:tasks.workunit.client.0.smithi110.stdout:1/142: symlink d2/d9/d23/l2c 0 2024-09-21T04:15:06.416 INFO:tasks.workunit.client.0.smithi110.stdout:1/143: chown d2/d9/d23/c25 5560 1 2024-09-21T04:15:06.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/152: chown d4/cc 41 1 2024-09-21T04:15:06.418 INFO:tasks.workunit.client.0.smithi110.stdout:3/54: creat d0/f11 x:0 0 0 2024-09-21T04:15:06.418 INFO:tasks.workunit.client.0.smithi110.stdout:5/47: truncate d1/f6 3579018 0 2024-09-21T04:15:06.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/57: unlink d17/l18 0 2024-09-21T04:15:06.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/58: creat d17/f1b x:0 0 0 2024-09-21T04:15:06.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/59: read - f14 zero size 2024-09-21T04:15:06.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/60: chown f12 12 1 2024-09-21T04:15:06.421 INFO:tasks.workunit.client.0.smithi110.stdout:7/30: dwrite f5 [0,4194304] 0 2024-09-21T04:15:06.422 INFO:tasks.workunit.client.0.smithi110.stdout:8/70: write d0/f11 [1943837,35558] 0 2024-09-21T04:15:06.423 INFO:tasks.workunit.client.0.smithi110.stdout:1/144: unlink d2/d9/l14 0 2024-09-21T04:15:06.424 INFO:tasks.workunit.client.0.smithi110.stdout:0/153: unlink d4/dd/d12/f28 0 2024-09-21T04:15:06.425 INFO:tasks.workunit.client.0.smithi110.stdout:3/55: truncate d0/f7 4160786 0 2024-09-21T04:15:06.428 INFO:tasks.workunit.client.0.smithi110.stdout:4/61: mkdir d17/d1c 0 2024-09-21T04:15:06.433 INFO:tasks.workunit.client.0.smithi110.stdout:1/145: rename d2/d9/f21 to d2/d9/d26/f2d 0 2024-09-21T04:15:06.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/154: mkdir d4/dd/d12/d2c 0 2024-09-21T04:15:06.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/155: write d4/dd/d10/f22 [899558,12693] 0 2024-09-21T04:15:06.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/156: getdents d4/dd/d12/d2c 0 2024-09-21T04:15:06.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/157: readlink d4/dd/d10/l26 0 2024-09-21T04:15:06.435 INFO:tasks.workunit.client.0.smithi110.stdout:0/158: creat d4/dd/d10/f2d x:0 0 0 2024-09-21T04:15:06.439 INFO:tasks.workunit.client.0.smithi110.stdout:8/71: write d0/d7/d18/f17 [3640578,30514] 0 2024-09-21T04:15:06.440 INFO:tasks.workunit.client.0.smithi110.stdout:1/146: rename d2/f1c to d2/f2e 0 2024-09-21T04:15:06.440 INFO:tasks.workunit.client.0.smithi110.stdout:1/147: chown d2/d9/c1e 337 1 2024-09-21T04:15:06.442 INFO:tasks.workunit.client.0.smithi110.stdout:0/159: mkdir d4/dd/d10/d29/d2e 0 2024-09-21T04:15:06.446 INFO:tasks.workunit.client.0.smithi110.stdout:8/72: rmdir d0/d7/d13 39 2024-09-21T04:15:06.447 INFO:tasks.workunit.client.0.smithi110.stdout:1/148: symlink d2/l2f 0 2024-09-21T04:15:06.448 INFO:tasks.workunit.client.0.smithi110.stdout:3/56: rmdir d0 39 2024-09-21T04:15:06.448 INFO:tasks.workunit.client.0.smithi110.stdout:0/160: chown d4/fb 20872 1 2024-09-21T04:15:06.449 INFO:tasks.workunit.client.0.smithi110.stdout:1/149: rmdir d2/d9/d26 39 2024-09-21T04:15:06.450 INFO:tasks.workunit.client.0.smithi110.stdout:4/62: getdents d17 0 2024-09-21T04:15:06.450 INFO:tasks.workunit.client.0.smithi110.stdout:4/63: write f4 [1582062,111639] 0 2024-09-21T04:15:06.451 INFO:tasks.workunit.client.0.smithi110.stdout:3/57: mkdir d0/d12 0 2024-09-21T04:15:06.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/161: symlink d4/dd/d10/l2f 0 2024-09-21T04:15:06.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/73: write d0/d7/d18/f17 [5976834,51253] 0 2024-09-21T04:15:06.460 INFO:tasks.workunit.client.0.smithi110.stdout:8/74: mknod d0/d7/c19 0 2024-09-21T04:15:06.460 INFO:tasks.workunit.client.0.smithi110.stdout:8/75: readlink d0/d7/lc 0 2024-09-21T04:15:06.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/58: write d0/f7 [3611530,3564] 0 2024-09-21T04:15:06.461 INFO:tasks.workunit.client.0.smithi110.stdout:5/48: truncate d1/d7/d8/fa 1247511 0 2024-09-21T04:15:06.463 INFO:tasks.workunit.client.0.smithi110.stdout:3/59: unlink d0/fa 0 2024-09-21T04:15:06.465 INFO:tasks.workunit.client.0.smithi110.stdout:5/49: getdents d1/d7 0 2024-09-21T04:15:06.466 INFO:tasks.workunit.client.0.smithi110.stdout:5/50: readlink - no filename 2024-09-21T04:15:06.466 INFO:tasks.workunit.client.0.smithi110.stdout:5/51: truncate d1/f6 1174447 0 2024-09-21T04:15:06.467 INFO:tasks.workunit.client.0.smithi110.stdout:0/162: read f2 [4785870,2158] 0 2024-09-21T04:15:06.467 INFO:tasks.workunit.client.0.smithi110.stdout:0/163: truncate d4/dd/d10/f2d 324449 0 2024-09-21T04:15:06.468 INFO:tasks.workunit.client.0.smithi110.stdout:0/164: mknod d4/dd/c30 0 2024-09-21T04:15:06.469 INFO:tasks.workunit.client.0.smithi110.stdout:4/64: truncate fa 837148 0 2024-09-21T04:15:06.470 INFO:tasks.workunit.client.0.smithi110.stdout:4/65: rmdir d17/d1c 0 2024-09-21T04:15:06.478 INFO:tasks.workunit.client.0.smithi110.stdout:7/31: dwrite f5 [4194304,4194304] 0 2024-09-21T04:15:06.478 INFO:tasks.workunit.client.0.smithi110.stdout:8/76: fsync d0/fb 0 2024-09-21T04:15:06.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/77: rename d0/d7/fa to d0/d7/f1a 0 2024-09-21T04:15:06.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/78: rmdir d0/d7/d18 39 2024-09-21T04:15:06.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/79: write d0/d7/f1a [279851,56120] 0 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/32: dread f2 [0,4194304] 0 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/33: creat fa x:0 0 0 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/34: fdatasync f5 0 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/35: stat l9 0 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/36: chown fa 39543690 1 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/37: dread - fa zero size 2024-09-21T04:15:06.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/38: chown l9 0 1 2024-09-21T04:15:06.483 INFO:tasks.workunit.client.0.smithi110.stdout:8/80: creat d0/d7/f1b x:0 0 0 2024-09-21T04:15:06.483 INFO:tasks.workunit.client.0.smithi110.stdout:8/81: dread - d0/d7/f1b zero size 2024-09-21T04:15:06.483 INFO:tasks.workunit.client.0.smithi110.stdout:7/39: unlink f5 0 2024-09-21T04:15:06.484 INFO:tasks.workunit.client.0.smithi110.stdout:8/82: mkdir d0/d7/d13/d1c 0 2024-09-21T04:15:06.485 INFO:tasks.workunit.client.0.smithi110.stdout:1/150: dwrite d2/d9/d23/f29 [0,4194304] 0 2024-09-21T04:15:06.485 INFO:tasks.workunit.client.0.smithi110.stdout:7/40: dread f2 [0,4194304] 0 2024-09-21T04:15:06.485 INFO:tasks.workunit.client.0.smithi110.stdout:5/52: write d1/dc/ff [3918758,94341] 0 2024-09-21T04:15:06.486 INFO:tasks.workunit.client.0.smithi110.stdout:7/41: write f2 [1491442,49461] 0 2024-09-21T04:15:06.486 INFO:tasks.workunit.client.0.smithi110.stdout:5/53: chown d1/d7/d8 0 1 2024-09-21T04:15:06.486 INFO:tasks.workunit.client.0.smithi110.stdout:1/151: dread d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:06.486 INFO:tasks.workunit.client.0.smithi110.stdout:1/152: readlink d2/d9/l20 0 2024-09-21T04:15:06.486 INFO:tasks.workunit.client.0.smithi110.stdout:1/153: rename d2 to d2/d9/d26/d30 22 2024-09-21T04:15:06.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/42: link l8 lb 0 2024-09-21T04:15:06.490 INFO:tasks.workunit.client.0.smithi110.stdout:1/154: creat d2/d9/d26/f31 x:0 0 0 2024-09-21T04:15:06.492 INFO:tasks.workunit.client.0.smithi110.stdout:5/54: rmdir d1 39 2024-09-21T04:15:06.494 INFO:tasks.workunit.client.0.smithi110.stdout:3/60: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:06.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/165: dwrite f2 [0,4194304] 0 2024-09-21T04:15:06.494 INFO:tasks.workunit.client.0.smithi110.stdout:7/43: mknod cc 0 2024-09-21T04:15:06.494 INFO:tasks.workunit.client.0.smithi110.stdout:7/44: truncate fa 347838 0 2024-09-21T04:15:06.495 INFO:tasks.workunit.client.0.smithi110.stdout:1/155: mknod d2/d9/c32 0 2024-09-21T04:15:06.495 INFO:tasks.workunit.client.0.smithi110.stdout:1/156: chown d2/f4 29229 1 2024-09-21T04:15:06.497 INFO:tasks.workunit.client.0.smithi110.stdout:5/55: symlink d1/d7/l10 0 2024-09-21T04:15:06.502 INFO:tasks.workunit.client.0.smithi110.stdout:7/45: unlink fa 0 2024-09-21T04:15:06.502 INFO:tasks.workunit.client.0.smithi110.stdout:7/46: write f2 [2027445,41726] 0 2024-09-21T04:15:06.508 INFO:tasks.workunit.client.0.smithi110.stdout:3/61: dread d0/f6 [0,4194304] 0 2024-09-21T04:15:06.509 INFO:tasks.workunit.client.0.smithi110.stdout:3/62: creat d0/d12/f13 x:0 0 0 2024-09-21T04:15:06.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/63: read d0/f9 [413583,33167] 0 2024-09-21T04:15:06.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/64: chown d0/f6 32416 1 2024-09-21T04:15:06.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/65: stat d0/d12/f13 0 2024-09-21T04:15:06.514 INFO:tasks.workunit.client.0.smithi110.stdout:8/83: dwrite d0/d7/d18/f17 [4194304,4194304] 0 2024-09-21T04:15:06.514 INFO:tasks.workunit.client.0.smithi110.stdout:8/84: truncate d0/d7/f1a 1211151 0 2024-09-21T04:15:06.515 INFO:tasks.workunit.client.0.smithi110.stdout:4/66: dwrite f3 [0,4194304] 0 2024-09-21T04:15:06.516 INFO:tasks.workunit.client.0.smithi110.stdout:8/85: creat d0/d7/d13/d1c/f1d x:0 0 0 2024-09-21T04:15:06.517 INFO:tasks.workunit.client.0.smithi110.stdout:0/166: dwrite d4/d1c/f25 [0,4194304] 0 2024-09-21T04:15:06.520 INFO:tasks.workunit.client.0.smithi110.stdout:4/67: mkdir d17/d1d 0 2024-09-21T04:15:06.520 INFO:tasks.workunit.client.0.smithi110.stdout:4/68: stat d17/l19 0 2024-09-21T04:15:06.520 INFO:tasks.workunit.client.0.smithi110.stdout:4/69: dread - d17/f1a zero size 2024-09-21T04:15:06.526 INFO:tasks.workunit.client.0.smithi110.stdout:1/157: fsync d2/d9/d26/f31 0 2024-09-21T04:15:06.526 INFO:tasks.workunit.client.0.smithi110.stdout:1/158: readlink d2/d9/l20 0 2024-09-21T04:15:06.526 INFO:tasks.workunit.client.0.smithi110.stdout:1/159: write d2/d9/d26/f31 [722191,30126] 0 2024-09-21T04:15:06.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/70: unlink f12 0 2024-09-21T04:15:06.531 INFO:tasks.workunit.client.0.smithi110.stdout:1/160: unlink d2/d9/c1e 0 2024-09-21T04:15:06.531 INFO:tasks.workunit.client.0.smithi110.stdout:1/161: fdatasync d2/f2b 0 2024-09-21T04:15:06.532 INFO:tasks.workunit.client.0.smithi110.stdout:3/66: dwrite d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:06.532 INFO:tasks.workunit.client.0.smithi110.stdout:4/71: mkdir d17/d1e 0 2024-09-21T04:15:06.535 INFO:tasks.workunit.client.0.smithi110.stdout:1/162: read d2/fe [926533,80718] 0 2024-09-21T04:15:06.536 INFO:tasks.workunit.client.0.smithi110.stdout:3/67: mknod d0/c14 0 2024-09-21T04:15:06.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/163: write d2/d9/d26/f2d [184469,9402] 0 2024-09-21T04:15:06.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/72: creat d17/d1e/f1f x:0 0 0 2024-09-21T04:15:06.538 INFO:tasks.workunit.client.0.smithi110.stdout:4/73: truncate fa 902162 0 2024-09-21T04:15:06.540 INFO:tasks.workunit.client.0.smithi110.stdout:1/164: rename d2/d9/d23/f24 to d2/f33 0 2024-09-21T04:15:06.547 INFO:tasks.workunit.client.0.smithi110.stdout:5/56: dwrite d1/dc/ff [0,4194304] 0 2024-09-21T04:15:06.547 INFO:tasks.workunit.client.0.smithi110.stdout:5/57: write d1/f6 [2192431,12487] 0 2024-09-21T04:15:06.547 INFO:tasks.workunit.client.0.smithi110.stdout:5/58: stat d1 0 2024-09-21T04:15:06.548 INFO:tasks.workunit.client.0.smithi110.stdout:7/47: dwrite f2 [0,4194304] 0 2024-09-21T04:15:06.549 INFO:tasks.workunit.client.0.smithi110.stdout:7/48: unlink lb 0 2024-09-21T04:15:06.551 INFO:tasks.workunit.client.0.smithi110.stdout:0/167: truncate d4/dd/d10/f22 16939 0 2024-09-21T04:15:06.558 INFO:tasks.workunit.client.0.smithi110.stdout:1/165: read d2/f8 [2418252,17093] 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/166: write d2/d9/f1f [1009815,125307] 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/167: creat d2/d9/d26/f34 x:0 0 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/168: stat f1 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/169: chown d2/d9/f1f 46 1 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/170: write d2/d9/f1a [917545,22291] 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/171: fdatasync d2/f2e 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/172: readlink d2/l2f 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/173: stat d2/d9/d23/c25 0 2024-09-21T04:15:06.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/174: stat d2/d9/c32 0 2024-09-21T04:15:06.562 INFO:tasks.workunit.client.0.smithi110.stdout:5/59: write d1/dc/ff [2291270,50784] 0 2024-09-21T04:15:06.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/49: write f2 [544162,118900] 0 2024-09-21T04:15:06.571 INFO:tasks.workunit.client.0.smithi110.stdout:4/74: dwrite fb [0,4194304] 0 2024-09-21T04:15:06.571 INFO:tasks.workunit.client.0.smithi110.stdout:4/75: dread - d17/f1a zero size 2024-09-21T04:15:06.572 INFO:tasks.workunit.client.0.smithi110.stdout:3/68: dwrite d0/fe [0,4194304] 0 2024-09-21T04:15:06.572 INFO:tasks.workunit.client.0.smithi110.stdout:0/168: write d4/f11 [3181742,124586] 0 2024-09-21T04:15:06.572 INFO:tasks.workunit.client.0.smithi110.stdout:7/50: mkdir dd 0 2024-09-21T04:15:06.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/76: mkdir d17/d1d/d20 0 2024-09-21T04:15:06.575 INFO:tasks.workunit.client.0.smithi110.stdout:4/77: dread f8 [0,4194304] 0 2024-09-21T04:15:06.575 INFO:tasks.workunit.client.0.smithi110.stdout:4/78: fsync d17/f1b 0 2024-09-21T04:15:06.579 INFO:tasks.workunit.client.0.smithi110.stdout:3/69: unlink d0/f4 0 2024-09-21T04:15:06.580 INFO:tasks.workunit.client.0.smithi110.stdout:5/60: truncate d1/f6 996622 0 2024-09-21T04:15:06.583 INFO:tasks.workunit.client.0.smithi110.stdout:3/70: mknod d0/d12/c15 0 2024-09-21T04:15:06.584 INFO:tasks.workunit.client.0.smithi110.stdout:5/61: mknod d1/dc/c11 0 2024-09-21T04:15:06.586 INFO:tasks.workunit.client.0.smithi110.stdout:0/169: dread d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:06.586 INFO:tasks.workunit.client.0.smithi110.stdout:0/170: chown d4/d1c 91948836 1 2024-09-21T04:15:06.587 INFO:tasks.workunit.client.0.smithi110.stdout:7/51: truncate f2 774289 0 2024-09-21T04:15:06.587 INFO:tasks.workunit.client.0.smithi110.stdout:7/52: chown f2 119776671 1 2024-09-21T04:15:06.587 INFO:tasks.workunit.client.0.smithi110.stdout:1/175: dwrite d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:06.587 INFO:tasks.workunit.client.0.smithi110.stdout:1/176: stat d2/d9/f1a 0 2024-09-21T04:15:06.590 INFO:tasks.workunit.client.0.smithi110.stdout:1/177: link d2/d9/c22 d2/d9/d26/c35 0 2024-09-21T04:15:06.591 INFO:tasks.workunit.client.0.smithi110.stdout:1/178: mknod d2/d9/d23/c36 0 2024-09-21T04:15:06.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/179: rename d2/d9/d26/c28 to d2/c37 0 2024-09-21T04:15:06.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/180: stat d2/lb 0 2024-09-21T04:15:06.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/181: dread - d2/f2b zero size 2024-09-21T04:15:06.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/182: chown d2/d9 421962162 1 2024-09-21T04:15:06.600 INFO:tasks.workunit.client.0.smithi110.stdout:0/171: dread d4/dd/fe [0,4194304] 0 2024-09-21T04:15:06.601 INFO:tasks.workunit.client.0.smithi110.stdout:0/172: fsync d4/dd/d10/f22 0 2024-09-21T04:15:06.601 INFO:tasks.workunit.client.0.smithi110.stdout:7/53: truncate f2 1004290 0 2024-09-21T04:15:06.601 INFO:tasks.workunit.client.0.smithi110.stdout:4/79: dwrite f3 [0,4194304] 0 2024-09-21T04:15:06.601 INFO:tasks.workunit.client.0.smithi110.stdout:3/71: dwrite d0/f6 [4194304,4194304] 0 2024-09-21T04:15:06.604 INFO:tasks.workunit.client.0.smithi110.stdout:5/62: dread d1/f6 [0,4194304] 0 2024-09-21T04:15:06.604 INFO:tasks.workunit.client.0.smithi110.stdout:0/173: unlink d4/dd/c20 0 2024-09-21T04:15:06.604 INFO:tasks.workunit.client.0.smithi110.stdout:0/174: readlink d4/dd/d10/l26 0 2024-09-21T04:15:06.604 INFO:tasks.workunit.client.0.smithi110.stdout:0/175: write d4/dd/d12/f27 [1040126,71609] 0 2024-09-21T04:15:06.605 INFO:tasks.workunit.client.0.smithi110.stdout:7/54: rename cc to dd/ce 0 2024-09-21T04:15:06.608 INFO:tasks.workunit.client.0.smithi110.stdout:5/63: unlink d1/d7/d8/fa 0 2024-09-21T04:15:06.609 INFO:tasks.workunit.client.0.smithi110.stdout:0/176: creat d4/f31 x:0 0 0 2024-09-21T04:15:06.609 INFO:tasks.workunit.client.0.smithi110.stdout:0/177: fsync d4/dd/d10/f22 0 2024-09-21T04:15:06.609 INFO:tasks.workunit.client.0.smithi110.stdout:1/183: rmdir d2 39 2024-09-21T04:15:06.611 INFO:tasks.workunit.client.0.smithi110.stdout:7/55: creat dd/ff x:0 0 0 2024-09-21T04:15:06.611 INFO:tasks.workunit.client.0.smithi110.stdout:7/56: write dd/ff [793849,57504] 0 2024-09-21T04:15:06.612 INFO:tasks.workunit.client.0.smithi110.stdout:5/64: creat d1/f12 x:0 0 0 2024-09-21T04:15:06.614 INFO:tasks.workunit.client.0.smithi110.stdout:0/178: unlink d4/cc 0 2024-09-21T04:15:06.616 INFO:tasks.workunit.client.0.smithi110.stdout:5/65: getdents d1/dc 0 2024-09-21T04:15:06.616 INFO:tasks.workunit.client.0.smithi110.stdout:5/66: chown d1/d7/l10 242117 1 2024-09-21T04:15:06.617 INFO:tasks.workunit.client.0.smithi110.stdout:1/184: dread f1 [0,4194304] 0 2024-09-21T04:15:06.618 INFO:tasks.workunit.client.0.smithi110.stdout:4/80: dread fb [0,4194304] 0 2024-09-21T04:15:06.619 INFO:tasks.workunit.client.0.smithi110.stdout:1/185: rename d2/d9/f10 to d2/d9/d23/f38 0 2024-09-21T04:15:06.619 INFO:tasks.workunit.client.0.smithi110.stdout:4/81: mkdir d17/d1e/d21 0 2024-09-21T04:15:06.620 INFO:tasks.workunit.client.0.smithi110.stdout:1/186: link d2/d9/c11 d2/d9/d23/c39 0 2024-09-21T04:15:06.621 INFO:tasks.workunit.client.0.smithi110.stdout:3/72: dwrite d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:06.621 INFO:tasks.workunit.client.0.smithi110.stdout:5/67: fsync d1/dc/ff 0 2024-09-21T04:15:06.621 INFO:tasks.workunit.client.0.smithi110.stdout:4/82: mkdir d17/d22 0 2024-09-21T04:15:06.621 INFO:tasks.workunit.client.0.smithi110.stdout:4/83: creat d17/d1e/f23 x:0 0 0 2024-09-21T04:15:06.622 INFO:tasks.workunit.client.0.smithi110.stdout:3/73: mkdir d0/d12/d16 0 2024-09-21T04:15:06.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/68: getdents d1/d7 0 2024-09-21T04:15:06.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/69: stat d1/d7 0 2024-09-21T04:15:06.623 INFO:tasks.workunit.client.0.smithi110.stdout:7/57: rmdir dd 39 2024-09-21T04:15:06.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/187: dread d2/f17 [0,4194304] 0 2024-09-21T04:15:06.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/188: fdatasync d2/d9/d23/f38 0 2024-09-21T04:15:06.625 INFO:tasks.workunit.client.0.smithi110.stdout:4/84: rmdir d17/d22 0 2024-09-21T04:15:06.625 INFO:tasks.workunit.client.0.smithi110.stdout:4/85: dread - d17/d1e/f1f zero size 2024-09-21T04:15:06.626 INFO:tasks.workunit.client.0.smithi110.stdout:4/86: fdatasync f14 0 2024-09-21T04:15:06.627 INFO:tasks.workunit.client.0.smithi110.stdout:3/74: mknod d0/d12/c17 0 2024-09-21T04:15:06.628 INFO:tasks.workunit.client.0.smithi110.stdout:0/179: write d4/d1c/f25 [4048593,74888] 0 2024-09-21T04:15:06.628 INFO:tasks.workunit.client.0.smithi110.stdout:0/180: chown d4/dd/d12/f23 12 1 2024-09-21T04:15:06.628 INFO:tasks.workunit.client.0.smithi110.stdout:5/70: creat d1/d7/f13 x:0 0 0 2024-09-21T04:15:06.630 INFO:tasks.workunit.client.0.smithi110.stdout:7/58: mknod dd/c10 0 2024-09-21T04:15:06.632 INFO:tasks.workunit.client.0.smithi110.stdout:1/189: symlink d2/l3a 0 2024-09-21T04:15:06.633 INFO:tasks.workunit.client.0.smithi110.stdout:0/181: write d4/dd/d10/f1a [1857364,116616] 0 2024-09-21T04:15:06.633 INFO:tasks.workunit.client.0.smithi110.stdout:0/182: write d4/dd/d10/f1a [4234092,122066] 0 2024-09-21T04:15:06.633 INFO:tasks.workunit.client.0.smithi110.stdout:3/75: symlink d0/l18 0 2024-09-21T04:15:06.634 INFO:tasks.workunit.client.0.smithi110.stdout:7/59: fdatasync dd/ff 0 2024-09-21T04:15:06.635 INFO:tasks.workunit.client.0.smithi110.stdout:1/190: mkdir d2/d9/d23/d3b 0 2024-09-21T04:15:06.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/183: getdents d4/dd/d10/d29 0 2024-09-21T04:15:06.637 INFO:tasks.workunit.client.0.smithi110.stdout:7/60: mknod dd/c11 0 2024-09-21T04:15:06.637 INFO:tasks.workunit.client.0.smithi110.stdout:3/76: dread d0/f9 [0,4194304] 0 2024-09-21T04:15:06.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/71: mkdir d1/d7/d14 0 2024-09-21T04:15:06.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/72: creat d1/f15 x:0 0 0 2024-09-21T04:15:06.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/184: unlink d4/dd/d10/l2f 0 2024-09-21T04:15:06.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/185: readlink d4/dd/l17 0 2024-09-21T04:15:06.640 INFO:tasks.workunit.client.0.smithi110.stdout:7/61: mknod dd/c12 0 2024-09-21T04:15:06.640 INFO:tasks.workunit.client.0.smithi110.stdout:4/87: dwrite d17/f1b [0,4194304] 0 2024-09-21T04:15:06.641 INFO:tasks.workunit.client.0.smithi110.stdout:3/77: creat d0/d12/d16/f19 x:0 0 0 2024-09-21T04:15:06.642 INFO:tasks.workunit.client.0.smithi110.stdout:1/191: write d2/d9/f1f [1287971,10584] 0 2024-09-21T04:15:06.644 INFO:tasks.workunit.client.0.smithi110.stdout:5/73: rmdir d1/d7/d14 0 2024-09-21T04:15:06.646 INFO:tasks.workunit.client.0.smithi110.stdout:0/186: rename d4/dd/d12/f27 to d4/dd/f32 0 2024-09-21T04:15:06.649 INFO:tasks.workunit.client.0.smithi110.stdout:3/78: mkdir d0/d12/d16/d1a 0 2024-09-21T04:15:06.649 INFO:tasks.workunit.client.0.smithi110.stdout:3/79: write d0/f7 [2619814,92449] 0 2024-09-21T04:15:06.650 INFO:tasks.workunit.client.0.smithi110.stdout:1/192: symlink d2/l3c 0 2024-09-21T04:15:06.650 INFO:tasks.workunit.client.0.smithi110.stdout:1/193: write d2/d9/f1a [1409044,30407] 0 2024-09-21T04:15:06.651 INFO:tasks.workunit.client.0.smithi110.stdout:1/194: write d2/d9/d23/f38 [4638520,102280] 0 2024-09-21T04:15:06.652 INFO:tasks.workunit.client.0.smithi110.stdout:5/74: stat d1/d7/l10 0 2024-09-21T04:15:06.659 INFO:tasks.workunit.client.0.smithi110.stdout:5/75: mknod d1/d7/c16 0 2024-09-21T04:15:06.659 INFO:tasks.workunit.client.0.smithi110.stdout:5/76: stat d1/f12 0 2024-09-21T04:15:06.669 INFO:tasks.workunit.client.0.smithi110.stdout:7/62: dwrite f2 [0,4194304] 0 2024-09-21T04:15:06.669 INFO:tasks.workunit.client.0.smithi110.stdout:7/63: chown dd/ff 765331785 1 2024-09-21T04:15:06.669 INFO:tasks.workunit.client.0.smithi110.stdout:7/64: rename dd/ce to dd/c13 0 2024-09-21T04:15:06.676 INFO:tasks.workunit.client.0.smithi110.stdout:1/195: dwrite d2/d9/f1b [0,4194304] 0 2024-09-21T04:15:06.676 INFO:tasks.workunit.client.0.smithi110.stdout:5/77: dwrite d1/f15 [0,4194304] 0 2024-09-21T04:15:06.676 INFO:tasks.workunit.client.0.smithi110.stdout:5/78: dread - d1/d7/f13 zero size 2024-09-21T04:15:06.678 INFO:tasks.workunit.client.0.smithi110.stdout:3/80: dwrite d0/f10 [0,4194304] 0 2024-09-21T04:15:06.679 INFO:tasks.workunit.client.0.smithi110.stdout:3/81: stat d0/fe 0 2024-09-21T04:15:06.679 INFO:tasks.workunit.client.0.smithi110.stdout:3/82: stat d0/f6 0 2024-09-21T04:15:06.679 INFO:tasks.workunit.client.0.smithi110.stdout:3/83: creat d0/d12/f1b x:0 0 0 2024-09-21T04:15:06.679 INFO:tasks.workunit.client.0.smithi110.stdout:4/88: truncate fb 1449057 0 2024-09-21T04:15:06.679 INFO:tasks.workunit.client.0.smithi110.stdout:4/89: chown f4 1411701 1 2024-09-21T04:15:06.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/187: dwrite d4/dd/d12/f23 [0,4194304] 0 2024-09-21T04:15:06.684 INFO:tasks.workunit.client.0.smithi110.stdout:5/79: dread d1/dc/ff [0,4194304] 0 2024-09-21T04:15:06.684 INFO:tasks.workunit.client.0.smithi110.stdout:5/80: chown d1/c2 39373320 1 2024-09-21T04:15:06.686 INFO:tasks.workunit.client.0.smithi110.stdout:3/84: rename d0/lc to d0/d12/d16/d1a/l1c 0 2024-09-21T04:15:06.686 INFO:tasks.workunit.client.0.smithi110.stdout:4/90: rename d17 to d17/d1d/d24 22 2024-09-21T04:15:06.686 INFO:tasks.workunit.client.0.smithi110.stdout:3/85: fsync d0/f10 0 2024-09-21T04:15:06.692 INFO:tasks.workunit.client.0.smithi110.stdout:0/188: dread d4/f21 [0,4194304] 0 2024-09-21T04:15:06.693 INFO:tasks.workunit.client.0.smithi110.stdout:1/196: dwrite d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:06.695 INFO:tasks.workunit.client.0.smithi110.stdout:0/189: mknod d4/c33 0 2024-09-21T04:15:06.695 INFO:tasks.workunit.client.0.smithi110.stdout:0/190: rename d4/dd to d4/dd/d34 22 2024-09-21T04:15:06.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/191: mknod d4/dd/d10/c35 0 2024-09-21T04:15:06.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/192: dread - d4/f31 zero size 2024-09-21T04:15:06.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/193: stat d4/dd/l17 0 2024-09-21T04:15:06.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/194: write d4/dd/d10/f1a [3426010,79669] 0 2024-09-21T04:15:06.698 INFO:tasks.workunit.client.0.smithi110.stdout:0/195: creat d4/dd/d10/d29/d2e/f36 x:0 0 0 2024-09-21T04:15:06.698 INFO:tasks.workunit.client.0.smithi110.stdout:0/196: write d4/dd/d10/d29/d2e/f36 [593471,129382] 0 2024-09-21T04:15:06.702 INFO:tasks.workunit.client.0.smithi110.stdout:3/86: write d0/f6 [5678259,47653] 0 2024-09-21T04:15:06.702 INFO:tasks.workunit.client.0.smithi110.stdout:3/87: write d0/f11 [853604,3647] 0 2024-09-21T04:15:06.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/197: dread d4/dd/d12/f23 [0,4194304] 0 2024-09-21T04:15:06.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/198: readlink d4/dd/l17 0 2024-09-21T04:15:06.717 INFO:tasks.workunit.client.0.smithi110.stdout:5/81: dwrite d1/dc/ff [4194304,4194304] 0 2024-09-21T04:15:06.717 INFO:tasks.workunit.client.0.smithi110.stdout:5/82: write d1/f12 [238367,35862] 0 2024-09-21T04:15:06.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/91: dwrite fa [0,4194304] 0 2024-09-21T04:15:06.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/92: write f4 [3534445,130269] 0 2024-09-21T04:15:06.718 INFO:tasks.workunit.client.0.smithi110.stdout:4/93: dread - d17/d1e/f23 zero size 2024-09-21T04:15:06.718 INFO:tasks.workunit.client.0.smithi110.stdout:4/94: dread - d17/f1a zero size 2024-09-21T04:15:06.718 INFO:tasks.workunit.client.0.smithi110.stdout:1/197: dwrite d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:06.719 INFO:tasks.workunit.client.0.smithi110.stdout:5/83: symlink d1/dc/l17 0 2024-09-21T04:15:06.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/199: dread d4/d1c/f25 [0,4194304] 0 2024-09-21T04:15:06.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/200: stat d4/f5 0 2024-09-21T04:15:06.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/201: write d4/dd/d10/d29/d2e/f36 [1064436,72271] 0 2024-09-21T04:15:06.723 INFO:tasks.workunit.client.0.smithi110.stdout:0/202: dread d4/dd/d10/f1a [0,4194304] 0 2024-09-21T04:15:06.726 INFO:tasks.workunit.client.0.smithi110.stdout:4/95: unlink l16 0 2024-09-21T04:15:06.726 INFO:tasks.workunit.client.0.smithi110.stdout:4/96: getdents d17/d1d/d20 0 2024-09-21T04:15:06.726 INFO:tasks.workunit.client.0.smithi110.stdout:1/198: mkdir d2/d9/d26/d3d 0 2024-09-21T04:15:06.728 INFO:tasks.workunit.client.0.smithi110.stdout:1/199: unlink d2/f17 0 2024-09-21T04:15:06.729 INFO:tasks.workunit.client.0.smithi110.stdout:5/84: mkdir d1/d7/d18 0 2024-09-21T04:15:06.730 INFO:tasks.workunit.client.0.smithi110.stdout:0/203: symlink d4/l37 0 2024-09-21T04:15:06.731 INFO:tasks.workunit.client.0.smithi110.stdout:1/200: truncate d2/fe 2524773 0 2024-09-21T04:15:06.732 INFO:tasks.workunit.client.0.smithi110.stdout:0/204: creat d4/dd/f38 x:0 0 0 2024-09-21T04:15:06.732 INFO:tasks.workunit.client.0.smithi110.stdout:0/205: creat d4/dd/d10/d29/d2e/f39 x:0 0 0 2024-09-21T04:15:06.732 INFO:tasks.workunit.client.0.smithi110.stdout:0/206: write d4/dd/d10/f1a [4106348,128071] 0 2024-09-21T04:15:06.734 INFO:tasks.workunit.client.0.smithi110.stdout:1/201: link d2/fc d2/d9/d23/f3e 0 2024-09-21T04:15:06.736 INFO:tasks.workunit.client.0.smithi110.stdout:1/202: creat d2/d9/d26/d3d/f3f x:0 0 0 2024-09-21T04:15:06.737 INFO:tasks.workunit.client.0.smithi110.stdout:5/85: dread d1/f15 [0,4194304] 0 2024-09-21T04:15:06.739 INFO:tasks.workunit.client.0.smithi110.stdout:1/203: creat d2/d9/d23/f40 x:0 0 0 2024-09-21T04:15:06.742 INFO:tasks.workunit.client.0.smithi110.stdout:4/97: dwrite d17/f1b [0,4194304] 0 2024-09-21T04:15:06.742 INFO:tasks.workunit.client.0.smithi110.stdout:4/98: chown c9 10 1 2024-09-21T04:15:06.754 INFO:tasks.workunit.client.0.smithi110.stdout:0/207: dwrite f2 [4194304,4194304] 0 2024-09-21T04:15:06.754 INFO:tasks.workunit.client.0.smithi110.stdout:0/208: stat d4/l37 0 2024-09-21T04:15:06.756 INFO:tasks.workunit.client.0.smithi110.stdout:4/99: write fa [864511,50973] 0 2024-09-21T04:15:06.758 INFO:tasks.workunit.client.0.smithi110.stdout:4/100: dread fb [0,4194304] 0 2024-09-21T04:15:06.775 INFO:tasks.workunit.client.0.smithi110.stdout:4/101: dwrite f4 [0,4194304] 0 2024-09-21T04:15:06.841 INFO:tasks.workunit.client.0.smithi110.stdout:0/209: mkdir d4/dd/d12/d2c/d3a 0 2024-09-21T04:15:06.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/102: symlink d17/d1d/d20/l25 0 2024-09-21T04:15:06.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/210: creat d4/f3b x:0 0 0 2024-09-21T04:15:06.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/211: creat d4/d1c/f3c x:0 0 0 2024-09-21T04:15:06.846 INFO:tasks.workunit.client.0.smithi110.stdout:4/103: rename d17/f1b to d17/d1d/d20/f26 0 2024-09-21T04:15:06.847 INFO:tasks.workunit.client.0.smithi110.stdout:1/204: rmdir d2 39 2024-09-21T04:15:06.848 INFO:tasks.workunit.client.0.smithi110.stdout:0/212: mknod d4/dd/d12/d2c/d3a/c3d 0 2024-09-21T04:15:06.848 INFO:tasks.workunit.client.0.smithi110.stdout:0/213: read - d4/d1c/f24 zero size 2024-09-21T04:15:06.851 INFO:tasks.workunit.client.0.smithi110.stdout:0/214: truncate f2 7119007 0 2024-09-21T04:15:06.851 INFO:tasks.workunit.client.0.smithi110.stdout:0/215: chown d4/dd/f1b 2449528 1 2024-09-21T04:15:06.852 INFO:tasks.workunit.client.0.smithi110.stdout:0/216: symlink d4/dd/l3e 0 2024-09-21T04:15:06.854 INFO:tasks.workunit.client.0.smithi110.stdout:0/217: symlink d4/dd/d10/d29/d2e/l3f 0 2024-09-21T04:15:06.854 INFO:tasks.workunit.client.0.smithi110.stdout:0/218: creat d4/dd/d12/f40 x:0 0 0 2024-09-21T04:15:06.854 INFO:tasks.workunit.client.0.smithi110.stdout:0/219: creat d4/dd/d10/f41 x:0 0 0 2024-09-21T04:15:06.854 INFO:tasks.workunit.client.0.smithi110.stdout:0/220: chown d4/dd/d12/d2c/d3a 280331 1 2024-09-21T04:15:06.856 INFO:tasks.workunit.client.0.smithi110.stdout:0/221: getdents d4/dd/d12/d2c 0 2024-09-21T04:15:06.857 INFO:tasks.workunit.client.0.smithi110.stdout:4/104: write f4 [1821541,37286] 0 2024-09-21T04:15:06.857 INFO:tasks.workunit.client.0.smithi110.stdout:4/105: dread - f5 zero size 2024-09-21T04:15:06.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/106: dread - d17/d1e/f1f zero size 2024-09-21T04:15:06.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/107: creat d17/f27 x:0 0 0 2024-09-21T04:15:06.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/108: fsync d17/d1e/f1f 0 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:1/205: dwrite d2/d9/d26/f34 [0,4194304] 0 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/222: dwrite d4/dd/f1b [0,4194304] 0 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:4/109: dwrite f14 [0,4194304] 0 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/223: dread - d4/dd/d10/f41 zero size 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/224: write d4/dd/d10/d29/d2e/f36 [1358033,18724] 0 2024-09-21T04:15:06.899 INFO:tasks.workunit.client.0.smithi110.stdout:4/110: write d17/d1e/f1f [530079,72513] 0 2024-09-21T04:15:06.900 INFO:tasks.workunit.client.0.smithi110.stdout:0/225: fsync d4/dd/d10/f2d 0 2024-09-21T04:15:06.900 INFO:tasks.workunit.client.0.smithi110.stdout:0/226: chown d4/dd/d12/d2c/d3a/c3d 36854 1 2024-09-21T04:15:06.902 INFO:tasks.workunit.client.0.smithi110.stdout:1/206: creat d2/d9/d23/d3b/f41 x:0 0 0 2024-09-21T04:15:06.902 INFO:tasks.workunit.client.0.smithi110.stdout:1/207: chown d2/d9/c32 1052 1 2024-09-21T04:15:06.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/111: rename cc to d17/d1d/c28 0 2024-09-21T04:15:06.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/112: write fa [2599676,21351] 0 2024-09-21T04:15:06.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/113: chown cf 31 1 2024-09-21T04:15:06.909 INFO:tasks.workunit.client.0.smithi110.stdout:1/208: rename d2/d9/c11 to d2/d9/d26/c42 0 2024-09-21T04:15:06.909 INFO:tasks.workunit.client.0.smithi110.stdout:4/114: creat d17/d1d/d20/f29 x:0 0 0 2024-09-21T04:15:06.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/227: rmdir d4/dd 39 2024-09-21T04:15:06.913 INFO:tasks.workunit.client.0.smithi110.stdout:1/209: creat d2/d9/d26/d3d/f43 x:0 0 0 2024-09-21T04:15:06.913 INFO:tasks.workunit.client.0.smithi110.stdout:1/210: fsync d2/d9/d26/f31 0 2024-09-21T04:15:06.914 INFO:tasks.workunit.client.0.smithi110.stdout:4/115: rename d17/d1e/f23 to d17/f2a 0 2024-09-21T04:15:06.914 INFO:tasks.workunit.client.0.smithi110.stdout:4/116: truncate d17/f27 73100 0 2024-09-21T04:15:06.914 INFO:tasks.workunit.client.0.smithi110.stdout:4/117: chown f4 3710039 1 2024-09-21T04:15:06.916 INFO:tasks.workunit.client.0.smithi110.stdout:0/228: mknod d4/dd/d12/d2c/d3a/c42 0 2024-09-21T04:15:06.925 INFO:tasks.workunit.client.0.smithi110.stdout:4/118: dread f8 [0,4194304] 0 2024-09-21T04:15:06.925 INFO:tasks.workunit.client.0.smithi110.stdout:4/119: fdatasync d17/d1d/d20/f26 0 2024-09-21T04:15:06.925 INFO:tasks.workunit.client.0.smithi110.stdout:4/120: stat d17/l19 0 2024-09-21T04:15:06.925 INFO:tasks.workunit.client.0.smithi110.stdout:4/121: write f5 [986290,43028] 0 2024-09-21T04:15:06.926 INFO:tasks.workunit.client.0.smithi110.stdout:4/122: mkdir d17/d1d/d20/d2b 0 2024-09-21T04:15:06.926 INFO:tasks.workunit.client.0.smithi110.stdout:4/123: truncate d17/f2a 462699 0 2024-09-21T04:15:06.926 INFO:tasks.workunit.client.0.smithi110.stdout:4/124: write d17/d1e/f1f [884157,32818] 0 2024-09-21T04:15:06.933 INFO:tasks.workunit.client.0.smithi110.stdout:1/211: dwrite d2/d9/d26/f34 [0,4194304] 0 2024-09-21T04:15:06.933 INFO:tasks.workunit.client.0.smithi110.stdout:1/212: readlink d2/l18 0 2024-09-21T04:15:06.935 INFO:tasks.workunit.client.0.smithi110.stdout:1/213: rename d2/f2b to d2/d9/d23/f44 0 2024-09-21T04:15:06.935 INFO:tasks.workunit.client.0.smithi110.stdout:0/229: dwrite d4/dd/d10/d29/d2e/f39 [0,4194304] 0 2024-09-21T04:15:06.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/214: rename d2/d9/d23/d3b/f41 to d2/f45 0 2024-09-21T04:15:06.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/215: chown d2 59 1 2024-09-21T04:15:06.938 INFO:tasks.workunit.client.0.smithi110.stdout:1/216: truncate d2/fc 244282 0 2024-09-21T04:15:06.939 INFO:tasks.workunit.client.0.smithi110.stdout:4/125: write f14 [1144724,73154] 0 2024-09-21T04:15:06.939 INFO:tasks.workunit.client.0.smithi110.stdout:4/126: creat d17/d1d/d20/f2c x:0 0 0 2024-09-21T04:15:06.941 INFO:tasks.workunit.client.0.smithi110.stdout:1/217: symlink d2/d9/d23/d3b/l46 0 2024-09-21T04:15:06.943 INFO:tasks.workunit.client.0.smithi110.stdout:1/218: getdents d2/d9/d23 0 2024-09-21T04:15:06.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/219: dread d2/d9/f1a [0,4194304] 0 2024-09-21T04:15:06.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/220: chown d2/d9/l1d 1 1 2024-09-21T04:15:06.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/221: write d2/fc [128902,70642] 0 2024-09-21T04:15:06.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/222: truncate d2/f45 305473 0 2024-09-21T04:15:06.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/223: write d2/d9/d23/f38 [2614152,42003] 0 2024-09-21T04:15:06.947 INFO:tasks.workunit.client.0.smithi110.stdout:1/224: link d2/d9/d23/c25 d2/d9/d26/c47 0 2024-09-21T04:15:06.955 INFO:tasks.workunit.client.0.smithi110.stdout:4/127: dwrite f8 [0,4194304] 0 2024-09-21T04:15:06.957 INFO:tasks.workunit.client.0.smithi110.stdout:1/225: dread d2/fc [0,4194304] 0 2024-09-21T04:15:06.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/25: sync 2024-09-21T04:15:06.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/26: readlink - no filename 2024-09-21T04:15:06.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/226: creat d2/f48 x:0 0 0 2024-09-21T04:15:06.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/227: write d2/d9/d26/d3d/f43 [977847,26573] 0 2024-09-21T04:15:06.968 INFO:tasks.workunit.client.0.smithi110.stdout:4/128: dwrite f3 [0,4194304] 0 2024-09-21T04:15:06.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/228: dwrite d2/f48 [0,4194304] 0 2024-09-21T04:15:06.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/229: truncate f1 4775520 0 2024-09-21T04:15:06.983 INFO:tasks.workunit.client.0.smithi110.stdout:1/230: dread d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:06.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/27: dwrite d2/f4 [0,4194304] 0 2024-09-21T04:15:06.989 INFO:tasks.workunit.client.0.smithi110.stdout:4/129: dwrite d17/f27 [0,4194304] 0 2024-09-21T04:15:06.993 INFO:tasks.workunit.client.0.smithi110.stdout:4/130: link d17/l19 d17/d1d/d20/l2d 0 2024-09-21T04:15:06.996 INFO:tasks.workunit.client.0.smithi110.stdout:4/131: dread fb [0,4194304] 0 2024-09-21T04:15:06.997 INFO:tasks.workunit.client.0.smithi110.stdout:4/132: truncate f14 724862 0 2024-09-21T04:15:07.003 INFO:tasks.workunit.client.0.smithi110.stdout:4/133: dread f4 [0,4194304] 0 2024-09-21T04:15:07.003 INFO:tasks.workunit.client.0.smithi110.stdout:1/231: dwrite d2/f4 [0,4194304] 0 2024-09-21T04:15:07.005 INFO:tasks.workunit.client.0.smithi110.stdout:4/134: rename f3 to d17/d1e/d21/f2e 0 2024-09-21T04:15:07.005 INFO:tasks.workunit.client.0.smithi110.stdout:4/135: stat f5 0 2024-09-21T04:15:07.005 INFO:tasks.workunit.client.0.smithi110.stdout:4/136: readlink d17/l19 0 2024-09-21T04:15:07.006 INFO:tasks.workunit.client.0.smithi110.stdout:4/137: mknod d17/d1e/c2f 0 2024-09-21T04:15:07.008 INFO:tasks.workunit.client.0.smithi110.stdout:4/138: mknod d17/d1e/d21/c30 0 2024-09-21T04:15:07.008 INFO:tasks.workunit.client.0.smithi110.stdout:4/139: write fb [2024161,65111] 0 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/140: creat d17/d1e/d21/f31 x:0 0 0 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/141: fsync d17/d1d/d20/f29 0 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/142: creat d17/f32 x:0 0 0 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/143: chown d17/d1d 8 1 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/144: creat d17/d1e/f33 x:0 0 0 2024-09-21T04:15:07.010 INFO:tasks.workunit.client.0.smithi110.stdout:1/232: read d2/f4 [1185947,47655] 0 2024-09-21T04:15:07.013 INFO:tasks.workunit.client.0.smithi110.stdout:4/145: mkdir d17/d34 0 2024-09-21T04:15:07.014 INFO:tasks.workunit.client.0.smithi110.stdout:1/233: mkdir d2/d9/d49 0 2024-09-21T04:15:07.014 INFO:tasks.workunit.client.0.smithi110.stdout:1/234: write d2/d9/d26/f2d [545490,49755] 0 2024-09-21T04:15:07.015 INFO:tasks.workunit.client.0.smithi110.stdout:4/146: mknod d17/d1e/d21/c35 0 2024-09-21T04:15:07.016 INFO:tasks.workunit.client.0.smithi110.stdout:1/235: mknod d2/d9/d26/d3d/c4a 0 2024-09-21T04:15:07.017 INFO:tasks.workunit.client.0.smithi110.stdout:4/147: symlink d17/l36 0 2024-09-21T04:15:07.019 INFO:tasks.workunit.client.0.smithi110.stdout:6/26: sync 2024-09-21T04:15:07.019 INFO:tasks.workunit.client.0.smithi110.stdout:9/12: sync 2024-09-21T04:15:07.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/86: sync 2024-09-21T04:15:07.019 INFO:tasks.workunit.client.0.smithi110.stdout:6/27: creat d4/f8 x:0 0 0 2024-09-21T04:15:07.025 INFO:tasks.workunit.client.0.smithi110.stdout:6/28: rmdir d4 39 2024-09-21T04:15:07.027 INFO:tasks.workunit.client.0.smithi110.stdout:1/236: dread d2/d9/d26/f34 [0,4194304] 0 2024-09-21T04:15:07.028 INFO:tasks.workunit.client.0.smithi110.stdout:1/237: symlink d2/l4b 0 2024-09-21T04:15:07.028 INFO:tasks.workunit.client.0.smithi110.stdout:1/238: fsync d2/f4 0 2024-09-21T04:15:07.030 INFO:tasks.workunit.client.0.smithi110.stdout:1/239: creat d2/d9/f4c x:0 0 0 2024-09-21T04:15:07.030 INFO:tasks.workunit.client.0.smithi110.stdout:1/240: read d2/f4 [1396699,14986] 0 2024-09-21T04:15:07.030 INFO:tasks.workunit.client.0.smithi110.stdout:1/241: chown d2/lb 1143214 1 2024-09-21T04:15:07.030 INFO:tasks.workunit.client.0.smithi110.stdout:1/242: chown d2/d9/c27 248 1 2024-09-21T04:15:07.031 INFO:tasks.workunit.client.0.smithi110.stdout:1/243: creat d2/d9/d26/f4d x:0 0 0 2024-09-21T04:15:07.031 INFO:tasks.workunit.client.0.smithi110.stdout:1/244: readlink d2/lb 0 2024-09-21T04:15:07.031 INFO:tasks.workunit.client.0.smithi110.stdout:1/245: fdatasync d2/f33 0 2024-09-21T04:15:07.032 INFO:tasks.workunit.client.0.smithi110.stdout:1/246: mkdir d2/d9/d23/d4e 0 2024-09-21T04:15:07.032 INFO:tasks.workunit.client.0.smithi110.stdout:1/247: write f1 [3713507,73775] 0 2024-09-21T04:15:07.032 INFO:tasks.workunit.client.0.smithi110.stdout:1/248: mkdir d2/d9/d49/d4f 0 2024-09-21T04:15:07.033 INFO:tasks.workunit.client.0.smithi110.stdout:8/87: dread d0/fb [0,4194304] 0 2024-09-21T04:15:07.034 INFO:tasks.workunit.client.0.smithi110.stdout:1/249: unlink d2/d9/f1b 0 2024-09-21T04:15:07.034 INFO:tasks.workunit.client.0.smithi110.stdout:1/250: write d2/f8 [2788494,71107] 0 2024-09-21T04:15:07.036 INFO:tasks.workunit.client.0.smithi110.stdout:8/88: creat d0/d7/d13/f1e x:0 0 0 2024-09-21T04:15:07.040 INFO:tasks.workunit.client.0.smithi110.stdout:4/148: dwrite d17/f2a [0,4194304] 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/251: dread d2/f48 [0,4194304] 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/252: fdatasync d2/d9/d23/f3e 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/253: read d2/d9/d23/f3e [31093,31923] 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/254: readlink d2/l3c 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/255: write d2/d9/d26/f2d [1005430,49177] 0 2024-09-21T04:15:07.042 INFO:tasks.workunit.client.0.smithi110.stdout:4/149: creat d17/d34/f37 x:0 0 0 2024-09-21T04:15:07.043 INFO:tasks.workunit.client.0.smithi110.stdout:4/150: mknod d17/d1e/c38 0 2024-09-21T04:15:07.043 INFO:tasks.workunit.client.0.smithi110.stdout:4/151: creat d17/f39 x:0 0 0 2024-09-21T04:15:07.044 INFO:tasks.workunit.client.0.smithi110.stdout:6/29: dwrite d4/f7 [0,4194304] 0 2024-09-21T04:15:07.046 INFO:tasks.workunit.client.0.smithi110.stdout:1/256: write d2/d9/d26/f31 [627261,130685] 0 2024-09-21T04:15:07.047 INFO:tasks.workunit.client.0.smithi110.stdout:4/152: unlink d17/d1e/d21/c35 0 2024-09-21T04:15:07.052 INFO:tasks.workunit.client.0.smithi110.stdout:1/257: mknod d2/c50 0 2024-09-21T04:15:07.052 INFO:tasks.workunit.client.0.smithi110.stdout:1/258: read d2/d9/d23/f38 [3196599,47334] 0 2024-09-21T04:15:07.055 INFO:tasks.workunit.client.0.smithi110.stdout:6/30: dread f3 [4194304,4194304] 0 2024-09-21T04:15:07.059 INFO:tasks.workunit.client.0.smithi110.stdout:8/89: dwrite d0/f11 [0,4194304] 0 2024-09-21T04:15:07.063 INFO:tasks.workunit.client.0.smithi110.stdout:8/90: creat d0/d7/d18/f1f x:0 0 0 2024-09-21T04:15:07.063 INFO:tasks.workunit.client.0.smithi110.stdout:8/91: fsync d0/fb 0 2024-09-21T04:15:07.064 INFO:tasks.workunit.client.0.smithi110.stdout:8/92: chown d0/l6 1065 1 2024-09-21T04:15:07.064 INFO:tasks.workunit.client.0.smithi110.stdout:8/93: write d0/d7/d18/f1f [962977,49492] 0 2024-09-21T04:15:07.064 INFO:tasks.workunit.client.0.smithi110.stdout:8/94: creat d0/d7/f20 x:0 0 0 2024-09-21T04:15:07.065 INFO:tasks.workunit.client.0.smithi110.stdout:6/31: write d4/f7 [1994953,22050] 0 2024-09-21T04:15:07.066 INFO:tasks.workunit.client.0.smithi110.stdout:8/95: mknod d0/d7/c21 0 2024-09-21T04:15:07.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/32: link d4/f7 d4/f9 0 2024-09-21T04:15:07.069 INFO:tasks.workunit.client.0.smithi110.stdout:4/153: dwrite d17/d1d/d20/f26 [0,4194304] 0 2024-09-21T04:15:07.069 INFO:tasks.workunit.client.0.smithi110.stdout:4/154: write d17/d1e/d21/f31 [1014701,129660] 0 2024-09-21T04:15:07.070 INFO:tasks.workunit.client.0.smithi110.stdout:1/259: dwrite d2/fe [0,4194304] 0 2024-09-21T04:15:07.070 INFO:tasks.workunit.client.0.smithi110.stdout:4/155: mknod d17/d1d/d20/c3a 0 2024-09-21T04:15:07.073 INFO:tasks.workunit.client.0.smithi110.stdout:6/33: getdents d4 0 2024-09-21T04:15:07.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/156: rename d17/d1e/d21/c30 to d17/d34/c3b 0 2024-09-21T04:15:07.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/157: creat d17/f3c x:0 0 0 2024-09-21T04:15:07.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/158: fdatasync fa 0 2024-09-21T04:15:07.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/159: chown d17/d1e/d21 55 1 2024-09-21T04:15:07.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/160: truncate f4 4940383 0 2024-09-21T04:15:07.077 INFO:tasks.workunit.client.0.smithi110.stdout:6/34: creat d4/fa x:0 0 0 2024-09-21T04:15:07.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/260: write d2/d9/d23/f29 [2902427,117204] 0 2024-09-21T04:15:07.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/261: fdatasync d2/f2e 0 2024-09-21T04:15:07.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/262: truncate d2/fc 366365 0 2024-09-21T04:15:07.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/263: write d2/d9/d23/f29 [1796014,6848] 0 2024-09-21T04:15:07.079 INFO:tasks.workunit.client.0.smithi110.stdout:1/264: mknod d2/d9/d26/c51 0 2024-09-21T04:15:07.079 INFO:tasks.workunit.client.0.smithi110.stdout:1/265: write d2/d9/d26/f2d [1964000,68656] 0 2024-09-21T04:15:07.079 INFO:tasks.workunit.client.0.smithi110.stdout:1/266: creat d2/d9/d49/d4f/f52 x:0 0 0 2024-09-21T04:15:07.080 INFO:tasks.workunit.client.0.smithi110.stdout:1/267: mkdir d2/d9/d49/d4f/d53 0 2024-09-21T04:15:07.080 INFO:tasks.workunit.client.0.smithi110.stdout:1/268: chown d2/d9/d26/d3d/c4a 7344683 1 2024-09-21T04:15:07.081 INFO:tasks.workunit.client.0.smithi110.stdout:6/35: write f3 [6086698,7277] 0 2024-09-21T04:15:07.081 INFO:tasks.workunit.client.0.smithi110.stdout:1/269: symlink d2/l54 0 2024-09-21T04:15:07.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/270: truncate d2/d9/d23/f38 924879 0 2024-09-21T04:15:07.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/271: write d2/d9/d26/d3d/f3f [879433,124765] 0 2024-09-21T04:15:07.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/272: creat d2/d9/d23/d3b/f55 x:0 0 0 2024-09-21T04:15:07.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/273: write d2/f2e [936181,104762] 0 2024-09-21T04:15:07.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/274: read d2/f4 [727004,56721] 0 2024-09-21T04:15:07.089 INFO:tasks.workunit.client.0.smithi110.stdout:1/275: read d2/d9/d26/f34 [29717,66699] 0 2024-09-21T04:15:07.090 INFO:tasks.workunit.client.0.smithi110.stdout:1/276: truncate d2/f4 1292945 0 2024-09-21T04:15:07.090 INFO:tasks.workunit.client.0.smithi110.stdout:1/277: mknod d2/d9/d26/d3d/c56 0 2024-09-21T04:15:07.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/278: mkdir d2/d9/d23/d4e/d57 0 2024-09-21T04:15:07.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/279: fsync d2/d9/f1a 0 2024-09-21T04:15:07.092 INFO:tasks.workunit.client.0.smithi110.stdout:4/161: dwrite d17/d34/f37 [0,4194304] 0 2024-09-21T04:15:07.096 INFO:tasks.workunit.client.0.smithi110.stdout:6/36: truncate f3 4331180 0 2024-09-21T04:15:07.100 INFO:tasks.workunit.client.0.smithi110.stdout:4/162: rename d17/d1d to d17/d34/d3d 0 2024-09-21T04:15:07.101 INFO:tasks.workunit.client.0.smithi110.stdout:4/163: unlink d17/d34/c3b 0 2024-09-21T04:15:07.112 INFO:tasks.workunit.client.0.smithi110.stdout:4/164: dread d17/f2a [0,4194304] 0 2024-09-21T04:15:07.113 INFO:tasks.workunit.client.0.smithi110.stdout:1/280: dwrite d2/d9/d23/f44 [0,4194304] 0 2024-09-21T04:15:07.113 INFO:tasks.workunit.client.0.smithi110.stdout:1/281: chown f1 293449957 1 2024-09-21T04:15:07.129 INFO:tasks.workunit.client.0.smithi110.stdout:7/65: sync 2024-09-21T04:15:07.129 INFO:tasks.workunit.client.0.smithi110.stdout:3/88: sync 2024-09-21T04:15:07.130 INFO:tasks.workunit.client.0.smithi110.stdout:5/86: sync 2024-09-21T04:15:07.130 INFO:tasks.workunit.client.0.smithi110.stdout:3/89: dread - d0/d12/d16/f19 zero size 2024-09-21T04:15:07.131 INFO:tasks.workunit.client.0.smithi110.stdout:3/90: creat d0/f1d x:0 0 0 2024-09-21T04:15:07.131 INFO:tasks.workunit.client.0.smithi110.stdout:3/91: stat d0/c14 0 2024-09-21T04:15:07.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/87: link d1/d7/f13 d1/d7/d8/f19 0 2024-09-21T04:15:07.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/88: chown d1/d7/d18 1 1 2024-09-21T04:15:07.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/89: chown d1/d7 19431262 1 2024-09-21T04:15:07.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/90: link d1/f6 d1/dc/f1a 0 2024-09-21T04:15:07.133 INFO:tasks.workunit.client.0.smithi110.stdout:5/91: creat d1/d7/d8/de/f1b x:0 0 0 2024-09-21T04:15:07.134 INFO:tasks.workunit.client.0.smithi110.stdout:5/92: creat d1/d7/d8/f1c x:0 0 0 2024-09-21T04:15:07.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/282: dwrite d2/d9/d23/f29 [0,4194304] 0 2024-09-21T04:15:07.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/283: chown d2/c37 7756533 1 2024-09-21T04:15:07.139 INFO:tasks.workunit.client.0.smithi110.stdout:3/92: read d0/f10 [3192065,12013] 0 2024-09-21T04:15:07.139 INFO:tasks.workunit.client.0.smithi110.stdout:3/93: truncate d0/fe 5100620 0 2024-09-21T04:15:07.139 INFO:tasks.workunit.client.0.smithi110.stdout:3/94: chown d0/d12 406610 1 2024-09-21T04:15:07.139 INFO:tasks.workunit.client.0.smithi110.stdout:3/95: chown d0/d12/d16 768382950 1 2024-09-21T04:15:07.139 INFO:tasks.workunit.client.0.smithi110.stdout:3/96: creat d0/f1e x:0 0 0 2024-09-21T04:15:07.141 INFO:tasks.workunit.client.0.smithi110.stdout:3/97: link d0/f10 d0/f1f 0 2024-09-21T04:15:07.142 INFO:tasks.workunit.client.0.smithi110.stdout:3/98: rename d0/f9 to d0/d12/d16/d1a/f20 0 2024-09-21T04:15:07.142 INFO:tasks.workunit.client.0.smithi110.stdout:7/66: dread f2 [0,4194304] 0 2024-09-21T04:15:07.143 INFO:tasks.workunit.client.0.smithi110.stdout:7/67: mkdir dd/d14 0 2024-09-21T04:15:07.148 INFO:tasks.workunit.client.0.smithi110.stdout:5/93: dwrite d1/d7/f13 [0,4194304] 0 2024-09-21T04:15:07.148 INFO:tasks.workunit.client.0.smithi110.stdout:7/68: dread f2 [0,4194304] 0 2024-09-21T04:15:07.151 INFO:tasks.workunit.client.0.smithi110.stdout:5/94: unlink d1/d7/l10 0 2024-09-21T04:15:07.152 INFO:tasks.workunit.client.0.smithi110.stdout:3/99: unlink d0/d12/d16/d1a/f20 0 2024-09-21T04:15:07.153 INFO:tasks.workunit.client.0.smithi110.stdout:7/69: read f2 [696794,123384] 0 2024-09-21T04:15:07.154 INFO:tasks.workunit.client.0.smithi110.stdout:3/100: unlink d0/d12/d16/d1a/l1c 0 2024-09-21T04:15:07.154 INFO:tasks.workunit.client.0.smithi110.stdout:5/95: write d1/d7/f13 [4709451,58085] 0 2024-09-21T04:15:07.154 INFO:tasks.workunit.client.0.smithi110.stdout:5/96: truncate d1/f12 972279 0 2024-09-21T04:15:07.156 INFO:tasks.workunit.client.0.smithi110.stdout:3/101: link d0/ld d0/l21 0 2024-09-21T04:15:07.158 INFO:tasks.workunit.client.0.smithi110.stdout:3/102: mkdir d0/d22 0 2024-09-21T04:15:07.158 INFO:tasks.workunit.client.0.smithi110.stdout:3/103: write d0/d12/d16/f19 [6610,84616] 0 2024-09-21T04:15:07.158 INFO:tasks.workunit.client.0.smithi110.stdout:7/70: dread f2 [0,4194304] 0 2024-09-21T04:15:07.159 INFO:tasks.workunit.client.0.smithi110.stdout:7/71: chown dd/c11 56901061 1 2024-09-21T04:15:07.160 INFO:tasks.workunit.client.0.smithi110.stdout:7/72: unlink dd/c11 0 2024-09-21T04:15:07.163 INFO:tasks.workunit.client.0.smithi110.stdout:7/73: dread dd/ff [0,4194304] 0 2024-09-21T04:15:07.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/104: dread d0/fe [0,4194304] 0 2024-09-21T04:15:07.168 INFO:tasks.workunit.client.0.smithi110.stdout:7/74: getdents dd 0 2024-09-21T04:15:07.169 INFO:tasks.workunit.client.0.smithi110.stdout:3/105: link d0/fe d0/d12/f23 0 2024-09-21T04:15:07.169 INFO:tasks.workunit.client.0.smithi110.stdout:5/97: dwrite d1/d7/d8/de/f1b [0,4194304] 0 2024-09-21T04:15:07.170 INFO:tasks.workunit.client.0.smithi110.stdout:7/75: creat dd/d14/f15 x:0 0 0 2024-09-21T04:15:07.171 INFO:tasks.workunit.client.0.smithi110.stdout:3/106: rename d0/f1e to d0/d12/d16/d1a/f24 0 2024-09-21T04:15:07.171 INFO:tasks.workunit.client.0.smithi110.stdout:3/107: creat d0/d12/d16/f25 x:0 0 0 2024-09-21T04:15:07.172 INFO:tasks.workunit.client.0.smithi110.stdout:5/98: creat d1/dc/f1d x:0 0 0 2024-09-21T04:15:07.177 INFO:tasks.workunit.client.0.smithi110.stdout:3/108: symlink d0/l26 0 2024-09-21T04:15:07.177 INFO:tasks.workunit.client.0.smithi110.stdout:5/99: mkdir d1/d7/d8/de/d1e 0 2024-09-21T04:15:07.178 INFO:tasks.workunit.client.0.smithi110.stdout:7/76: rmdir dd/d14 39 2024-09-21T04:15:07.179 INFO:tasks.workunit.client.0.smithi110.stdout:3/109: mkdir d0/d12/d16/d1a/d27 0 2024-09-21T04:15:07.179 INFO:tasks.workunit.client.0.smithi110.stdout:3/110: creat d0/f28 x:0 0 0 2024-09-21T04:15:07.179 INFO:tasks.workunit.client.0.smithi110.stdout:3/111: write d0/f1d [399675,91793] 0 2024-09-21T04:15:07.180 INFO:tasks.workunit.client.0.smithi110.stdout:5/100: truncate d1/dc/ff 6917726 0 2024-09-21T04:15:07.180 INFO:tasks.workunit.client.0.smithi110.stdout:5/101: chown d1/dc/f1a 6906 1 2024-09-21T04:15:07.180 INFO:tasks.workunit.client.0.smithi110.stdout:5/102: write d1/f6 [1172084,57532] 0 2024-09-21T04:15:07.180 INFO:tasks.workunit.client.0.smithi110.stdout:7/77: link dd/c10 dd/d14/c16 0 2024-09-21T04:15:07.187 INFO:tasks.workunit.client.0.smithi110.stdout:5/103: getdents d1/d7/d8/de 0 2024-09-21T04:15:07.187 INFO:tasks.workunit.client.0.smithi110.stdout:5/104: chown d1/dc/c11 56142932 1 2024-09-21T04:15:07.187 INFO:tasks.workunit.client.0.smithi110.stdout:5/105: fsync d1/dc/f1d 0 2024-09-21T04:15:07.208 INFO:tasks.workunit.client.0.smithi110.stdout:7/78: dwrite dd/d14/f15 [0,4194304] 0 2024-09-21T04:15:07.211 INFO:tasks.workunit.client.0.smithi110.stdout:3/112: dwrite d0/f10 [0,4194304] 0 2024-09-21T04:15:07.212 INFO:tasks.workunit.client.0.smithi110.stdout:5/106: dwrite d1/f6 [0,4194304] 0 2024-09-21T04:15:07.212 INFO:tasks.workunit.client.0.smithi110.stdout:5/107: getdents d1/d7/d8/de/d1e 0 2024-09-21T04:15:07.213 INFO:tasks.workunit.client.0.smithi110.stdout:3/113: mkdir d0/d22/d29 0 2024-09-21T04:15:07.213 INFO:tasks.workunit.client.0.smithi110.stdout:3/114: write d0/d12/d16/f25 [802244,18612] 0 2024-09-21T04:15:07.213 INFO:tasks.workunit.client.0.smithi110.stdout:5/108: symlink d1/d7/d8/de/l1f 0 2024-09-21T04:15:07.213 INFO:tasks.workunit.client.0.smithi110.stdout:5/109: readlink d1/dc/l17 0 2024-09-21T04:15:07.213 INFO:tasks.workunit.client.0.smithi110.stdout:0/230: sync 2024-09-21T04:15:07.214 INFO:tasks.workunit.client.0.smithi110.stdout:3/115: rename d0/d12/d16/d1a/d27 to d0/d12/d2a 0 2024-09-21T04:15:07.216 INFO:tasks.workunit.client.0.smithi110.stdout:0/231: creat d4/dd/d12/d2c/d3a/f43 x:0 0 0 2024-09-21T04:15:07.219 INFO:tasks.workunit.client.0.smithi110.stdout:0/232: rename l3 to d4/dd/d12/d2c/l44 0 2024-09-21T04:15:07.225 INFO:tasks.workunit.client.0.smithi110.stdout:5/110: dread d1/d7/f13 [0,4194304] 0 2024-09-21T04:15:07.227 INFO:tasks.workunit.client.0.smithi110.stdout:5/111: symlink d1/d7/d8/de/d1e/l20 0 2024-09-21T04:15:07.231 INFO:tasks.workunit.client.0.smithi110.stdout:7/79: dwrite f2 [0,4194304] 0 2024-09-21T04:15:07.232 INFO:tasks.workunit.client.0.smithi110.stdout:7/80: chown dd/ff 5640964 1 2024-09-21T04:15:07.232 INFO:tasks.workunit.client.0.smithi110.stdout:7/81: write f2 [4500533,93193] 0 2024-09-21T04:15:07.232 INFO:tasks.workunit.client.0.smithi110.stdout:3/116: dwrite d0/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:07.232 INFO:tasks.workunit.client.0.smithi110.stdout:3/117: write d0/f7 [4104921,105991] 0 2024-09-21T04:15:07.239 INFO:tasks.workunit.client.0.smithi110.stdout:0/233: dwrite d4/f3b [0,4194304] 0 2024-09-21T04:15:07.241 INFO:tasks.workunit.client.0.smithi110.stdout:0/234: mkdir d4/dd/d10/d29/d45 0 2024-09-21T04:15:07.241 INFO:tasks.workunit.client.0.smithi110.stdout:0/235: chown d4/dd/d12/d2c/d3a/c3d 4077 1 2024-09-21T04:15:07.252 INFO:tasks.workunit.client.0.smithi110.stdout:7/82: dread dd/d14/f15 [0,4194304] 0 2024-09-21T04:15:07.254 INFO:tasks.workunit.client.0.smithi110.stdout:0/236: dread d4/f5 [0,4194304] 0 2024-09-21T04:15:07.258 INFO:tasks.workunit.client.0.smithi110.stdout:7/83: dread dd/d14/f15 [0,4194304] 0 2024-09-21T04:15:07.259 INFO:tasks.workunit.client.0.smithi110.stdout:5/112: dwrite d1/dc/f1a [0,4194304] 0 2024-09-21T04:15:07.259 INFO:tasks.workunit.client.0.smithi110.stdout:3/118: dwrite d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:07.259 INFO:tasks.workunit.client.0.smithi110.stdout:3/119: dread - d0/d12/f1b zero size 2024-09-21T04:15:07.260 INFO:tasks.workunit.client.0.smithi110.stdout:7/84: fsync f2 0 2024-09-21T04:15:07.261 INFO:tasks.workunit.client.0.smithi110.stdout:3/120: dread d0/fe [4194304,4194304] 0 2024-09-21T04:15:07.265 INFO:tasks.workunit.client.0.smithi110.stdout:9/13: sync 2024-09-21T04:15:07.265 INFO:tasks.workunit.client.0.smithi110.stdout:2/28: sync 2024-09-21T04:15:07.265 INFO:tasks.workunit.client.0.smithi110.stdout:8/96: sync 2024-09-21T04:15:07.265 INFO:tasks.workunit.client.0.smithi110.stdout:9/14: readlink - no filename 2024-09-21T04:15:07.266 INFO:tasks.workunit.client.0.smithi110.stdout:2/29: rename d2/f4 to d2/f6 0 2024-09-21T04:15:07.266 INFO:tasks.workunit.client.0.smithi110.stdout:2/30: readlink - no filename 2024-09-21T04:15:07.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/97: symlink d0/d7/d18/l22 0 2024-09-21T04:15:07.268 INFO:tasks.workunit.client.0.smithi110.stdout:9/15: getdents . 0 2024-09-21T04:15:07.268 INFO:tasks.workunit.client.0.smithi110.stdout:9/16: write f2 [4328116,51258] 0 2024-09-21T04:15:07.274 INFO:tasks.workunit.client.0.smithi110.stdout:2/31: dread d2/f6 [0,4194304] 0 2024-09-21T04:15:07.274 INFO:tasks.workunit.client.0.smithi110.stdout:0/237: dwrite d4/dd/d10/f2d [0,4194304] 0 2024-09-21T04:15:07.278 INFO:tasks.workunit.client.0.smithi110.stdout:0/238: mknod d4/dd/d10/d29/c46 0 2024-09-21T04:15:07.292 INFO:tasks.workunit.client.0.smithi110.stdout:0/239: dread f2 [4194304,4194304] 0 2024-09-21T04:15:07.295 INFO:tasks.workunit.client.0.smithi110.stdout:0/240: rename d4/dd/d12 to d4/d47 0 2024-09-21T04:15:07.295 INFO:tasks.workunit.client.0.smithi110.stdout:0/241: write d4/f31 [128619,119703] 0 2024-09-21T04:15:07.295 INFO:tasks.workunit.client.0.smithi110.stdout:0/242: truncate d4/d47/d2c/d3a/f43 249501 0 2024-09-21T04:15:07.299 INFO:tasks.workunit.client.0.smithi110.stdout:0/243: dread d4/f5 [0,4194304] 0 2024-09-21T04:15:07.300 INFO:tasks.workunit.client.0.smithi110.stdout:7/85: dwrite dd/ff [0,4194304] 0 2024-09-21T04:15:07.300 INFO:tasks.workunit.client.0.smithi110.stdout:7/86: chown dd/ff 476603120 1 2024-09-21T04:15:07.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/98: dwrite d0/d7/d18/f1f [0,4194304] 0 2024-09-21T04:15:07.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/121: dwrite d0/d12/f23 [0,4194304] 0 2024-09-21T04:15:07.301 INFO:tasks.workunit.client.0.smithi110.stdout:9/17: dwrite f2 [0,4194304] 0 2024-09-21T04:15:07.302 INFO:tasks.workunit.client.0.smithi110.stdout:4/165: sync 2024-09-21T04:15:07.302 INFO:tasks.workunit.client.0.smithi110.stdout:4/166: dread - d17/d1e/f33 zero size 2024-09-21T04:15:07.303 INFO:tasks.workunit.client.0.smithi110.stdout:0/244: symlink d4/dd/d10/d29/d45/l48 0 2024-09-21T04:15:07.304 INFO:tasks.workunit.client.0.smithi110.stdout:6/37: sync 2024-09-21T04:15:07.304 INFO:tasks.workunit.client.0.smithi110.stdout:1/284: sync 2024-09-21T04:15:07.312 INFO:tasks.workunit.client.0.smithi110.stdout:4/167: dread f8 [0,4194304] 0 2024-09-21T04:15:07.325 INFO:tasks.workunit.client.0.smithi110.stdout:7/87: dwrite dd/d14/f15 [4194304,4194304] 0 2024-09-21T04:15:07.330 INFO:tasks.workunit.client.0.smithi110.stdout:8/99: dwrite d0/d7/f1b [0,4194304] 0 2024-09-21T04:15:07.330 INFO:tasks.workunit.client.0.smithi110.stdout:4/168: dwrite d17/f32 [0,4194304] 0 2024-09-21T04:15:07.384 INFO:tasks.workunit.client.0.smithi110.stdout:0/245: mknod d4/dd/d10/d29/d2e/c49 0 2024-09-21T04:15:07.386 INFO:tasks.workunit.client.0.smithi110.stdout:1/285: link d2/d9/d23/f29 d2/d9/d26/d3d/f58 0 2024-09-21T04:15:07.388 INFO:tasks.workunit.client.0.smithi110.stdout:7/88: mkdir dd/d14/d17 0 2024-09-21T04:15:07.388 INFO:tasks.workunit.client.0.smithi110.stdout:7/89: chown dd/d14/c16 6343 1 2024-09-21T04:15:07.389 INFO:tasks.workunit.client.0.smithi110.stdout:8/100: mkdir d0/d7/d23 0 2024-09-21T04:15:07.389 INFO:tasks.workunit.client.0.smithi110.stdout:8/101: chown d0/d7/le 2066850 1 2024-09-21T04:15:07.391 INFO:tasks.workunit.client.0.smithi110.stdout:4/169: mkdir d17/d3e 0 2024-09-21T04:15:07.391 INFO:tasks.workunit.client.0.smithi110.stdout:4/170: write d17/f27 [4778681,76416] 0 2024-09-21T04:15:07.392 INFO:tasks.workunit.client.0.smithi110.stdout:9/18: truncate f2 160933 0 2024-09-21T04:15:07.392 INFO:tasks.workunit.client.0.smithi110.stdout:9/19: chown f2 7842 1 2024-09-21T04:15:07.394 INFO:tasks.workunit.client.0.smithi110.stdout:1/286: link d2/d9/d26/f34 d2/d9/d49/d4f/f59 0 2024-09-21T04:15:07.394 INFO:tasks.workunit.client.0.smithi110.stdout:1/287: chown d2/l54 115 1 2024-09-21T04:15:07.395 INFO:tasks.workunit.client.0.smithi110.stdout:7/90: creat dd/f18 x:0 0 0 2024-09-21T04:15:07.395 INFO:tasks.workunit.client.0.smithi110.stdout:4/171: symlink d17/d3e/l3f 0 2024-09-21T04:15:07.397 INFO:tasks.workunit.client.0.smithi110.stdout:9/20: symlink l4 0 2024-09-21T04:15:07.398 INFO:tasks.workunit.client.0.smithi110.stdout:6/38: link d4/f9 d4/fb 0 2024-09-21T04:15:07.398 INFO:tasks.workunit.client.0.smithi110.stdout:6/39: stat d4/f9 0 2024-09-21T04:15:07.401 INFO:tasks.workunit.client.0.smithi110.stdout:8/102: dread d0/f11 [0,4194304] 0 2024-09-21T04:15:07.401 INFO:tasks.workunit.client.0.smithi110.stdout:1/288: mkdir d2/d9/d49/d5a 0 2024-09-21T04:15:07.402 INFO:tasks.workunit.client.0.smithi110.stdout:9/21: mknod c5 0 2024-09-21T04:15:07.406 INFO:tasks.workunit.client.0.smithi110.stdout:0/246: dwrite f2 [0,4194304] 0 2024-09-21T04:15:07.406 INFO:tasks.workunit.client.0.smithi110.stdout:0/247: fsync d4/f5 0 2024-09-21T04:15:07.410 INFO:tasks.workunit.client.0.smithi110.stdout:7/91: dread dd/ff [0,4194304] 0 2024-09-21T04:15:07.413 INFO:tasks.workunit.client.0.smithi110.stdout:4/172: dwrite d17/d34/d3d/d20/f29 [0,4194304] 0 2024-09-21T04:15:07.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/173: write fb [2127852,6375] 0 2024-09-21T04:15:07.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/248: write d4/dd/fe [3163782,41469] 0 2024-09-21T04:15:07.418 INFO:tasks.workunit.client.0.smithi110.stdout:0/249: creat d4/d47/d2c/d3a/f4a x:0 0 0 2024-09-21T04:15:07.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/103: dread d0/d7/d18/f1f [0,4194304] 0 2024-09-21T04:15:07.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/104: dread - d0/d7/f20 zero size 2024-09-21T04:15:07.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/105: write d0/d7/f20 [49608,124214] 0 2024-09-21T04:15:07.425 INFO:tasks.workunit.client.0.smithi110.stdout:1/289: dwrite d2/f33 [0,4194304] 0 2024-09-21T04:15:07.425 INFO:tasks.workunit.client.0.smithi110.stdout:6/40: dread d4/f7 [0,4194304] 0 2024-09-21T04:15:07.426 INFO:tasks.workunit.client.0.smithi110.stdout:6/41: chown d4 211396267 1 2024-09-21T04:15:07.426 INFO:tasks.workunit.client.0.smithi110.stdout:9/22: dwrite f3 [0,4194304] 0 2024-09-21T04:15:07.427 INFO:tasks.workunit.client.0.smithi110.stdout:1/290: getdents d2/d9/d49 0 2024-09-21T04:15:07.432 INFO:tasks.workunit.client.0.smithi110.stdout:6/42: creat d4/fc x:0 0 0 2024-09-21T04:15:07.433 INFO:tasks.workunit.client.0.smithi110.stdout:9/23: creat f6 x:0 0 0 2024-09-21T04:15:07.435 INFO:tasks.workunit.client.0.smithi110.stdout:1/291: unlink d2/d9/f4c 0 2024-09-21T04:15:07.435 INFO:tasks.workunit.client.0.smithi110.stdout:1/292: chown d2/d9/d23/l2c 8514341 1 2024-09-21T04:15:07.436 INFO:tasks.workunit.client.0.smithi110.stdout:1/293: link d2/d9/d23/f44 d2/d9/d49/f5b 0 2024-09-21T04:15:07.438 INFO:tasks.workunit.client.0.smithi110.stdout:1/294: rmdir d2/d9/d49/d5a 0 2024-09-21T04:15:07.438 INFO:tasks.workunit.client.0.smithi110.stdout:1/295: truncate d2/fc 395699 0 2024-09-21T04:15:07.441 INFO:tasks.workunit.client.0.smithi110.stdout:8/106: dread d0/fb [0,4194304] 0 2024-09-21T04:15:07.441 INFO:tasks.workunit.client.0.smithi110.stdout:8/107: chown d0/l12 14131 1 2024-09-21T04:15:07.442 INFO:tasks.workunit.client.0.smithi110.stdout:8/108: mknod d0/d7/d18/c24 0 2024-09-21T04:15:07.448 INFO:tasks.workunit.client.0.smithi110.stdout:6/43: rmdir d4 39 2024-09-21T04:15:07.448 INFO:tasks.workunit.client.0.smithi110.stdout:8/109: dread d0/d7/d18/f1f [0,4194304] 0 2024-09-21T04:15:07.449 INFO:tasks.workunit.client.0.smithi110.stdout:7/92: dwrite dd/ff [0,4194304] 0 2024-09-21T04:15:07.449 INFO:tasks.workunit.client.0.smithi110.stdout:4/174: dwrite d17/d34/d3d/d20/f2c [0,4194304] 0 2024-09-21T04:15:07.451 INFO:tasks.workunit.client.0.smithi110.stdout:8/110: symlink d0/d7/d18/l25 0 2024-09-21T04:15:07.451 INFO:tasks.workunit.client.0.smithi110.stdout:8/111: write d0/d7/f20 [111964,40431] 0 2024-09-21T04:15:07.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/175: link d17/d34/d3d/d20/f26 d17/d3e/f40 0 2024-09-21T04:15:07.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/176: chown d17/d1e/d21 23228 1 2024-09-21T04:15:07.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/177: fsync fb 0 2024-09-21T04:15:07.454 INFO:tasks.workunit.client.0.smithi110.stdout:6/44: mknod d4/cd 0 2024-09-21T04:15:07.455 INFO:tasks.workunit.client.0.smithi110.stdout:8/112: truncate d0/d7/d18/f1f 1905078 0 2024-09-21T04:15:07.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/178: creat d17/d34/d3d/f41 x:0 0 0 2024-09-21T04:15:07.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/179: write f4 [5259648,13113] 0 2024-09-21T04:15:07.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/180: fdatasync d17/f1a 0 2024-09-21T04:15:07.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/113: mkdir d0/d26 0 2024-09-21T04:15:07.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/114: creat d0/d7/d18/f27 x:0 0 0 2024-09-21T04:15:07.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/115: chown d0/fb 1489727 1 2024-09-21T04:15:07.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/116: creat d0/d7/d18/f28 x:0 0 0 2024-09-21T04:15:07.464 INFO:tasks.workunit.client.0.smithi110.stdout:7/93: dread f2 [0,4194304] 0 2024-09-21T04:15:07.465 INFO:tasks.workunit.client.0.smithi110.stdout:7/94: creat dd/d14/d17/f19 x:0 0 0 2024-09-21T04:15:07.465 INFO:tasks.workunit.client.0.smithi110.stdout:9/24: dwrite f6 [0,4194304] 0 2024-09-21T04:15:07.467 INFO:tasks.workunit.client.0.smithi110.stdout:9/25: mkdir d7 0 2024-09-21T04:15:07.467 INFO:tasks.workunit.client.0.smithi110.stdout:9/26: chown f6 1057445208 1 2024-09-21T04:15:07.468 INFO:tasks.workunit.client.0.smithi110.stdout:1/296: dwrite d2/fc [0,4194304] 0 2024-09-21T04:15:07.471 INFO:tasks.workunit.client.0.smithi110.stdout:1/297: mkdir d2/d9/d49/d5c 0 2024-09-21T04:15:07.472 INFO:tasks.workunit.client.0.smithi110.stdout:5/113: sync 2024-09-21T04:15:07.473 INFO:tasks.workunit.client.0.smithi110.stdout:1/298: mknod d2/d9/d49/c5d 0 2024-09-21T04:15:07.475 INFO:tasks.workunit.client.0.smithi110.stdout:1/299: mkdir d2/d9/d23/d3b/d5e 0 2024-09-21T04:15:07.475 INFO:tasks.workunit.client.0.smithi110.stdout:1/300: stat d2/d9/l1d 0 2024-09-21T04:15:07.476 INFO:tasks.workunit.client.0.smithi110.stdout:9/27: write f3 [658559,52660] 0 2024-09-21T04:15:07.477 INFO:tasks.workunit.client.0.smithi110.stdout:1/301: rename d2/d9/d49/d4f/f52 to d2/d9/d23/d3b/d5e/f5f 0 2024-09-21T04:15:07.480 INFO:tasks.workunit.client.0.smithi110.stdout:1/302: dread d2/d9/f1a [0,4194304] 0 2024-09-21T04:15:07.485 INFO:tasks.workunit.client.0.smithi110.stdout:7/95: write dd/ff [3776764,111336] 0 2024-09-21T04:15:07.488 INFO:tasks.workunit.client.0.smithi110.stdout:6/45: dwrite d4/f9 [4194304,4194304] 0 2024-09-21T04:15:07.491 INFO:tasks.workunit.client.0.smithi110.stdout:9/28: write f6 [2068358,121562] 0 2024-09-21T04:15:07.491 INFO:tasks.workunit.client.0.smithi110.stdout:7/96: rmdir dd 39 2024-09-21T04:15:07.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/97: unlink l9 0 2024-09-21T04:15:07.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/98: creat dd/f1a x:0 0 0 2024-09-21T04:15:07.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/99: readlink l8 0 2024-09-21T04:15:07.494 INFO:tasks.workunit.client.0.smithi110.stdout:4/181: dwrite fb [0,4194304] 0 2024-09-21T04:15:07.494 INFO:tasks.workunit.client.0.smithi110.stdout:7/100: symlink dd/d14/d17/l1b 0 2024-09-21T04:15:07.496 INFO:tasks.workunit.client.0.smithi110.stdout:7/101: unlink dd/d14/f15 0 2024-09-21T04:15:07.496 INFO:tasks.workunit.client.0.smithi110.stdout:7/102: readlink dd/d14/d17/l1b 0 2024-09-21T04:15:07.497 INFO:tasks.workunit.client.0.smithi110.stdout:7/103: dread - dd/f1a zero size 2024-09-21T04:15:07.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/104: mkdir dd/d1c 0 2024-09-21T04:15:07.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/105: rename dd/d1c to dd/d1c/d1d 22 2024-09-21T04:15:07.499 INFO:tasks.workunit.client.0.smithi110.stdout:7/106: creat dd/d14/d17/f1e x:0 0 0 2024-09-21T04:15:07.499 INFO:tasks.workunit.client.0.smithi110.stdout:7/107: chown dd/d14/d17 13 1 2024-09-21T04:15:07.500 INFO:tasks.workunit.client.0.smithi110.stdout:5/114: dwrite d1/d7/d8/de/f1b [0,4194304] 0 2024-09-21T04:15:07.500 INFO:tasks.workunit.client.0.smithi110.stdout:5/115: creat d1/d7/d8/f21 x:0 0 0 2024-09-21T04:15:07.501 INFO:tasks.workunit.client.0.smithi110.stdout:7/108: rename l8 to dd/d14/d17/l1f 0 2024-09-21T04:15:07.504 INFO:tasks.workunit.client.0.smithi110.stdout:5/116: mkdir d1/d22 0 2024-09-21T04:15:07.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/109: creat dd/d14/f20 x:0 0 0 2024-09-21T04:15:07.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/110: fsync dd/f18 0 2024-09-21T04:15:07.509 INFO:tasks.workunit.client.0.smithi110.stdout:5/117: symlink d1/d7/d8/de/d1e/l23 0 2024-09-21T04:15:07.511 INFO:tasks.workunit.client.0.smithi110.stdout:7/111: unlink dd/ff 0 2024-09-21T04:15:07.511 INFO:tasks.workunit.client.0.smithi110.stdout:7/112: truncate dd/d14/f20 578215 0 2024-09-21T04:15:07.512 INFO:tasks.workunit.client.0.smithi110.stdout:5/118: creat d1/d7/d8/de/f24 x:0 0 0 2024-09-21T04:15:07.512 INFO:tasks.workunit.client.0.smithi110.stdout:5/119: creat d1/d7/d8/de/f25 x:0 0 0 2024-09-21T04:15:07.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/113: mknod dd/d14/c21 0 2024-09-21T04:15:07.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/114: stat dd/d14/d17 0 2024-09-21T04:15:07.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/115: stat dd/f1a 0 2024-09-21T04:15:07.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/116: read dd/d14/f20 [18011,89004] 0 2024-09-21T04:15:07.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/117: creat dd/d14/f22 x:0 0 0 2024-09-21T04:15:07.516 INFO:tasks.workunit.client.0.smithi110.stdout:1/303: dwrite d2/d9/d49/d4f/f59 [0,4194304] 0 2024-09-21T04:15:07.520 INFO:tasks.workunit.client.0.smithi110.stdout:7/118: symlink dd/d1c/l23 0 2024-09-21T04:15:07.522 INFO:tasks.workunit.client.0.smithi110.stdout:6/46: dwrite d4/fb [8388608,4194304] 0 2024-09-21T04:15:07.523 INFO:tasks.workunit.client.0.smithi110.stdout:4/182: dwrite f14 [0,4194304] 0 2024-09-21T04:15:07.530 INFO:tasks.workunit.client.0.smithi110.stdout:7/119: mknod dd/d14/d17/c24 0 2024-09-21T04:15:07.531 INFO:tasks.workunit.client.0.smithi110.stdout:6/47: write d4/f5 [615572,104080] 0 2024-09-21T04:15:07.531 INFO:tasks.workunit.client.0.smithi110.stdout:6/48: readlink - no filename 2024-09-21T04:15:07.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/120: mknod dd/d14/c25 0 2024-09-21T04:15:07.535 INFO:tasks.workunit.client.0.smithi110.stdout:7/121: truncate dd/d14/d17/f1e 230634 0 2024-09-21T04:15:07.535 INFO:tasks.workunit.client.0.smithi110.stdout:7/122: creat dd/d14/d17/f26 x:0 0 0 2024-09-21T04:15:07.535 INFO:tasks.workunit.client.0.smithi110.stdout:7/123: write dd/f18 [1007767,60060] 0 2024-09-21T04:15:07.546 INFO:tasks.workunit.client.0.smithi110.stdout:5/120: dwrite d1/d7/f13 [0,4194304] 0 2024-09-21T04:15:07.548 INFO:tasks.workunit.client.0.smithi110.stdout:5/121: mknod d1/dc/c26 0 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:5/122: fdatasync d1/d7/d8/f1c 0 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:5/123: stat d1/d7/d8 0 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:5/124: chown d1/f6 8 1 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:1/304: dwrite f1 [0,4194304] 0 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:9/29: dwrite f3 [0,4194304] 0 2024-09-21T04:15:07.549 INFO:tasks.workunit.client.0.smithi110.stdout:5/125: creat d1/d22/f27 x:0 0 0 2024-09-21T04:15:07.554 INFO:tasks.workunit.client.0.smithi110.stdout:1/305: symlink d2/d9/d23/d3b/d5e/l60 0 2024-09-21T04:15:07.556 INFO:tasks.workunit.client.0.smithi110.stdout:9/30: creat d7/f8 x:0 0 0 2024-09-21T04:15:07.558 INFO:tasks.workunit.client.0.smithi110.stdout:5/126: creat d1/d7/d18/f28 x:0 0 0 2024-09-21T04:15:07.559 INFO:tasks.workunit.client.0.smithi110.stdout:6/49: dwrite d4/f8 [0,4194304] 0 2024-09-21T04:15:07.565 INFO:tasks.workunit.client.0.smithi110.stdout:5/127: write d1/d7/d8/f19 [1391584,30317] 0 2024-09-21T04:15:07.568 INFO:tasks.workunit.client.0.smithi110.stdout:6/50: write d4/f6 [239573,13477] 0 2024-09-21T04:15:07.573 INFO:tasks.workunit.client.0.smithi110.stdout:2/32: sync 2024-09-21T04:15:07.574 INFO:tasks.workunit.client.0.smithi110.stdout:6/51: mkdir d4/de 0 2024-09-21T04:15:07.576 INFO:tasks.workunit.client.0.smithi110.stdout:2/33: mkdir d2/d3/d7 0 2024-09-21T04:15:07.576 INFO:tasks.workunit.client.0.smithi110.stdout:2/34: creat d2/f8 x:0 0 0 2024-09-21T04:15:07.577 INFO:tasks.workunit.client.0.smithi110.stdout:6/52: link d4/f5 d4/de/ff 0 2024-09-21T04:15:07.577 INFO:tasks.workunit.client.0.smithi110.stdout:6/53: stat d4/fb 0 2024-09-21T04:15:07.580 INFO:tasks.workunit.client.0.smithi110.stdout:6/54: dread f3 [0,4194304] 0 2024-09-21T04:15:07.581 INFO:tasks.workunit.client.0.smithi110.stdout:2/35: rmdir d2/d3/d7 0 2024-09-21T04:15:07.581 INFO:tasks.workunit.client.0.smithi110.stdout:2/36: write d2/f6 [4414343,99302] 0 2024-09-21T04:15:07.583 INFO:tasks.workunit.client.0.smithi110.stdout:6/55: creat d4/de/f10 x:0 0 0 2024-09-21T04:15:07.585 INFO:tasks.workunit.client.0.smithi110.stdout:2/37: mkdir d2/d3/d9 0 2024-09-21T04:15:07.585 INFO:tasks.workunit.client.0.smithi110.stdout:2/38: readlink - no filename 2024-09-21T04:15:07.585 INFO:tasks.workunit.client.0.smithi110.stdout:2/39: stat d2/d3 0 2024-09-21T04:15:07.585 INFO:tasks.workunit.client.0.smithi110.stdout:7/124: rmdir dd 39 2024-09-21T04:15:07.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/125: mknod dd/d14/c27 0 2024-09-21T04:15:07.592 INFO:tasks.workunit.client.0.smithi110.stdout:7/126: creat dd/d14/f28 x:0 0 0 2024-09-21T04:15:07.592 INFO:tasks.workunit.client.0.smithi110.stdout:7/127: write dd/f18 [2079898,5812] 0 2024-09-21T04:15:07.596 INFO:tasks.workunit.client.0.smithi110.stdout:7/128: unlink dd/c13 0 2024-09-21T04:15:07.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/56: dwrite f3 [0,4194304] 0 2024-09-21T04:15:07.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/57: creat d4/de/f11 x:0 0 0 2024-09-21T04:15:07.602 INFO:tasks.workunit.client.0.smithi110.stdout:2/40: dwrite d2/f6 [4194304,4194304] 0 2024-09-21T04:15:07.606 INFO:tasks.workunit.client.0.smithi110.stdout:2/41: mkdir d2/d3/da 0 2024-09-21T04:15:07.606 INFO:tasks.workunit.client.0.smithi110.stdout:2/42: chown d2/f8 2017 1 2024-09-21T04:15:07.608 INFO:tasks.workunit.client.0.smithi110.stdout:6/58: mkdir d4/d12 0 2024-09-21T04:15:07.610 INFO:tasks.workunit.client.0.smithi110.stdout:2/43: symlink d2/d3/da/lb 0 2024-09-21T04:15:07.611 INFO:tasks.workunit.client.0.smithi110.stdout:5/128: rmdir d1/d22 39 2024-09-21T04:15:07.614 INFO:tasks.workunit.client.0.smithi110.stdout:6/59: unlink d4/f9 0 2024-09-21T04:15:07.618 INFO:tasks.workunit.client.0.smithi110.stdout:5/129: dread d1/d7/f13 [0,4194304] 0 2024-09-21T04:15:07.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/60: rename d4/f5 to d4/f13 0 2024-09-21T04:15:07.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/61: getdents d4/d12 0 2024-09-21T04:15:07.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/62: read - d4/fa zero size 2024-09-21T04:15:07.620 INFO:tasks.workunit.client.0.smithi110.stdout:2/44: dwrite d2/f6 [4194304,4194304] 0 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:6/63: mknod d4/de/c14 0 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:6/64: dread - d4/fc zero size 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/130: dread d1/d7/d8/f19 [0,4194304] 0 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/131: fsync d1/d7/d18/f28 0 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/132: chown d1/d7/d8/f21 3505 1 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/133: chown d1/d7/f13 4388 1 2024-09-21T04:15:07.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/134: rename d1/d7/d8 to d1/d7/d8/de/d29 22 2024-09-21T04:15:07.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/135: dread d1/f12 [0,4194304] 0 2024-09-21T04:15:07.627 INFO:tasks.workunit.client.0.smithi110.stdout:2/45: mknod d2/d3/da/cc 0 2024-09-21T04:15:07.629 INFO:tasks.workunit.client.0.smithi110.stdout:6/65: mkdir d4/de/d15 0 2024-09-21T04:15:07.629 INFO:tasks.workunit.client.0.smithi110.stdout:6/66: stat d4/cd 0 2024-09-21T04:15:07.629 INFO:tasks.workunit.client.0.smithi110.stdout:5/136: write d1/f15 [3070587,47140] 0 2024-09-21T04:15:07.630 INFO:tasks.workunit.client.0.smithi110.stdout:5/137: read d1/d7/d8/f19 [2553454,75338] 0 2024-09-21T04:15:07.634 INFO:tasks.workunit.client.0.smithi110.stdout:5/138: unlink d1/d7/c16 0 2024-09-21T04:15:07.645 INFO:tasks.workunit.client.0.smithi110.stdout:6/67: dwrite d4/de/f10 [0,4194304] 0 2024-09-21T04:15:07.651 INFO:tasks.workunit.client.0.smithi110.stdout:7/129: getdents dd 0 2024-09-21T04:15:07.656 INFO:tasks.workunit.client.0.smithi110.stdout:7/130: read f2 [2483101,14875] 0 2024-09-21T04:15:07.671 INFO:tasks.workunit.client.0.smithi110.stdout:0/250: sync 2024-09-21T04:15:07.671 INFO:tasks.workunit.client.0.smithi110.stdout:3/122: sync 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/251: creat d4/dd/d10/d29/d2e/f4b x:0 0 0 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/252: chown d4/dd/d10/d29/d45/l48 160825818 1 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/253: fsync d4/fb 0 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/254: write d4/dd/f38 [578457,27604] 0 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/255: fdatasync d4/d1c/f3c 0 2024-09-21T04:15:07.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/256: fsync d4/f1f 0 2024-09-21T04:15:07.675 INFO:tasks.workunit.client.0.smithi110.stdout:0/257: getdents d4/dd/d10/d29/d45 0 2024-09-21T04:15:07.675 INFO:tasks.workunit.client.0.smithi110.stdout:0/258: truncate d4/dd/d10/d29/d2e/f4b 692695 0 2024-09-21T04:15:07.676 INFO:tasks.workunit.client.0.smithi110.stdout:0/259: symlink d4/dd/d10/d29/l4c 0 2024-09-21T04:15:07.685 INFO:tasks.workunit.client.0.smithi110.stdout:0/260: dread d4/dd/f1b [0,4194304] 0 2024-09-21T04:15:07.685 INFO:tasks.workunit.client.0.smithi110.stdout:0/261: readlink d4/dd/d10/l26 0 2024-09-21T04:15:07.687 INFO:tasks.workunit.client.0.smithi110.stdout:0/262: link d4/dd/d10/d29/d45/l48 d4/l4d 0 2024-09-21T04:15:07.687 INFO:tasks.workunit.client.0.smithi110.stdout:0/263: fsync d4/f1f 0 2024-09-21T04:15:07.735 INFO:tasks.workunit.client.0.smithi110.stdout:2/46: dwrite d2/f6 [0,4194304] 0 2024-09-21T04:15:07.742 INFO:tasks.workunit.client.0.smithi110.stdout:2/47: write d2/f6 [4298019,48822] 0 2024-09-21T04:15:07.742 INFO:tasks.workunit.client.0.smithi110.stdout:2/48: dread - d2/f8 zero size 2024-09-21T04:15:07.742 INFO:tasks.workunit.client.0.smithi110.stdout:2/49: getdents d2/d3/d9 0 2024-09-21T04:15:07.742 INFO:tasks.workunit.client.0.smithi110.stdout:2/50: write d2/f6 [6961472,59119] 0 2024-09-21T04:15:07.743 INFO:tasks.workunit.client.0.smithi110.stdout:2/51: creat d2/d3/fd x:0 0 0 2024-09-21T04:15:07.743 INFO:tasks.workunit.client.0.smithi110.stdout:2/52: chown d2/c5 65084 1 2024-09-21T04:15:07.765 INFO:tasks.workunit.client.0.smithi110.stdout:6/68: dwrite d4/de/f11 [0,4194304] 0 2024-09-21T04:15:07.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/123: dwrite d0/f11 [0,4194304] 0 2024-09-21T04:15:07.766 INFO:tasks.workunit.client.0.smithi110.stdout:7/131: dwrite dd/d14/d17/f26 [0,4194304] 0 2024-09-21T04:15:07.766 INFO:tasks.workunit.client.0.smithi110.stdout:7/132: dread - dd/d14/d17/f19 zero size 2024-09-21T04:15:07.766 INFO:tasks.workunit.client.0.smithi110.stdout:7/133: chown dd/d14/f20 28134 1 2024-09-21T04:15:07.767 INFO:tasks.workunit.client.0.smithi110.stdout:0/264: dwrite d4/f31 [0,4194304] 0 2024-09-21T04:15:07.770 INFO:tasks.workunit.client.0.smithi110.stdout:6/69: rename d4/de/d15 to d4/d12/d16 0 2024-09-21T04:15:07.780 INFO:tasks.workunit.client.0.smithi110.stdout:7/134: symlink dd/d14/d17/l29 0 2024-09-21T04:15:07.781 INFO:tasks.workunit.client.0.smithi110.stdout:7/135: creat dd/d14/f2a x:0 0 0 2024-09-21T04:15:07.790 INFO:tasks.workunit.client.0.smithi110.stdout:6/70: mkdir d4/d12/d17 0 2024-09-21T04:15:07.790 INFO:tasks.workunit.client.0.smithi110.stdout:6/71: chown d4/de/ff 45149047 1 2024-09-21T04:15:07.790 INFO:tasks.workunit.client.0.smithi110.stdout:6/72: chown d4/d12/d16 9 1 2024-09-21T04:15:07.790 INFO:tasks.workunit.client.0.smithi110.stdout:6/73: write f3 [4651774,1412] 0 2024-09-21T04:15:07.790 INFO:tasks.workunit.client.0.smithi110.stdout:3/124: getdents d0/d12/d16/d1a 0 2024-09-21T04:15:07.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/265: mknod d4/c4e 0 2024-09-21T04:15:07.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/266: truncate d4/d1c/f24 1009960 0 2024-09-21T04:15:07.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/267: write d4/d47/d2c/d3a/f4a [777562,128734] 0 2024-09-21T04:15:07.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/268: chown d4/d47/d2c/d3a/f43 932368186 1 2024-09-21T04:15:07.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/269: truncate d4/fb 4297629 0 2024-09-21T04:15:07.795 INFO:tasks.workunit.client.0.smithi110.stdout:6/74: symlink d4/d12/d17/l18 0 2024-09-21T04:15:07.797 INFO:tasks.workunit.client.0.smithi110.stdout:3/125: symlink d0/d12/d16/l2b 0 2024-09-21T04:15:07.799 INFO:tasks.workunit.client.0.smithi110.stdout:7/136: dwrite dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:07.800 INFO:tasks.workunit.client.0.smithi110.stdout:6/75: symlink d4/de/l19 0 2024-09-21T04:15:07.800 INFO:tasks.workunit.client.0.smithi110.stdout:6/76: write d4/fa [563380,26110] 0 2024-09-21T04:15:07.802 INFO:tasks.workunit.client.0.smithi110.stdout:3/126: mknod d0/d12/d16/d1a/c2c 0 2024-09-21T04:15:07.803 INFO:tasks.workunit.client.0.smithi110.stdout:0/270: getdents d4/dd/d10/d29/d2e 0 2024-09-21T04:15:07.804 INFO:tasks.workunit.client.0.smithi110.stdout:0/271: fsync d4/dd/f32 0 2024-09-21T04:15:07.811 INFO:tasks.workunit.client.0.smithi110.stdout:3/127: write d0/fe [2326975,90581] 0 2024-09-21T04:15:07.820 INFO:tasks.workunit.client.0.smithi110.stdout:6/77: symlink d4/l1a 0 2024-09-21T04:15:07.823 INFO:tasks.workunit.client.0.smithi110.stdout:6/78: mknod d4/c1b 0 2024-09-21T04:15:07.828 INFO:tasks.workunit.client.0.smithi110.stdout:3/128: dwrite d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:07.828 INFO:tasks.workunit.client.0.smithi110.stdout:3/129: chown d0/l26 36583 1 2024-09-21T04:15:07.828 INFO:tasks.workunit.client.0.smithi110.stdout:3/130: write d0/d12/d16/d1a/f24 [371242,43040] 0 2024-09-21T04:15:07.830 INFO:tasks.workunit.client.0.smithi110.stdout:3/131: rename d0/d12/d16/f19 to d0/d22/f2d 0 2024-09-21T04:15:07.831 INFO:tasks.workunit.client.0.smithi110.stdout:3/132: write d0/f1d [367495,66527] 0 2024-09-21T04:15:07.833 INFO:tasks.workunit.client.0.smithi110.stdout:3/133: symlink d0/d22/l2e 0 2024-09-21T04:15:07.835 INFO:tasks.workunit.client.0.smithi110.stdout:3/134: rmdir d0/d22 39 2024-09-21T04:15:07.835 INFO:tasks.workunit.client.0.smithi110.stdout:3/135: fdatasync d0/f6 0 2024-09-21T04:15:07.835 INFO:tasks.workunit.client.0.smithi110.stdout:3/136: write d0/f10 [4716651,40298] 0 2024-09-21T04:15:07.889 INFO:tasks.workunit.client.0.smithi110.stdout:2/53: rmdir d2 39 2024-09-21T04:15:07.889 INFO:tasks.workunit.client.0.smithi110.stdout:2/54: stat d2/d3 0 2024-09-21T04:15:07.891 INFO:tasks.workunit.client.0.smithi110.stdout:2/55: unlink d2/d3/da/cc 0 2024-09-21T04:15:07.891 INFO:tasks.workunit.client.0.smithi110.stdout:2/56: fsync d2/f8 0 2024-09-21T04:15:07.891 INFO:tasks.workunit.client.0.smithi110.stdout:2/57: truncate d2/d3/fd 201173 0 2024-09-21T04:15:07.891 INFO:tasks.workunit.client.0.smithi110.stdout:2/58: chown d2/d3/fd 47490808 1 2024-09-21T04:15:07.910 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:07 smithi148 ceph-mon[34957]: pgmap v268: 65 pgs: 65 active+clean; 348 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 22 MiB/s wr, 599 op/s 2024-09-21T04:15:07.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:07 smithi110 ceph-mon[27723]: pgmap v268: 65 pgs: 65 active+clean; 348 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 22 MiB/s wr, 599 op/s 2024-09-21T04:15:08.040 INFO:tasks.workunit.client.0.smithi110.stdout:4/183: sync 2024-09-21T04:15:08.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/117: sync 2024-09-21T04:15:08.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/118: truncate d0/d7/d13/f1e 646092 0 2024-09-21T04:15:08.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/184: truncate d17/d1e/d21/f31 1180501 0 2024-09-21T04:15:08.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/185: chown f8 6717559 1 2024-09-21T04:15:08.043 INFO:tasks.workunit.client.0.smithi110.stdout:8/119: mknod d0/d7/d18/c29 0 2024-09-21T04:15:08.043 INFO:tasks.workunit.client.0.smithi110.stdout:8/120: fsync d0/d7/f1b 0 2024-09-21T04:15:08.052 INFO:tasks.workunit.client.0.smithi110.stdout:7/137: getdents dd/d14 0 2024-09-21T04:15:08.054 INFO:tasks.workunit.client.0.smithi110.stdout:4/186: dwrite fa [4194304,4194304] 0 2024-09-21T04:15:08.056 INFO:tasks.workunit.client.0.smithi110.stdout:6/79: truncate f3 3925023 0 2024-09-21T04:15:08.058 INFO:tasks.workunit.client.0.smithi110.stdout:4/187: symlink d17/d34/l42 0 2024-09-21T04:15:08.060 INFO:tasks.workunit.client.0.smithi110.stdout:4/188: mkdir d17/d34/d3d/d20/d43 0 2024-09-21T04:15:08.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/189: symlink d17/d1e/l44 0 2024-09-21T04:15:08.065 INFO:tasks.workunit.client.0.smithi110.stdout:7/138: dwrite f2 [4194304,4194304] 0 2024-09-21T04:15:08.065 INFO:tasks.workunit.client.0.smithi110.stdout:7/139: dread - dd/d14/d17/f19 zero size 2024-09-21T04:15:08.065 INFO:tasks.workunit.client.0.smithi110.stdout:7/140: truncate dd/d14/d17/f1e 910467 0 2024-09-21T04:15:08.067 INFO:tasks.workunit.client.0.smithi110.stdout:7/141: rmdir dd/d14/d17 39 2024-09-21T04:15:08.067 INFO:tasks.workunit.client.0.smithi110.stdout:7/142: readlink dd/d14/d17/l1f 0 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:7/143: chown dd/d14/c16 3 1 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:5/139: sync 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:9/31: sync 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:1/306: sync 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:9/32: chown c5 118960162 1 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:5/140: chown d1/f6 54 1 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:1/307: fsync d2/d9/d23/f3e 0 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:1/308: write d2/d9/d26/f2d [2298838,96632] 0 2024-09-21T04:15:08.068 INFO:tasks.workunit.client.0.smithi110.stdout:7/144: creat dd/d14/d17/f2b x:0 0 0 2024-09-21T04:15:08.071 INFO:tasks.workunit.client.0.smithi110.stdout:5/141: symlink d1/d7/d8/l2a 0 2024-09-21T04:15:08.072 INFO:tasks.workunit.client.0.smithi110.stdout:1/309: creat d2/d9/d23/d4e/f61 x:0 0 0 2024-09-21T04:15:08.076 INFO:tasks.workunit.client.0.smithi110.stdout:7/145: rename dd/d14/d17/f19 to dd/d1c/f2c 0 2024-09-21T04:15:08.077 INFO:tasks.workunit.client.0.smithi110.stdout:7/146: chown dd/d14/c21 73 1 2024-09-21T04:15:08.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/33: getdents d7 0 2024-09-21T04:15:08.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/34: getdents d7 0 2024-09-21T04:15:08.080 INFO:tasks.workunit.client.0.smithi110.stdout:9/35: mknod d7/c9 0 2024-09-21T04:15:08.081 INFO:tasks.workunit.client.0.smithi110.stdout:1/310: write d2/d9/d23/f29 [3873208,22285] 0 2024-09-21T04:15:08.081 INFO:tasks.workunit.client.0.smithi110.stdout:4/190: dwrite d17/f32 [0,4194304] 0 2024-09-21T04:15:08.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/147: dread dd/f18 [0,4194304] 0 2024-09-21T04:15:08.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/148: truncate dd/d14/d17/f2b 583568 0 2024-09-21T04:15:08.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/311: symlink d2/d9/d26/l62 0 2024-09-21T04:15:08.087 INFO:tasks.workunit.client.0.smithi110.stdout:9/36: read f3 [3193744,12611] 0 2024-09-21T04:15:08.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/191: symlink d17/d34/d3d/l45 0 2024-09-21T04:15:08.094 INFO:tasks.workunit.client.0.smithi110.stdout:1/312: write d2/d9/d23/f38 [211449,114304] 0 2024-09-21T04:15:08.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/142: dwrite d1/d7/d8/de/f24 [0,4194304] 0 2024-09-21T04:15:08.096 INFO:tasks.workunit.client.0.smithi110.stdout:7/149: rename dd/d14/d17/c24 to dd/d14/c2d 0 2024-09-21T04:15:08.096 INFO:tasks.workunit.client.0.smithi110.stdout:7/150: readlink dd/d14/d17/l1f 0 2024-09-21T04:15:08.109 INFO:tasks.workunit.client.0.smithi110.stdout:7/151: unlink dd/c12 0 2024-09-21T04:15:08.109 INFO:tasks.workunit.client.0.smithi110.stdout:7/152: fdatasync dd/d14/f22 0 2024-09-21T04:15:08.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/153: fdatasync f2 0 2024-09-21T04:15:08.110 INFO:tasks.workunit.client.0.smithi110.stdout:9/37: mknod d7/ca 0 2024-09-21T04:15:08.112 INFO:tasks.workunit.client.0.smithi110.stdout:5/143: dread d1/f6 [0,4194304] 0 2024-09-21T04:15:08.120 INFO:tasks.workunit.client.0.smithi110.stdout:9/38: dread f6 [0,4194304] 0 2024-09-21T04:15:08.121 INFO:tasks.workunit.client.0.smithi110.stdout:1/313: dwrite d2/f48 [0,4194304] 0 2024-09-21T04:15:08.125 INFO:tasks.workunit.client.0.smithi110.stdout:4/192: dwrite d17/f3c [0,4194304] 0 2024-09-21T04:15:08.125 INFO:tasks.workunit.client.0.smithi110.stdout:1/314: symlink d2/d9/d49/d5c/l63 0 2024-09-21T04:15:08.126 INFO:tasks.workunit.client.0.smithi110.stdout:9/39: dread f6 [0,4194304] 0 2024-09-21T04:15:08.126 INFO:tasks.workunit.client.0.smithi110.stdout:9/40: stat f3 0 2024-09-21T04:15:08.126 INFO:tasks.workunit.client.0.smithi110.stdout:9/41: read f2 [117646,116523] 0 2024-09-21T04:15:08.130 INFO:tasks.workunit.client.0.smithi110.stdout:4/193: mknod d17/d34/c46 0 2024-09-21T04:15:08.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/42: dread f3 [0,4194304] 0 2024-09-21T04:15:08.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/43: chown f3 6151274 1 2024-09-21T04:15:08.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/44: write f6 [1551298,8510] 0 2024-09-21T04:15:08.130 INFO:tasks.workunit.client.0.smithi110.stdout:5/144: dwrite d1/d7/d8/f19 [0,4194304] 0 2024-09-21T04:15:08.131 INFO:tasks.workunit.client.0.smithi110.stdout:1/315: creat d2/d9/d23/f64 x:0 0 0 2024-09-21T04:15:08.132 INFO:tasks.workunit.client.0.smithi110.stdout:4/194: mknod d17/d34/c47 0 2024-09-21T04:15:08.132 INFO:tasks.workunit.client.0.smithi110.stdout:4/195: creat d17/d1e/f48 x:0 0 0 2024-09-21T04:15:08.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/145: truncate d1/f6 1397966 0 2024-09-21T04:15:08.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/146: write d1/d22/f27 [217186,51831] 0 2024-09-21T04:15:08.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/147: stat d1/d7/d8/de/l1f 0 2024-09-21T04:15:08.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/316: unlink d2/fc 0 2024-09-21T04:15:08.139 INFO:tasks.workunit.client.0.smithi110.stdout:4/196: link d17/f27 d17/d34/d3d/d20/f49 0 2024-09-21T04:15:08.140 INFO:tasks.workunit.client.0.smithi110.stdout:9/45: mkdir d7/db 0 2024-09-21T04:15:08.142 INFO:tasks.workunit.client.0.smithi110.stdout:5/148: creat d1/d7/f2b x:0 0 0 2024-09-21T04:15:08.145 INFO:tasks.workunit.client.0.smithi110.stdout:1/317: symlink d2/d9/d49/l65 0 2024-09-21T04:15:08.147 INFO:tasks.workunit.client.0.smithi110.stdout:4/197: write d17/d34/d3d/d20/f26 [3376085,127955] 0 2024-09-21T04:15:08.148 INFO:tasks.workunit.client.0.smithi110.stdout:4/198: fsync f4 0 2024-09-21T04:15:08.153 INFO:tasks.workunit.client.0.smithi110.stdout:1/318: mknod d2/c66 0 2024-09-21T04:15:08.154 INFO:tasks.workunit.client.0.smithi110.stdout:1/319: creat d2/d9/d49/d4f/f67 x:0 0 0 2024-09-21T04:15:08.161 INFO:tasks.workunit.client.0.smithi110.stdout:1/320: dread d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:08.165 INFO:tasks.workunit.client.0.smithi110.stdout:9/46: dwrite f2 [0,4194304] 0 2024-09-21T04:15:08.165 INFO:tasks.workunit.client.0.smithi110.stdout:9/47: write f3 [4151397,73273] 0 2024-09-21T04:15:08.166 INFO:tasks.workunit.client.0.smithi110.stdout:5/149: dwrite d1/d7/d18/f28 [0,4194304] 0 2024-09-21T04:15:08.172 INFO:tasks.workunit.client.0.smithi110.stdout:5/150: symlink d1/d7/d8/l2c 0 2024-09-21T04:15:08.175 INFO:tasks.workunit.client.0.smithi110.stdout:5/151: rmdir d1/d7/d18 39 2024-09-21T04:15:08.177 INFO:tasks.workunit.client.0.smithi110.stdout:5/152: rename d1/d7/d8/f19 to d1/d7/d8/f2d 0 2024-09-21T04:15:08.177 INFO:tasks.workunit.client.0.smithi110.stdout:5/153: chown d1/d7/d8/f1c 56506950 1 2024-09-21T04:15:08.179 INFO:tasks.workunit.client.0.smithi110.stdout:5/154: unlink d1/dc/l17 0 2024-09-21T04:15:08.183 INFO:tasks.workunit.client.0.smithi110.stdout:1/321: dwrite d2/d9/d23/d4e/f61 [0,4194304] 0 2024-09-21T04:15:08.183 INFO:tasks.workunit.client.0.smithi110.stdout:1/322: write d2/d9/d23/f29 [2122635,128465] 0 2024-09-21T04:15:08.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/323: write d2/f8 [2382076,56339] 0 2024-09-21T04:15:08.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/324: write d2/d9/d26/f4d [894370,84078] 0 2024-09-21T04:15:08.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/325: chown d2/d9/d23/d3b/l46 27858 1 2024-09-21T04:15:08.188 INFO:tasks.workunit.client.0.smithi110.stdout:1/326: rename d2/l3c to d2/d9/d23/d3b/d5e/l68 0 2024-09-21T04:15:08.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/327: mknod d2/d9/d49/d4f/d53/c69 0 2024-09-21T04:15:08.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/328: creat d2/d9/d49/d4f/f6a x:0 0 0 2024-09-21T04:15:08.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/329: write d2/f8 [1985535,51807] 0 2024-09-21T04:15:08.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/330: creat d2/d9/d23/d3b/f6b x:0 0 0 2024-09-21T04:15:08.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/331: creat d2/d9/f6c x:0 0 0 2024-09-21T04:15:08.195 INFO:tasks.workunit.client.0.smithi110.stdout:1/332: symlink d2/d9/d26/l6d 0 2024-09-21T04:15:08.195 INFO:tasks.workunit.client.0.smithi110.stdout:1/333: dread - d2/d9/d49/d4f/f67 zero size 2024-09-21T04:15:08.197 INFO:tasks.workunit.client.0.smithi110.stdout:1/334: unlink d2/fe 0 2024-09-21T04:15:08.199 INFO:tasks.workunit.client.0.smithi110.stdout:1/335: rename d2/d9/d26/c35 to d2/d9/d49/c6e 0 2024-09-21T04:15:08.199 INFO:tasks.workunit.client.0.smithi110.stdout:1/336: truncate d2/d9/d23/f64 355898 0 2024-09-21T04:15:08.203 INFO:tasks.workunit.client.0.smithi110.stdout:1/337: write d2/d9/f1f [2550939,34057] 0 2024-09-21T04:15:08.204 INFO:tasks.workunit.client.0.smithi110.stdout:1/338: creat d2/d9/d26/d3d/f6f x:0 0 0 2024-09-21T04:15:08.210 INFO:tasks.workunit.client.0.smithi110.stdout:1/339: dread d2/d9/d23/f29 [0,4194304] 0 2024-09-21T04:15:08.212 INFO:tasks.workunit.client.0.smithi110.stdout:1/340: mknod d2/d9/d49/d4f/d53/c70 0 2024-09-21T04:15:08.214 INFO:tasks.workunit.client.0.smithi110.stdout:1/341: unlink d2/d9/d23/c36 0 2024-09-21T04:15:08.216 INFO:tasks.workunit.client.0.smithi110.stdout:1/342: rename d2/d9/d26/l62 to d2/d9/d23/d4e/l71 0 2024-09-21T04:15:08.218 INFO:tasks.workunit.client.0.smithi110.stdout:1/343: unlink d2/d9/d23/f44 0 2024-09-21T04:15:08.220 INFO:tasks.workunit.client.0.smithi110.stdout:1/344: mknod d2/d9/d26/c72 0 2024-09-21T04:15:08.220 INFO:tasks.workunit.client.0.smithi110.stdout:1/345: fsync d2/d9/d26/d3d/f43 0 2024-09-21T04:15:08.222 INFO:tasks.workunit.client.0.smithi110.stdout:1/346: mknod d2/d9/d23/d4e/d57/c73 0 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/347: symlink d2/d9/d23/d3b/l74 0 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/348: chown d2/l4b 232957 1 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/349: chown d2/f4 1023090 1 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/350: truncate d2/d9/d23/d3b/f55 768539 0 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/351: dread - d2/d9/d23/d3b/d5e/f5f zero size 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:8/121: rmdir d0/d7 39 2024-09-21T04:15:08.224 INFO:tasks.workunit.client.0.smithi110.stdout:1/352: dread d2/d9/d23/d3b/f55 [0,4194304] 0 2024-09-21T04:15:08.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/353: dread d2/f45 [0,4194304] 0 2024-09-21T04:15:08.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/354: readlink d2/d9/d23/d3b/l74 0 2024-09-21T04:15:08.230 INFO:tasks.workunit.client.0.smithi110.stdout:1/355: dread d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:08.233 INFO:tasks.workunit.client.0.smithi110.stdout:8/122: write d0/d7/f1b [3396819,56848] 0 2024-09-21T04:15:08.236 INFO:tasks.workunit.client.0.smithi110.stdout:8/123: creat d0/d7/f2a x:0 0 0 2024-09-21T04:15:08.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/356: write d2/d9/d23/d4e/f61 [2827991,124571] 0 2024-09-21T04:15:08.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/357: fdatasync d2/d9/d26/d3d/f43 0 2024-09-21T04:15:08.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/358: readlink d2/l12 0 2024-09-21T04:15:08.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/359: creat d2/d9/d23/f75 x:0 0 0 2024-09-21T04:15:08.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/360: mknod d2/d9/d26/c76 0 2024-09-21T04:15:08.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/361: fdatasync d2/d9/d23/f40 0 2024-09-21T04:15:08.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/362: write d2/d9/d49/d4f/f67 [906979,90600] 0 2024-09-21T04:15:08.245 INFO:tasks.workunit.client.0.smithi110.stdout:1/363: dread d2/f4 [0,4194304] 0 2024-09-21T04:15:08.247 INFO:tasks.workunit.client.0.smithi110.stdout:1/364: dread d2/d9/f1a [0,4194304] 0 2024-09-21T04:15:08.250 INFO:tasks.workunit.client.0.smithi110.stdout:1/365: creat d2/d9/d49/f77 x:0 0 0 2024-09-21T04:15:08.254 INFO:tasks.workunit.client.0.smithi110.stdout:8/124: dwrite d0/d7/d18/f28 [0,4194304] 0 2024-09-21T04:15:08.257 INFO:tasks.workunit.client.0.smithi110.stdout:8/125: dread d0/d7/f20 [0,4194304] 0 2024-09-21T04:15:08.257 INFO:tasks.workunit.client.0.smithi110.stdout:8/126: fdatasync d0/d7/f1b 0 2024-09-21T04:15:08.257 INFO:tasks.workunit.client.0.smithi110.stdout:8/127: dread d0/d7/d13/f1e [0,4194304] 0 2024-09-21T04:15:08.268 INFO:tasks.workunit.client.0.smithi110.stdout:7/154: getdents dd/d14/d17 0 2024-09-21T04:15:08.269 INFO:tasks.workunit.client.0.smithi110.stdout:7/155: write dd/d14/d17/f1e [826632,123556] 0 2024-09-21T04:15:08.271 INFO:tasks.workunit.client.0.smithi110.stdout:8/128: rmdir d0 39 2024-09-21T04:15:08.271 INFO:tasks.workunit.client.0.smithi110.stdout:8/129: dread - d0/d7/d13/d1c/f1d zero size 2024-09-21T04:15:08.273 INFO:tasks.workunit.client.0.smithi110.stdout:1/366: dwrite d2/d9/d23/d4e/f61 [0,4194304] 0 2024-09-21T04:15:08.273 INFO:tasks.workunit.client.0.smithi110.stdout:7/156: rename f2 to dd/f2e 0 2024-09-21T04:15:08.273 INFO:tasks.workunit.client.0.smithi110.stdout:1/367: stat d2/d9 0 2024-09-21T04:15:08.274 INFO:tasks.workunit.client.0.smithi110.stdout:0/272: sync 2024-09-21T04:15:08.274 INFO:tasks.workunit.client.0.smithi110.stdout:8/130: mknod d0/d26/c2b 0 2024-09-21T04:15:08.276 INFO:tasks.workunit.client.0.smithi110.stdout:7/157: symlink dd/d14/l2f 0 2024-09-21T04:15:08.276 INFO:tasks.workunit.client.0.smithi110.stdout:7/158: write dd/d14/f22 [943569,42158] 0 2024-09-21T04:15:08.276 INFO:tasks.workunit.client.0.smithi110.stdout:7/159: fdatasync dd/d14/f20 0 2024-09-21T04:15:08.277 INFO:tasks.workunit.client.0.smithi110.stdout:7/160: write dd/d14/d17/f2b [154188,95454] 0 2024-09-21T04:15:08.277 INFO:tasks.workunit.client.0.smithi110.stdout:7/161: dread - dd/f1a zero size 2024-09-21T04:15:08.277 INFO:tasks.workunit.client.0.smithi110.stdout:1/368: symlink d2/d9/d49/d4f/d53/l78 0 2024-09-21T04:15:08.278 INFO:tasks.workunit.client.0.smithi110.stdout:0/273: mkdir d4/dd/d10/d4f 0 2024-09-21T04:15:08.281 INFO:tasks.workunit.client.0.smithi110.stdout:8/131: symlink d0/d7/d13/l2c 0 2024-09-21T04:15:08.281 INFO:tasks.workunit.client.0.smithi110.stdout:8/132: chown d0/d7/d13/f1e 161494823 1 2024-09-21T04:15:08.284 INFO:tasks.workunit.client.0.smithi110.stdout:8/133: read d0/d7/d18/f1f [538363,26605] 0 2024-09-21T04:15:08.284 INFO:tasks.workunit.client.0.smithi110.stdout:7/162: read dd/d14/f20 [805109,130156] 0 2024-09-21T04:15:08.284 INFO:tasks.workunit.client.0.smithi110.stdout:7/163: write dd/d1c/f2c [193129,55346] 0 2024-09-21T04:15:08.285 INFO:tasks.workunit.client.0.smithi110.stdout:0/274: unlink d4/d47/d2c/d3a/f4a 0 2024-09-21T04:15:08.285 INFO:tasks.workunit.client.0.smithi110.stdout:0/275: write d4/d47/f40 [572880,34129] 0 2024-09-21T04:15:08.286 INFO:tasks.workunit.client.0.smithi110.stdout:0/276: write d4/d1c/f24 [1914551,120392] 0 2024-09-21T04:15:08.286 INFO:tasks.workunit.client.0.smithi110.stdout:0/277: dread d4/d47/d2c/d3a/f43 [0,4194304] 0 2024-09-21T04:15:08.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/134: unlink d0/d7/l10 0 2024-09-21T04:15:08.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/164: rename dd/d14/c25 to dd/d14/d17/c30 0 2024-09-21T04:15:08.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/165: chown dd/f1a 0 1 2024-09-21T04:15:08.300 INFO:tasks.workunit.client.0.smithi110.stdout:1/369: dwrite d2/d9/d23/f64 [0,4194304] 0 2024-09-21T04:15:08.304 INFO:tasks.workunit.client.0.smithi110.stdout:7/166: fdatasync dd/d14/f22 0 2024-09-21T04:15:08.304 INFO:tasks.workunit.client.0.smithi110.stdout:0/278: write d4/dd/d10/f13 [186913,1136] 0 2024-09-21T04:15:08.307 INFO:tasks.workunit.client.0.smithi110.stdout:1/370: rename d2/d9/l1d to d2/d9/d23/d3b/l79 0 2024-09-21T04:15:08.308 INFO:tasks.workunit.client.0.smithi110.stdout:1/371: chown d2/d9/d23/d4e/d57 7816 1 2024-09-21T04:15:08.308 INFO:tasks.workunit.client.0.smithi110.stdout:1/372: chown d2/d9/d49/d5c 6932 1 2024-09-21T04:15:08.308 INFO:tasks.workunit.client.0.smithi110.stdout:1/373: write d2/d9/d23/d3b/f55 [664732,32565] 0 2024-09-21T04:15:08.316 INFO:tasks.workunit.client.0.smithi110.stdout:0/279: dread d4/d1c/f25 [0,4194304] 0 2024-09-21T04:15:08.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/135: dwrite d0/f11 [0,4194304] 0 2024-09-21T04:15:08.320 INFO:tasks.workunit.client.0.smithi110.stdout:1/374: mkdir d2/d9/d49/d7a 0 2024-09-21T04:15:08.320 INFO:tasks.workunit.client.0.smithi110.stdout:1/375: chown d2/d9/f1f 228750 1 2024-09-21T04:15:08.320 INFO:tasks.workunit.client.0.smithi110.stdout:7/167: dwrite dd/d14/d17/f2b [0,4194304] 0 2024-09-21T04:15:08.323 INFO:tasks.workunit.client.0.smithi110.stdout:3/137: sync 2024-09-21T04:15:08.323 INFO:tasks.workunit.client.0.smithi110.stdout:8/136: rmdir d0/d7/d18 39 2024-09-21T04:15:08.324 INFO:tasks.workunit.client.0.smithi110.stdout:8/137: fsync d0/d7/d18/f17 0 2024-09-21T04:15:08.325 INFO:tasks.workunit.client.0.smithi110.stdout:0/280: creat d4/dd/d10/f50 x:0 0 0 2024-09-21T04:15:08.329 INFO:tasks.workunit.client.0.smithi110.stdout:3/138: truncate d0/f6 1459858 0 2024-09-21T04:15:08.329 INFO:tasks.workunit.client.0.smithi110.stdout:3/139: fsync d0/f11 0 2024-09-21T04:15:08.329 INFO:tasks.workunit.client.0.smithi110.stdout:3/140: fdatasync d0/d12/d16/f25 0 2024-09-21T04:15:08.330 INFO:tasks.workunit.client.0.smithi110.stdout:8/138: creat d0/d7/d13/f2d x:0 0 0 2024-09-21T04:15:08.332 INFO:tasks.workunit.client.0.smithi110.stdout:3/141: truncate d0/d12/f13 2017706 0 2024-09-21T04:15:08.332 INFO:tasks.workunit.client.0.smithi110.stdout:3/142: write d0/d12/f23 [2491626,128409] 0 2024-09-21T04:15:08.337 INFO:tasks.workunit.client.0.smithi110.stdout:3/143: creat d0/d22/d29/f2f x:0 0 0 2024-09-21T04:15:08.338 INFO:tasks.workunit.client.0.smithi110.stdout:8/139: symlink d0/l2e 0 2024-09-21T04:15:08.340 INFO:tasks.workunit.client.0.smithi110.stdout:3/144: creat d0/d12/d2a/f30 x:0 0 0 2024-09-21T04:15:08.342 INFO:tasks.workunit.client.0.smithi110.stdout:3/145: link d0/f11 d0/d12/d2a/f31 0 2024-09-21T04:15:08.342 INFO:tasks.workunit.client.0.smithi110.stdout:3/146: creat d0/d12/f32 x:0 0 0 2024-09-21T04:15:08.343 INFO:tasks.workunit.client.0.smithi110.stdout:3/147: chown d0/d12 480318674 1 2024-09-21T04:15:08.346 INFO:tasks.workunit.client.0.smithi110.stdout:3/148: rename d0/c8 to d0/d12/d2a/c33 0 2024-09-21T04:15:08.346 INFO:tasks.workunit.client.0.smithi110.stdout:3/149: readlink d0/l21 0 2024-09-21T04:15:08.348 INFO:tasks.workunit.client.0.smithi110.stdout:7/168: dwrite dd/d14/f28 [0,4194304] 0 2024-09-21T04:15:08.349 INFO:tasks.workunit.client.0.smithi110.stdout:3/150: link d0/f7 d0/d12/d16/f34 0 2024-09-21T04:15:08.355 INFO:tasks.workunit.client.0.smithi110.stdout:0/281: dwrite d4/dd/fe [0,4194304] 0 2024-09-21T04:15:08.356 INFO:tasks.workunit.client.0.smithi110.stdout:0/282: creat d4/d47/d2c/d3a/f51 x:0 0 0 2024-09-21T04:15:08.359 INFO:tasks.workunit.client.0.smithi110.stdout:8/140: dwrite d0/d7/f20 [0,4194304] 0 2024-09-21T04:15:08.359 INFO:tasks.workunit.client.0.smithi110.stdout:0/283: unlink d4/dd/d10/f2d 0 2024-09-21T04:15:08.359 INFO:tasks.workunit.client.0.smithi110.stdout:0/284: chown d4/l37 20 1 2024-09-21T04:15:08.364 INFO:tasks.workunit.client.0.smithi110.stdout:0/285: mknod d4/dd/d10/d29/d45/c52 0 2024-09-21T04:15:08.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/141: dread d0/d7/f20 [0,4194304] 0 2024-09-21T04:15:08.371 INFO:tasks.workunit.client.0.smithi110.stdout:0/286: dread d4/dd/d10/d29/d2e/f36 [0,4194304] 0 2024-09-21T04:15:08.372 INFO:tasks.workunit.client.0.smithi110.stdout:8/142: mknod d0/d7/d23/c2f 0 2024-09-21T04:15:08.376 INFO:tasks.workunit.client.0.smithi110.stdout:0/287: unlink d4/c19 0 2024-09-21T04:15:08.377 INFO:tasks.workunit.client.0.smithi110.stdout:1/376: getdents d2 0 2024-09-21T04:15:08.378 INFO:tasks.workunit.client.0.smithi110.stdout:3/151: dwrite d0/d22/f2d [0,4194304] 0 2024-09-21T04:15:08.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/143: mkdir d0/d30 0 2024-09-21T04:15:08.382 INFO:tasks.workunit.client.0.smithi110.stdout:3/152: link d0/d12/d2a/f30 d0/d12/d16/d1a/f35 0 2024-09-21T04:15:08.382 INFO:tasks.workunit.client.0.smithi110.stdout:3/153: chown d0/d12/d2a/f30 113 1 2024-09-21T04:15:08.382 INFO:tasks.workunit.client.0.smithi110.stdout:3/154: dread - d0/d22/d29/f2f zero size 2024-09-21T04:15:08.386 INFO:tasks.workunit.client.0.smithi110.stdout:8/144: creat d0/d30/f31 x:0 0 0 2024-09-21T04:15:08.386 INFO:tasks.workunit.client.0.smithi110.stdout:8/145: readlink d0/d7/d13/l2c 0 2024-09-21T04:15:08.386 INFO:tasks.workunit.client.0.smithi110.stdout:3/155: mknod d0/d12/d16/c36 0 2024-09-21T04:15:08.388 INFO:tasks.workunit.client.0.smithi110.stdout:7/169: dwrite dd/d14/f28 [0,4194304] 0 2024-09-21T04:15:08.388 INFO:tasks.workunit.client.0.smithi110.stdout:0/288: dread d4/f31 [0,4194304] 0 2024-09-21T04:15:08.389 INFO:tasks.workunit.client.0.smithi110.stdout:0/289: fdatasync d4/dd/d10/d29/d2e/f4b 0 2024-09-21T04:15:08.392 INFO:tasks.workunit.client.0.smithi110.stdout:8/146: write d0/f11 [260076,43179] 0 2024-09-21T04:15:08.397 INFO:tasks.workunit.client.0.smithi110.stdout:7/170: creat dd/f31 x:0 0 0 2024-09-21T04:15:08.397 INFO:tasks.workunit.client.0.smithi110.stdout:7/171: fdatasync dd/f2e 0 2024-09-21T04:15:08.400 INFO:tasks.workunit.client.0.smithi110.stdout:0/290: mkdir d4/dd/d53 0 2024-09-21T04:15:08.415 INFO:tasks.workunit.client.0.smithi110.stdout:7/172: write dd/d14/d17/f26 [313061,118638] 0 2024-09-21T04:15:08.415 INFO:tasks.workunit.client.0.smithi110.stdout:7/173: chown dd/d14/d17/f2b 24758015 1 2024-09-21T04:15:08.418 INFO:tasks.workunit.client.0.smithi110.stdout:0/291: dwrite d4/dd/d10/f2a [0,4194304] 0 2024-09-21T04:15:08.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/147: dwrite d0/fb [4194304,4194304] 0 2024-09-21T04:15:08.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/174: creat dd/d1c/f32 x:0 0 0 2024-09-21T04:15:08.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/148: chown d0/d7/f20 39338695 1 2024-09-21T04:15:08.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/149: chown d0/d7/c21 269 1 2024-09-21T04:15:08.425 INFO:tasks.workunit.client.0.smithi110.stdout:0/292: symlink d4/dd/l54 0 2024-09-21T04:15:08.429 INFO:tasks.workunit.client.0.smithi110.stdout:1/377: chown d2/d9/d23/d3b/d5e 22 1 2024-09-21T04:15:08.443 INFO:tasks.workunit.client.0.smithi110.stdout:1/378: read d2/d9/d23/d4e/f61 [1524663,95871] 0 2024-09-21T04:15:08.444 INFO:tasks.workunit.client.0.smithi110.stdout:8/150: dwrite d0/d7/f1a [0,4194304] 0 2024-09-21T04:15:08.445 INFO:tasks.workunit.client.0.smithi110.stdout:8/151: truncate d0/fb 8920982 0 2024-09-21T04:15:08.445 INFO:tasks.workunit.client.0.smithi110.stdout:7/175: dwrite dd/d14/d17/f26 [0,4194304] 0 2024-09-21T04:15:08.450 INFO:tasks.workunit.client.0.smithi110.stdout:8/152: mkdir d0/d26/d32 0 2024-09-21T04:15:08.450 INFO:tasks.workunit.client.0.smithi110.stdout:2/59: sync 2024-09-21T04:15:08.450 INFO:tasks.workunit.client.0.smithi110.stdout:6/80: sync 2024-09-21T04:15:08.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/176: unlink dd/d14/d17/f1e 0 2024-09-21T04:15:08.456 INFO:tasks.workunit.client.0.smithi110.stdout:8/153: creat d0/d7/d23/f33 x:0 0 0 2024-09-21T04:15:08.456 INFO:tasks.workunit.client.0.smithi110.stdout:8/154: readlink d0/d7/le 0 2024-09-21T04:15:08.458 INFO:tasks.workunit.client.0.smithi110.stdout:2/60: mkdir d2/d3/da/de 0 2024-09-21T04:15:08.461 INFO:tasks.workunit.client.0.smithi110.stdout:7/177: mknod dd/d1c/c33 0 2024-09-21T04:15:08.463 INFO:tasks.workunit.client.0.smithi110.stdout:2/61: unlink d2/f6 0 2024-09-21T04:15:08.465 INFO:tasks.workunit.client.0.smithi110.stdout:6/81: getdents d4/de 0 2024-09-21T04:15:08.467 INFO:tasks.workunit.client.0.smithi110.stdout:7/178: unlink dd/d14/d17/f26 0 2024-09-21T04:15:08.470 INFO:tasks.workunit.client.0.smithi110.stdout:6/82: rename c1 to d4/d12/d16/c1c 0 2024-09-21T04:15:08.473 INFO:tasks.workunit.client.0.smithi110.stdout:6/83: link d4/f13 d4/d12/f1d 0 2024-09-21T04:15:08.476 INFO:tasks.workunit.client.0.smithi110.stdout:6/84: mknod d4/d12/d16/c1e 0 2024-09-21T04:15:08.478 INFO:tasks.workunit.client.0.smithi110.stdout:6/85: read d4/f13 [333554,46483] 0 2024-09-21T04:15:08.478 INFO:tasks.workunit.client.0.smithi110.stdout:6/86: chown d4/d12/d16/c1e 104053 1 2024-09-21T04:15:08.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/87: creat d4/de/f1f x:0 0 0 2024-09-21T04:15:08.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/155: dwrite d0/d7/d13/d1c/f1d [0,4194304] 0 2024-09-21T04:15:08.481 INFO:tasks.workunit.client.0.smithi110.stdout:2/62: dwrite d2/d3/fd [0,4194304] 0 2024-09-21T04:15:08.481 INFO:tasks.workunit.client.0.smithi110.stdout:6/88: rmdir d4/d12/d17 39 2024-09-21T04:15:08.482 INFO:tasks.workunit.client.0.smithi110.stdout:6/89: fdatasync d4/fb 0 2024-09-21T04:15:08.482 INFO:tasks.workunit.client.0.smithi110.stdout:6/90: chown d4/d12/d16/c1e 0 1 2024-09-21T04:15:08.489 INFO:tasks.workunit.client.0.smithi110.stdout:7/179: dwrite dd/f31 [0,4194304] 0 2024-09-21T04:15:08.493 INFO:tasks.workunit.client.0.smithi110.stdout:6/91: getdents d4/d12/d17 0 2024-09-21T04:15:08.495 INFO:tasks.workunit.client.0.smithi110.stdout:8/156: dwrite d0/d7/d23/f33 [0,4194304] 0 2024-09-21T04:15:08.499 INFO:tasks.workunit.client.0.smithi110.stdout:8/157: truncate d0/d7/d13/f1e 10937 0 2024-09-21T04:15:08.501 INFO:tasks.workunit.client.0.smithi110.stdout:8/158: mkdir d0/d30/d34 0 2024-09-21T04:15:08.513 INFO:tasks.workunit.client.0.smithi110.stdout:6/92: dwrite d4/de/f1f [0,4194304] 0 2024-09-21T04:15:08.517 INFO:tasks.workunit.client.0.smithi110.stdout:6/93: getdents d4 0 2024-09-21T04:15:08.519 INFO:tasks.workunit.client.0.smithi110.stdout:6/94: mknod d4/d12/d16/c20 0 2024-09-21T04:15:08.520 INFO:tasks.workunit.client.0.smithi110.stdout:6/95: mknod d4/d12/c21 0 2024-09-21T04:15:08.523 INFO:tasks.workunit.client.0.smithi110.stdout:3/156: dread d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:08.523 INFO:tasks.workunit.client.0.smithi110.stdout:3/157: dread - d0/d22/d29/f2f zero size 2024-09-21T04:15:08.523 INFO:tasks.workunit.client.0.smithi110.stdout:3/158: truncate d0/d22/d29/f2f 649757 0 2024-09-21T04:15:08.524 INFO:tasks.workunit.client.0.smithi110.stdout:6/96: dread d4/f13 [0,4194304] 0 2024-09-21T04:15:08.524 INFO:tasks.workunit.client.0.smithi110.stdout:6/97: readlink d4/de/l19 0 2024-09-21T04:15:08.524 INFO:tasks.workunit.client.0.smithi110.stdout:6/98: stat f3 0 2024-09-21T04:15:08.525 INFO:tasks.workunit.client.0.smithi110.stdout:3/159: mknod d0/d12/d16/c37 0 2024-09-21T04:15:08.525 INFO:tasks.workunit.client.0.smithi110.stdout:3/160: stat d0/f6 0 2024-09-21T04:15:08.527 INFO:tasks.workunit.client.0.smithi110.stdout:6/99: symlink d4/de/l22 0 2024-09-21T04:15:08.529 INFO:tasks.workunit.client.0.smithi110.stdout:3/161: rename d0/d12/d2a/c33 to d0/d12/c38 0 2024-09-21T04:15:08.532 INFO:tasks.workunit.client.0.smithi110.stdout:3/162: dread d0/d12/f13 [0,4194304] 0 2024-09-21T04:15:08.532 INFO:tasks.workunit.client.0.smithi110.stdout:3/163: chown d0/d12/d16 0 1 2024-09-21T04:15:08.532 INFO:tasks.workunit.client.0.smithi110.stdout:3/164: fsync d0/d12/d16/f34 0 2024-09-21T04:15:08.533 INFO:tasks.workunit.client.0.smithi110.stdout:6/100: write f3 [2636621,6647] 0 2024-09-21T04:15:08.539 INFO:tasks.workunit.client.0.smithi110.stdout:6/101: mknod d4/d12/d16/c23 0 2024-09-21T04:15:08.539 INFO:tasks.workunit.client.0.smithi110.stdout:6/102: readlink d4/l1a 0 2024-09-21T04:15:08.539 INFO:tasks.workunit.client.0.smithi110.stdout:6/103: chown d4/fa 1612029 1 2024-09-21T04:15:08.539 INFO:tasks.workunit.client.0.smithi110.stdout:6/104: chown d4/de/l19 1459875 1 2024-09-21T04:15:08.545 INFO:tasks.workunit.client.0.smithi110.stdout:3/165: dwrite d0/f28 [0,4194304] 0 2024-09-21T04:15:08.567 INFO:tasks.workunit.client.0.smithi110.stdout:4/199: sync 2024-09-21T04:15:08.567 INFO:tasks.workunit.client.0.smithi110.stdout:5/155: sync 2024-09-21T04:15:08.567 INFO:tasks.workunit.client.0.smithi110.stdout:9/48: sync 2024-09-21T04:15:08.567 INFO:tasks.workunit.client.0.smithi110.stdout:4/200: creat d17/d34/d3d/f4a x:0 0 0 2024-09-21T04:15:08.569 INFO:tasks.workunit.client.0.smithi110.stdout:5/156: symlink d1/d7/d8/l2e 0 2024-09-21T04:15:08.569 INFO:tasks.workunit.client.0.smithi110.stdout:5/157: fsync d1/d7/d8/de/f24 0 2024-09-21T04:15:08.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/158: rename d1/f12 to d1/d7/d8/de/d1e/f2f 0 2024-09-21T04:15:08.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/159: truncate d1/d7/d8/de/d1e/f2f 1336479 0 2024-09-21T04:15:08.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/160: creat d1/dc/f30 x:0 0 0 2024-09-21T04:15:08.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/161: chown d1/d7/d8/l2a 14258172 1 2024-09-21T04:15:08.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/49: dwrite f2 [0,4194304] 0 2024-09-21T04:15:08.587 INFO:tasks.workunit.client.0.smithi110.stdout:9/50: symlink d7/db/lc 0 2024-09-21T04:15:08.589 INFO:tasks.workunit.client.0.smithi110.stdout:9/51: symlink d7/db/ld 0 2024-09-21T04:15:08.592 INFO:tasks.workunit.client.0.smithi110.stdout:4/201: dwrite d17/d1e/d21/f2e [0,4194304] 0 2024-09-21T04:15:08.594 INFO:tasks.workunit.client.0.smithi110.stdout:5/162: dwrite d1/d7/d8/f21 [0,4194304] 0 2024-09-21T04:15:08.595 INFO:tasks.workunit.client.0.smithi110.stdout:4/202: getdents d17 0 2024-09-21T04:15:08.599 INFO:tasks.workunit.client.0.smithi110.stdout:5/163: rename d1/d7/d8/de/d1e/l23 to d1/d7/d8/l31 0 2024-09-21T04:15:08.615 INFO:tasks.workunit.client.0.smithi110.stdout:4/203: dwrite d17/d1e/f33 [0,4194304] 0 2024-09-21T04:15:08.615 INFO:tasks.workunit.client.0.smithi110.stdout:5/164: dwrite d1/f15 [4194304,4194304] 0 2024-09-21T04:15:08.615 INFO:tasks.workunit.client.0.smithi110.stdout:4/204: fdatasync d17/f1a 0 2024-09-21T04:15:08.620 INFO:tasks.workunit.client.0.smithi110.stdout:4/205: symlink d17/d34/d3d/d20/l4b 0 2024-09-21T04:15:08.627 INFO:tasks.workunit.client.0.smithi110.stdout:4/206: dread f8 [4194304,4194304] 0 2024-09-21T04:15:08.627 INFO:tasks.workunit.client.0.smithi110.stdout:4/207: creat d17/d1e/f4c x:0 0 0 2024-09-21T04:15:08.627 INFO:tasks.workunit.client.0.smithi110.stdout:4/208: creat d17/d1e/d21/f4d x:0 0 0 2024-09-21T04:15:08.634 INFO:tasks.workunit.client.0.smithi110.stdout:4/209: rename d17/d34/d3d/c28 to d17/d1e/d21/c4e 0 2024-09-21T04:15:08.637 INFO:tasks.workunit.client.0.smithi110.stdout:4/210: symlink d17/d1e/d21/l4f 0 2024-09-21T04:15:08.644 INFO:tasks.workunit.client.0.smithi110.stdout:4/211: dread f8 [0,4194304] 0 2024-09-21T04:15:08.644 INFO:tasks.workunit.client.0.smithi110.stdout:4/212: getdents d17/d34/d3d/d20/d43 0 2024-09-21T04:15:08.646 INFO:tasks.workunit.client.0.smithi110.stdout:4/213: truncate fa 2088195 0 2024-09-21T04:15:08.649 INFO:tasks.workunit.client.0.smithi110.stdout:8/159: write d0/d7/d23/f33 [4694818,19525] 0 2024-09-21T04:15:08.650 INFO:tasks.workunit.client.0.smithi110.stdout:8/160: creat d0/d7/d18/f35 x:0 0 0 2024-09-21T04:15:08.651 INFO:tasks.workunit.client.0.smithi110.stdout:8/161: link d0/d30/f31 d0/d7/f36 0 2024-09-21T04:15:08.652 INFO:tasks.workunit.client.0.smithi110.stdout:8/162: dread - d0/d7/d18/f35 zero size 2024-09-21T04:15:08.655 INFO:tasks.workunit.client.0.smithi110.stdout:4/214: dread d17/d34/f37 [0,4194304] 0 2024-09-21T04:15:08.658 INFO:tasks.workunit.client.0.smithi110.stdout:4/215: dread d17/d1e/d21/f31 [0,4194304] 0 2024-09-21T04:15:08.661 INFO:tasks.workunit.client.0.smithi110.stdout:3/166: rmdir d0/d12/d2a 39 2024-09-21T04:15:08.663 INFO:tasks.workunit.client.0.smithi110.stdout:8/163: write d0/d7/d13/d1c/f1d [2108794,49875] 0 2024-09-21T04:15:08.663 INFO:tasks.workunit.client.0.smithi110.stdout:8/164: truncate d0/d7/fd 5235645 0 2024-09-21T04:15:08.663 INFO:tasks.workunit.client.0.smithi110.stdout:8/165: creat d0/d7/d13/f37 x:0 0 0 2024-09-21T04:15:08.678 INFO:tasks.workunit.client.0.smithi110.stdout:4/216: dread f14 [0,4194304] 0 2024-09-21T04:15:08.679 INFO:tasks.workunit.client.0.smithi110.stdout:4/217: creat d17/f50 x:0 0 0 2024-09-21T04:15:08.688 INFO:tasks.workunit.client.0.smithi110.stdout:3/167: dwrite d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:08.688 INFO:tasks.workunit.client.0.smithi110.stdout:3/168: read d0/f7 [276157,103522] 0 2024-09-21T04:15:08.688 INFO:tasks.workunit.client.0.smithi110.stdout:3/169: fsync d0/d22/d29/f2f 0 2024-09-21T04:15:08.688 INFO:tasks.workunit.client.0.smithi110.stdout:8/166: dwrite d0/d7/d13/f2d [0,4194304] 0 2024-09-21T04:15:08.690 INFO:tasks.workunit.client.0.smithi110.stdout:3/170: link d0/f10 d0/d12/d16/d1a/f39 0 2024-09-21T04:15:08.693 INFO:tasks.workunit.client.0.smithi110.stdout:8/167: link d0/d7/le d0/d30/l38 0 2024-09-21T04:15:08.693 INFO:tasks.workunit.client.0.smithi110.stdout:8/168: fsync d0/f11 0 2024-09-21T04:15:08.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/171: mkdir d0/d3a 0 2024-09-21T04:15:08.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/172: chown d0/d12/d2a/f31 359710099 1 2024-09-21T04:15:08.698 INFO:tasks.workunit.client.0.smithi110.stdout:8/169: mknod d0/d7/c39 0 2024-09-21T04:15:08.698 INFO:tasks.workunit.client.0.smithi110.stdout:8/170: rename d0 to d0/d30/d3a 22 2024-09-21T04:15:08.698 INFO:tasks.workunit.client.0.smithi110.stdout:8/171: creat d0/d30/f3b x:0 0 0 2024-09-21T04:15:08.702 INFO:tasks.workunit.client.0.smithi110.stdout:8/172: rename d0/d7/d18/l22 to d0/d7/l3c 0 2024-09-21T04:15:08.708 INFO:tasks.workunit.client.0.smithi110.stdout:3/173: dwrite d0/d22/d29/f2f [0,4194304] 0 2024-09-21T04:15:08.712 INFO:tasks.workunit.client.0.smithi110.stdout:3/174: unlink d0/d12/d16/d1a/f24 0 2024-09-21T04:15:08.715 INFO:tasks.workunit.client.0.smithi110.stdout:3/175: dread d0/f6 [0,4194304] 0 2024-09-21T04:15:08.716 INFO:tasks.workunit.client.0.smithi110.stdout:3/176: unlink d0/d12/f13 0 2024-09-21T04:15:08.720 INFO:tasks.workunit.client.0.smithi110.stdout:8/173: dwrite d0/fb [0,4194304] 0 2024-09-21T04:15:08.721 INFO:tasks.workunit.client.0.smithi110.stdout:8/174: creat d0/f3d x:0 0 0 2024-09-21T04:15:08.724 INFO:tasks.workunit.client.0.smithi110.stdout:3/177: write d0/f10 [911407,75523] 0 2024-09-21T04:15:08.724 INFO:tasks.workunit.client.0.smithi110.stdout:3/178: read d0/fe [2345514,47495] 0 2024-09-21T04:15:08.725 INFO:tasks.workunit.client.0.smithi110.stdout:3/179: creat d0/d12/d16/d1a/f3b x:0 0 0 2024-09-21T04:15:08.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/180: mkdir d0/d12/d3c 0 2024-09-21T04:15:08.727 INFO:tasks.workunit.client.0.smithi110.stdout:3/181: truncate d0/fe 335805 0 2024-09-21T04:15:08.727 INFO:tasks.workunit.client.0.smithi110.stdout:3/182: chown d0/d12/d16/d1a/f3b 23 1 2024-09-21T04:15:08.728 INFO:tasks.workunit.client.0.smithi110.stdout:4/218: truncate f8 189812 0 2024-09-21T04:15:08.728 INFO:tasks.workunit.client.0.smithi110.stdout:8/175: rmdir d0 39 2024-09-21T04:15:08.728 INFO:tasks.workunit.client.0.smithi110.stdout:8/176: write d0/f3d [325297,113602] 0 2024-09-21T04:15:08.728 INFO:tasks.workunit.client.0.smithi110.stdout:3/183: mknod d0/d12/d3c/c3d 0 2024-09-21T04:15:08.730 INFO:tasks.workunit.client.0.smithi110.stdout:8/177: rename d0/d26 to d0/d30/d34/d3e 0 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:0/57: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:1/59: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:2/74: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:3/17: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:7/28: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:5/8: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:8/16: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:9/34: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:4/10: sync 2024-09-21T04:15:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:6/28: sync 2024-09-21T04:15:08.734 INFO:tasks.workunit.client.1.smithi148.stdout:1/60: rename f4 to db/f15 0 2024-09-21T04:15:08.736 INFO:tasks.workunit.client.1.smithi148.stdout:0/58: link d5/d6/d7/cd d5/d6/d7/c10 0 2024-09-21T04:15:08.736 INFO:tasks.workunit.client.0.smithi110.stdout:4/219: write f8 [920891,50248] 0 2024-09-21T04:15:08.737 INFO:tasks.workunit.client.1.smithi148.stdout:5/9: mknod d3/c6 0 2024-09-21T04:15:08.737 INFO:tasks.workunit.client.1.smithi148.stdout:5/10: fdatasync f0 0 2024-09-21T04:15:08.737 INFO:tasks.workunit.client.0.smithi110.stdout:4/220: chown d17/d3e/l3f 252312 1 2024-09-21T04:15:08.738 INFO:tasks.workunit.client.1.smithi148.stdout:7/29: mkdir d5/d8 0 2024-09-21T04:15:08.742 INFO:tasks.workunit.client.1.smithi148.stdout:2/75: creat d0/d9/dc/f12 x:0 0 0 2024-09-21T04:15:08.742 INFO:tasks.workunit.client.1.smithi148.stdout:8/17: truncate f0 7671790 0 2024-09-21T04:15:08.742 INFO:tasks.workunit.client.1.smithi148.stdout:8/18: creat f3 x:0 0 0 2024-09-21T04:15:08.742 INFO:tasks.workunit.client.1.smithi148.stdout:9/35: truncate f0 1354239 0 2024-09-21T04:15:08.742 INFO:tasks.workunit.client.1.smithi148.stdout:3/18: getdents d3 0 2024-09-21T04:15:08.744 INFO:tasks.workunit.client.1.smithi148.stdout:3/19: write d3/f4 [32302,2364] 0 2024-09-21T04:15:08.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/11: rename d3/f4 to d3/f7 0 2024-09-21T04:15:08.745 INFO:tasks.workunit.client.1.smithi148.stdout:0/59: mkdir d5/d6/d7/d11 0 2024-09-21T04:15:08.745 INFO:tasks.workunit.client.1.smithi148.stdout:7/30: creat d5/d8/f9 x:0 0 0 2024-09-21T04:15:08.745 INFO:tasks.workunit.client.1.smithi148.stdout:7/31: dread - d5/d8/f9 zero size 2024-09-21T04:15:08.747 INFO:tasks.workunit.client.1.smithi148.stdout:4/11: rmdir d4 39 2024-09-21T04:15:08.749 INFO:tasks.workunit.client.1.smithi148.stdout:9/36: creat d5/fd x:0 0 0 2024-09-21T04:15:08.749 INFO:tasks.workunit.client.1.smithi148.stdout:9/37: truncate d5/f9 875279 0 2024-09-21T04:15:08.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/29: dread d0/f1 [0,4194304] 0 2024-09-21T04:15:08.751 INFO:tasks.workunit.client.1.smithi148.stdout:5/12: rename f2 to d3/f8 0 2024-09-21T04:15:08.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/60: mknod d5/d6/d7/c12 0 2024-09-21T04:15:08.753 INFO:tasks.workunit.client.1.smithi148.stdout:7/32: mknod d5/ca 0 2024-09-21T04:15:08.753 INFO:tasks.workunit.client.1.smithi148.stdout:8/19: unlink f3 0 2024-09-21T04:15:08.754 INFO:tasks.workunit.client.0.smithi110.stdout:4/221: dread d17/d34/d3d/d20/f49 [0,4194304] 0 2024-09-21T04:15:08.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/33: write f0 [324517,127511] 0 2024-09-21T04:15:08.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/38: rename d5/f9 to d5/fe 0 2024-09-21T04:15:08.755 INFO:tasks.workunit.client.0.smithi110.stdout:4/222: mkdir d17/d34/d3d/d20/d43/d51 0 2024-09-21T04:15:08.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/30: rename d0/c2 to d0/d5/c6 0 2024-09-21T04:15:08.757 INFO:tasks.workunit.client.0.smithi110.stdout:4/223: mkdir d17/d34/d3d/d20/d43/d51/d52 0 2024-09-21T04:15:08.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/61: rename d5/d6/d7/c12 to d5/d6/d7/d11/c13 0 2024-09-21T04:15:08.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/62: readlink - no filename 2024-09-21T04:15:08.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/63: chown d5/d6/d7/d11/c13 3729588 1 2024-09-21T04:15:08.762 INFO:tasks.workunit.client.1.smithi148.stdout:6/31: mkdir d0/d5/d7 0 2024-09-21T04:15:08.763 INFO:tasks.workunit.client.0.smithi110.stdout:4/224: dread f14 [0,4194304] 0 2024-09-21T04:15:08.763 INFO:tasks.workunit.client.0.smithi110.stdout:4/225: rmdir d17/d3e 39 2024-09-21T04:15:08.765 INFO:tasks.workunit.client.0.smithi110.stdout:4/226: unlink d17/d1e/f1f 0 2024-09-21T04:15:08.766 INFO:tasks.workunit.client.0.smithi110.stdout:4/227: mknod d17/d34/d3d/d20/d43/d51/c53 0 2024-09-21T04:15:08.766 INFO:tasks.workunit.client.0.smithi110.stdout:4/228: chown d17 635042562 1 2024-09-21T04:15:08.766 INFO:tasks.workunit.client.0.smithi110.stdout:4/229: creat d17/d1e/d21/f54 x:0 0 0 2024-09-21T04:15:08.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/61: dwrite db/f15 [0,4194304] 0 2024-09-21T04:15:08.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/62: dread - db/dd/f12 zero size 2024-09-21T04:15:08.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/34: dread f2 [0,4194304] 0 2024-09-21T04:15:08.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/35: write f0 [130445,74820] 0 2024-09-21T04:15:08.772 INFO:tasks.workunit.client.0.smithi110.stdout:4/230: dread f14 [0,4194304] 0 2024-09-21T04:15:08.773 INFO:tasks.workunit.client.0.smithi110.stdout:4/231: rmdir d17/d34/d3d/d20/d43/d51 39 2024-09-21T04:15:08.773 INFO:tasks.workunit.client.0.smithi110.stdout:4/232: stat d17/d34/d3d/d20/d2b 0 2024-09-21T04:15:08.777 INFO:tasks.workunit.client.0.smithi110.stdout:3/184: dwrite d0/d12/d16/f34 [0,4194304] 0 2024-09-21T04:15:08.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/178: dwrite d0/d7/d13/f2d [0,4194304] 0 2024-09-21T04:15:08.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/179: write d0/d7/d18/f35 [600003,120927] 0 2024-09-21T04:15:08.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/180: stat d0/d7/d23/f33 0 2024-09-21T04:15:08.779 INFO:tasks.workunit.client.1.smithi148.stdout:4/12: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:08.779 INFO:tasks.workunit.client.1.smithi148.stdout:4/13: write f3 [1001723,73366] 0 2024-09-21T04:15:08.779 INFO:tasks.workunit.client.1.smithi148.stdout:4/14: dread - f1 zero size 2024-09-21T04:15:08.779 INFO:tasks.workunit.client.0.smithi110.stdout:3/185: rename d0/d22/d29/f2f to d0/d12/d2a/f3e 0 2024-09-21T04:15:08.779 INFO:tasks.workunit.client.1.smithi148.stdout:1/63: dread f6 [4194304,4194304] 0 2024-09-21T04:15:08.782 INFO:tasks.workunit.client.0.smithi110.stdout:4/233: write d17/d34/d3d/d20/f29 [1623727,72322] 0 2024-09-21T04:15:08.782 INFO:tasks.workunit.client.0.smithi110.stdout:4/234: fsync d17/d1e/f48 0 2024-09-21T04:15:08.782 INFO:tasks.workunit.client.0.smithi110.stdout:4/235: truncate d17/d1e/f4c 808257 0 2024-09-21T04:15:08.783 INFO:tasks.workunit.client.0.smithi110.stdout:4/236: chown d17/d34/d3d/d20/f2c 43 1 2024-09-21T04:15:08.783 INFO:tasks.workunit.client.1.smithi148.stdout:6/32: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:15:08.784 INFO:tasks.workunit.client.1.smithi148.stdout:7/36: dread f0 [0,4194304] 0 2024-09-21T04:15:08.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/64: mknod db/c16 0 2024-09-21T04:15:08.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/65: write db/dd/f13 [255066,14724] 0 2024-09-21T04:15:08.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/181: symlink d0/d30/d34/d3e/d32/l3f 0 2024-09-21T04:15:08.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/33: mkdir d0/d8 0 2024-09-21T04:15:08.792 INFO:tasks.workunit.client.0.smithi110.stdout:3/186: mknod d0/d12/d16/c3f 0 2024-09-21T04:15:08.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/39: getdents d5 0 2024-09-21T04:15:08.797 INFO:tasks.workunit.client.1.smithi148.stdout:7/37: mknod d5/cb 0 2024-09-21T04:15:08.800 INFO:tasks.workunit.client.0.smithi110.stdout:4/237: rename l6 to d17/d3e/l55 0 2024-09-21T04:15:08.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/76: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:08.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/77: stat d0/d9 0 2024-09-21T04:15:08.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/187: mknod d0/d12/d3c/c40 0 2024-09-21T04:15:08.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/188: write d0/d12/d2a/f30 [4778443,16166] 0 2024-09-21T04:15:08.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/189: truncate d0/d12/d16/d1a/f3b 1026837 0 2024-09-21T04:15:08.805 INFO:tasks.workunit.client.1.smithi148.stdout:1/66: rename la to db/l17 0 2024-09-21T04:15:08.806 INFO:tasks.workunit.client.0.smithi110.stdout:8/182: dwrite d0/f3d [0,4194304] 0 2024-09-21T04:15:08.806 INFO:tasks.workunit.client.1.smithi148.stdout:6/34: mknod d0/d8/c9 0 2024-09-21T04:15:08.807 INFO:tasks.workunit.client.0.smithi110.stdout:3/190: mknod d0/d12/d2a/c41 0 2024-09-21T04:15:08.807 INFO:tasks.workunit.client.0.smithi110.stdout:3/191: write d0/d12/f1b [709053,116944] 0 2024-09-21T04:15:08.810 INFO:tasks.workunit.client.1.smithi148.stdout:7/38: truncate f2 4026303 0 2024-09-21T04:15:08.810 INFO:tasks.workunit.client.1.smithi148.stdout:7/39: dread - d5/d8/f9 zero size 2024-09-21T04:15:08.812 INFO:tasks.workunit.client.0.smithi110.stdout:8/183: creat d0/d7/d18/f40 x:0 0 0 2024-09-21T04:15:08.814 INFO:tasks.workunit.client.0.smithi110.stdout:3/192: truncate d0/d12/d2a/f3e 1940859 0 2024-09-21T04:15:08.815 INFO:tasks.workunit.client.1.smithi148.stdout:2/78: mkdir d0/d9/dc/d13 0 2024-09-21T04:15:08.815 INFO:tasks.workunit.client.0.smithi110.stdout:8/184: mkdir d0/d7/d13/d1c/d41 0 2024-09-21T04:15:08.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/64: write d5/f8 [1590780,86015] 0 2024-09-21T04:15:08.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/65: chown d5 0 1 2024-09-21T04:15:08.817 INFO:tasks.workunit.client.0.smithi110.stdout:3/193: symlink d0/d12/d16/d1a/l42 0 2024-09-21T04:15:08.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/40: rename f3 to d5/ff 0 2024-09-21T04:15:08.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/41: chown d5/c6 27445 1 2024-09-21T04:15:08.819 INFO:tasks.workunit.client.1.smithi148.stdout:4/15: truncate d4/f6 1715290 0 2024-09-21T04:15:08.822 INFO:tasks.workunit.client.0.smithi110.stdout:8/185: symlink d0/d7/d13/d1c/l42 0 2024-09-21T04:15:08.822 INFO:tasks.workunit.client.0.smithi110.stdout:0/293: sync 2024-09-21T04:15:08.822 INFO:tasks.workunit.client.0.smithi110.stdout:0/294: write d4/dd/fe [4604734,11078] 0 2024-09-21T04:15:08.822 INFO:tasks.workunit.client.1.smithi148.stdout:6/35: link d0/d5/c6 d0/d5/ca 0 2024-09-21T04:15:08.825 INFO:tasks.workunit.client.0.smithi110.stdout:4/238: dwrite d17/d1e/d21/f31 [0,4194304] 0 2024-09-21T04:15:08.826 INFO:tasks.workunit.client.0.smithi110.stdout:0/295: link d4/dd/d10/c35 d4/dd/d10/d29/d45/c55 0 2024-09-21T04:15:08.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/20: dwrite f1 [0,4194304] 0 2024-09-21T04:15:08.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/21: readlink l2 0 2024-09-21T04:15:08.830 INFO:tasks.workunit.client.1.smithi148.stdout:2/79: dread d0/f4 [4194304,4194304] 0 2024-09-21T04:15:08.832 INFO:tasks.workunit.client.1.smithi148.stdout:7/40: dwrite f4 [0,4194304] 0 2024-09-21T04:15:08.832 INFO:tasks.workunit.client.1.smithi148.stdout:9/42: rename d5/f8 to d5/f10 0 2024-09-21T04:15:08.833 INFO:tasks.workunit.client.0.smithi110.stdout:0/296: dread d4/dd/d10/f13 [0,4194304] 0 2024-09-21T04:15:08.834 INFO:tasks.workunit.client.0.smithi110.stdout:8/186: write d0/d7/d23/f33 [1608741,30609] 0 2024-09-21T04:15:08.835 INFO:tasks.workunit.client.0.smithi110.stdout:4/239: mknod d17/c56 0 2024-09-21T04:15:08.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/80: link d0/d9/fe d0/f14 0 2024-09-21T04:15:08.843 INFO:tasks.workunit.client.0.smithi110.stdout:8/187: fsync d0/d7/d18/f40 0 2024-09-21T04:15:08.844 INFO:tasks.workunit.client.1.smithi148.stdout:7/41: mkdir d5/dc 0 2024-09-21T04:15:08.845 INFO:tasks.workunit.client.1.smithi148.stdout:1/67: truncate db/f15 2377300 0 2024-09-21T04:15:08.848 INFO:tasks.workunit.client.0.smithi110.stdout:4/240: rmdir d17/d34/d3d/d20 39 2024-09-21T04:15:08.848 INFO:tasks.workunit.client.0.smithi110.stdout:4/241: chown d17/f50 60365279 1 2024-09-21T04:15:08.848 INFO:tasks.workunit.client.1.smithi148.stdout:3/22: dwrite d3/f4 [0,4194304] 0 2024-09-21T04:15:08.850 INFO:tasks.workunit.client.0.smithi110.stdout:0/297: rename d4/dd/d10/d29/d2e/f4b to d4/f56 0 2024-09-21T04:15:08.850 INFO:tasks.workunit.client.0.smithi110.stdout:0/298: chown d4/f5 65963 1 2024-09-21T04:15:08.850 INFO:tasks.workunit.client.0.smithi110.stdout:0/299: chown d4/f21 31936660 1 2024-09-21T04:15:08.851 INFO:tasks.workunit.client.1.smithi148.stdout:1/68: symlink db/l18 0 2024-09-21T04:15:08.851 INFO:tasks.workunit.client.1.smithi148.stdout:1/69: chown db 163986583 1 2024-09-21T04:15:08.852 INFO:tasks.workunit.client.0.smithi110.stdout:8/188: creat d0/d7/d13/d1c/d41/f43 x:0 0 0 2024-09-21T04:15:08.852 INFO:tasks.workunit.client.0.smithi110.stdout:8/189: read - d0/d30/f31 zero size 2024-09-21T04:15:08.854 INFO:tasks.workunit.client.0.smithi110.stdout:4/242: mkdir d17/d34/d3d/d57 0 2024-09-21T04:15:08.854 INFO:tasks.workunit.client.0.smithi110.stdout:4/243: creat d17/f58 x:0 0 0 2024-09-21T04:15:08.855 INFO:tasks.workunit.client.1.smithi148.stdout:6/36: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:15:08.855 INFO:tasks.workunit.client.1.smithi148.stdout:2/81: dwrite d0/ff [0,4194304] 0 2024-09-21T04:15:08.856 INFO:tasks.workunit.client.1.smithi148.stdout:2/82: chown d0 64644775 1 2024-09-21T04:15:08.856 INFO:tasks.workunit.client.1.smithi148.stdout:2/83: truncate d0/d9/dc/f12 323036 0 2024-09-21T04:15:08.857 INFO:tasks.workunit.client.0.smithi110.stdout:8/190: rmdir d0/d7 39 2024-09-21T04:15:08.858 INFO:tasks.workunit.client.0.smithi110.stdout:3/194: truncate d0/d12/f1b 283203 0 2024-09-21T04:15:08.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/23: mkdir d3/d5 0 2024-09-21T04:15:08.860 INFO:tasks.workunit.client.1.smithi148.stdout:7/42: dread f2 [0,4194304] 0 2024-09-21T04:15:08.862 INFO:tasks.workunit.client.1.smithi148.stdout:1/70: rename db/dd/l10 to db/dd/l19 0 2024-09-21T04:15:08.862 INFO:tasks.workunit.client.1.smithi148.stdout:4/16: dread d4/f6 [0,4194304] 0 2024-09-21T04:15:08.862 INFO:tasks.workunit.client.1.smithi148.stdout:4/17: dread - f1 zero size 2024-09-21T04:15:08.862 INFO:tasks.workunit.client.1.smithi148.stdout:4/18: truncate f3 1509106 0 2024-09-21T04:15:08.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/244: creat d17/d34/d3d/d20/d43/d51/d52/f59 x:0 0 0 2024-09-21T04:15:08.865 INFO:tasks.workunit.client.0.smithi110.stdout:0/300: getdents d4/d47/d2c/d3a 0 2024-09-21T04:15:08.869 INFO:tasks.workunit.client.0.smithi110.stdout:8/191: mknod d0/d30/c44 0 2024-09-21T04:15:08.869 INFO:tasks.workunit.client.0.smithi110.stdout:8/192: creat d0/d7/d18/f45 x:0 0 0 2024-09-21T04:15:08.869 INFO:tasks.workunit.client.0.smithi110.stdout:4/245: dread d17/f27 [0,4194304] 0 2024-09-21T04:15:08.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/24: dread d3/f4 [0,4194304] 0 2024-09-21T04:15:08.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/25: chown l2 5539 1 2024-09-21T04:15:08.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/37: creat d0/d5/fb x:0 0 0 2024-09-21T04:15:08.872 INFO:tasks.workunit.client.0.smithi110.stdout:3/195: unlink d0/d22/f2d 0 2024-09-21T04:15:08.875 INFO:tasks.workunit.client.1.smithi148.stdout:0/66: dwrite f4 [4194304,4194304] 0 2024-09-21T04:15:08.875 INFO:tasks.workunit.client.1.smithi148.stdout:0/67: write d5/f8 [7119061,87295] 0 2024-09-21T04:15:08.875 INFO:tasks.workunit.client.1.smithi148.stdout:2/84: unlink d0/f6 0 2024-09-21T04:15:08.875 INFO:tasks.workunit.client.1.smithi148.stdout:2/85: chown d0/d9/dc/f12 670669065 1 2024-09-21T04:15:08.876 INFO:tasks.workunit.client.1.smithi148.stdout:2/86: getdents d0/d9/dc/d13 0 2024-09-21T04:15:08.877 INFO:tasks.workunit.client.1.smithi148.stdout:7/43: mknod d5/d8/cd 0 2024-09-21T04:15:08.877 INFO:tasks.workunit.client.0.smithi110.stdout:0/301: creat d4/f57 x:0 0 0 2024-09-21T04:15:08.877 INFO:tasks.workunit.client.0.smithi110.stdout:0/302: fsync d4/d1c/f3c 0 2024-09-21T04:15:08.879 INFO:tasks.workunit.client.0.smithi110.stdout:3/196: dread d0/d12/d16/f25 [0,4194304] 0 2024-09-21T04:15:08.880 INFO:tasks.workunit.client.1.smithi148.stdout:4/19: unlink l2 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:3/197: read d0/f6 [184824,23697] 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:3/198: fdatasync d0/d12/d16/f25 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:0/303: link d4/f1f d4/f58 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:0/304: fsync d4/f21 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:0/305: fsync d4/fb 0 2024-09-21T04:15:08.883 INFO:tasks.workunit.client.0.smithi110.stdout:0/306: truncate d4/f56 1443260 0 2024-09-21T04:15:08.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/43: write d5/ff [4723531,58258] 0 2024-09-21T04:15:08.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/44: write d5/fd [524116,36938] 0 2024-09-21T04:15:08.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/45: stat f1 0 2024-09-21T04:15:08.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/199: mknod d0/d12/d3c/c43 0 2024-09-21T04:15:08.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/200: chown d0/d12 92 1 2024-09-21T04:15:08.885 INFO:tasks.workunit.client.1.smithi148.stdout:0/68: symlink d5/d6/l14 0 2024-09-21T04:15:08.886 INFO:tasks.workunit.client.0.smithi110.stdout:0/307: rename d4/dd/d10/d29/d45/c52 to d4/dd/d53/c59 0 2024-09-21T04:15:08.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/87: symlink d0/l15 0 2024-09-21T04:15:08.888 INFO:tasks.workunit.client.0.smithi110.stdout:3/201: symlink d0/d12/l44 0 2024-09-21T04:15:08.889 INFO:tasks.workunit.client.1.smithi148.stdout:7/44: rename d5/cb to d5/dc/ce 0 2024-09-21T04:15:08.892 INFO:tasks.workunit.client.0.smithi110.stdout:8/193: dwrite d0/d7/d13/f37 [0,4194304] 0 2024-09-21T04:15:08.892 INFO:tasks.workunit.client.1.smithi148.stdout:1/71: truncate db/f15 3075162 0 2024-09-21T04:15:08.895 INFO:tasks.workunit.client.0.smithi110.stdout:4/246: dwrite f5 [0,4194304] 0 2024-09-21T04:15:08.895 INFO:tasks.workunit.client.0.smithi110.stdout:4/247: chown l2 17105220 1 2024-09-21T04:15:08.895 INFO:tasks.workunit.client.0.smithi110.stdout:3/202: rename d0/f11 to d0/d12/d3c/f45 0 2024-09-21T04:15:08.895 INFO:tasks.workunit.client.1.smithi148.stdout:4/20: symlink d4/l7 0 2024-09-21T04:15:08.897 INFO:tasks.workunit.client.1.smithi148.stdout:3/26: truncate d3/f4 990344 0 2024-09-21T04:15:08.899 INFO:tasks.workunit.client.1.smithi148.stdout:7/45: dread f2 [0,4194304] 0 2024-09-21T04:15:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:7/46: write d5/d8/f9 [150750,112455] 0 2024-09-21T04:15:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:7/47: dread - d5/f6 zero size 2024-09-21T04:15:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:9/46: rename d5/ca to d5/c11 0 2024-09-21T04:15:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:9/47: fsync f0 0 2024-09-21T04:15:08.901 INFO:tasks.workunit.client.0.smithi110.stdout:8/194: chown d0/l15 13531988 1 2024-09-21T04:15:08.901 INFO:tasks.workunit.client.0.smithi110.stdout:8/195: write d0/d7/d13/d1c/f1d [5070088,87675] 0 2024-09-21T04:15:08.902 INFO:tasks.workunit.client.1.smithi148.stdout:0/69: creat d5/d6/f15 x:0 0 0 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.1.smithi148.stdout:0/70: chown f4 147181 1 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.1.smithi148.stdout:0/71: creat d5/d6/f16 x:0 0 0 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/38: fsync d0/d5/fb 0 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/39: chown d0/d8 568724 1 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/40: dread - d0/d5/fb zero size 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/308: dwrite d4/dd/d10/f1a [0,4194304] 0 2024-09-21T04:15:08.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/309: chown d4/d1c/f3c 93090140 1 2024-09-21T04:15:08.904 INFO:tasks.workunit.client.1.smithi148.stdout:2/88: mkdir d0/d9/d16 0 2024-09-21T04:15:08.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/72: write db/ff [1373735,41019] 0 2024-09-21T04:15:08.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/248: creat d17/d34/d3d/d57/f5a x:0 0 0 2024-09-21T04:15:08.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/249: write d17/d34/d3d/d20/f29 [2114967,9020] 0 2024-09-21T04:15:08.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/250: creat d17/d1e/d21/f5b x:0 0 0 2024-09-21T04:15:08.908 INFO:tasks.workunit.client.0.smithi110.stdout:4/251: write d17/d34/d3d/f41 [629115,64333] 0 2024-09-21T04:15:08.909 INFO:tasks.workunit.client.1.smithi148.stdout:4/21: rename f1 to d4/f8 0 2024-09-21T04:15:08.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/48: dread f2 [0,4194304] 0 2024-09-21T04:15:08.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/49: creat d5/d8/ff x:0 0 0 2024-09-21T04:15:08.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/73: fsync db/ff 0 2024-09-21T04:15:08.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/74: readlink db/dd/l19 0 2024-09-21T04:15:08.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/75: chown db/f15 927123135 1 2024-09-21T04:15:08.911 INFO:tasks.workunit.client.0.smithi110.stdout:8/196: mkdir d0/d7/d13/d46 0 2024-09-21T04:15:08.912 INFO:tasks.workunit.client.1.smithi148.stdout:3/27: mkdir d3/d5/d6 0 2024-09-21T04:15:08.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/48: creat d5/f12 x:0 0 0 2024-09-21T04:15:08.918 INFO:tasks.workunit.client.1.smithi148.stdout:0/72: getdents d5/d6/d7/d11 0 2024-09-21T04:15:08.920 INFO:tasks.workunit.client.1.smithi148.stdout:6/41: mkdir d0/d5/dc 0 2024-09-21T04:15:08.921 INFO:tasks.workunit.client.0.smithi110.stdout:0/310: creat d4/dd/d53/f5a x:0 0 0 2024-09-21T04:15:08.921 INFO:tasks.workunit.client.0.smithi110.stdout:0/311: dread - d4/d47/d2c/d3a/f51 zero size 2024-09-21T04:15:08.927 INFO:tasks.workunit.client.1.smithi148.stdout:2/89: dwrite d0/d9/fe [0,4194304] 0 2024-09-21T04:15:08.927 INFO:tasks.workunit.client.0.smithi110.stdout:8/197: link d0/d30/d34/d3e/c2b d0/d7/d13/d46/c47 0 2024-09-21T04:15:08.927 INFO:tasks.workunit.client.0.smithi110.stdout:8/198: read d0/d7/d23/f33 [2003384,95] 0 2024-09-21T04:15:08.928 INFO:tasks.workunit.client.1.smithi148.stdout:7/50: dwrite d5/f6 [0,4194304] 0 2024-09-21T04:15:08.928 INFO:tasks.workunit.client.1.smithi148.stdout:4/22: creat d4/f9 x:0 0 0 2024-09-21T04:15:08.930 INFO:tasks.workunit.client.0.smithi110.stdout:0/312: rename d4/f5 to d4/dd/d10/d29/d45/f5b 0 2024-09-21T04:15:08.930 INFO:tasks.workunit.client.1.smithi148.stdout:6/42: dread d0/f1 [0,4194304] 0 2024-09-21T04:15:08.933 INFO:tasks.workunit.client.0.smithi110.stdout:4/252: getdents d17/d34 0 2024-09-21T04:15:08.934 INFO:tasks.workunit.client.1.smithi148.stdout:1/76: mknod db/c1a 0 2024-09-21T04:15:08.934 INFO:tasks.workunit.client.1.smithi148.stdout:1/77: fsync db/dd/f12 0 2024-09-21T04:15:08.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/28: mkdir d3/d5/d7 0 2024-09-21T04:15:08.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/29: getdents d3/d5/d6 0 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:2/63: sync 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:7/180: sync 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/379: sync 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:8/199: symlink d0/d7/l48 0 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/380: write d2/d9/d26/d3d/f43 [444408,17663] 0 2024-09-21T04:15:08.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/381: creat d2/d9/d26/f7b x:0 0 0 2024-09-21T04:15:08.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/49: link d5/l7 d5/l13 0 2024-09-21T04:15:08.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/181: rename dd/d1c/f32 to dd/d1c/f34 0 2024-09-21T04:15:08.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/182: fsync dd/f18 0 2024-09-21T04:15:08.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/73: rename d5/f8 to d5/d6/d7/d11/f17 0 2024-09-21T04:15:08.940 INFO:tasks.workunit.client.0.smithi110.stdout:2/64: mkdir d2/d3/d9/df 0 2024-09-21T04:15:08.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/382: unlink d2/d9/c32 0 2024-09-21T04:15:08.948 INFO:tasks.workunit.client.0.smithi110.stdout:7/183: mkdir dd/d1c/d35 0 2024-09-21T04:15:08.948 INFO:tasks.workunit.client.0.smithi110.stdout:7/184: chown dd/f18 22 1 2024-09-21T04:15:08.951 INFO:tasks.workunit.client.0.smithi110.stdout:7/185: rmdir dd/d1c/d35 0 2024-09-21T04:15:08.951 INFO:tasks.workunit.client.0.smithi110.stdout:4/253: dread d17/d34/d3d/d20/f2c [0,4194304] 0 2024-09-21T04:15:08.952 INFO:tasks.workunit.client.1.smithi148.stdout:7/51: creat d5/dc/f10 x:0 0 0 2024-09-21T04:15:08.955 INFO:tasks.workunit.client.0.smithi110.stdout:7/186: unlink dd/d1c/c33 0 2024-09-21T04:15:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/74: dread f4 [0,4194304] 0 2024-09-21T04:15:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:2/90: dread d0/d9/dc/f12 [0,4194304] 0 2024-09-21T04:15:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/75: dread - d5/d6/f16 zero size 2024-09-21T04:15:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/76: fdatasync d5/d6/d7/d11/f17 0 2024-09-21T04:15:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/77: truncate d5/d6/f15 272407 0 2024-09-21T04:15:08.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/254: mknod d17/d34/d3d/d20/d43/d51/c5c 0 2024-09-21T04:15:08.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/255: chown d17/d34/d3d/d20/d43/d51/d52/f59 8 1 2024-09-21T04:15:08.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/256: write f14 [5106217,86330] 0 2024-09-21T04:15:08.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/257: fsync f5 0 2024-09-21T04:15:08.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/383: rename d2/d9/d23/f3e to d2/d9/d49/d7a/f7c 0 2024-09-21T04:15:08.962 INFO:tasks.workunit.client.1.smithi148.stdout:3/30: mkdir d3/d5/d8 0 2024-09-21T04:15:08.963 INFO:tasks.workunit.client.1.smithi148.stdout:9/50: mknod d5/c14 0 2024-09-21T04:15:08.964 INFO:tasks.workunit.client.0.smithi110.stdout:1/384: dread d2/d9/d26/d3d/f3f [0,4194304] 0 2024-09-21T04:15:08.964 INFO:tasks.workunit.client.1.smithi148.stdout:9/51: read d5/fe [310467,65189] 0 2024-09-21T04:15:08.965 INFO:tasks.workunit.client.0.smithi110.stdout:8/200: dwrite d0/d7/d18/f1f [0,4194304] 0 2024-09-21T04:15:08.966 INFO:tasks.workunit.client.0.smithi110.stdout:1/385: unlink d2/c66 0 2024-09-21T04:15:08.966 INFO:tasks.workunit.client.0.smithi110.stdout:0/313: dwrite d4/f58 [0,4194304] 0 2024-09-21T04:15:08.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/91: creat d0/d9/f17 x:0 0 0 2024-09-21T04:15:08.969 INFO:tasks.workunit.client.1.smithi148.stdout:4/23: rmdir d4 39 2024-09-21T04:15:08.970 INFO:tasks.workunit.client.0.smithi110.stdout:2/65: dwrite d2/d3/fd [0,4194304] 0 2024-09-21T04:15:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:0/78: mkdir d5/d6/d7/d18 0 2024-09-21T04:15:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:0/79: stat d5/d6/l14 0 2024-09-21T04:15:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:0/80: write d5/d6/f15 [306162,100062] 0 2024-09-21T04:15:08.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/52: fsync d5/d8/ff 0 2024-09-21T04:15:08.976 INFO:tasks.workunit.client.1.smithi148.stdout:9/52: mknod d5/c15 0 2024-09-21T04:15:08.976 INFO:tasks.workunit.client.1.smithi148.stdout:9/53: creat d5/f16 x:0 0 0 2024-09-21T04:15:08.976 INFO:tasks.workunit.client.1.smithi148.stdout:9/54: write d5/fd [729042,108215] 0 2024-09-21T04:15:08.976 INFO:tasks.workunit.client.0.smithi110.stdout:8/201: mknod d0/d30/d34/c49 0 2024-09-21T04:15:08.977 INFO:tasks.workunit.client.0.smithi110.stdout:7/187: dwrite dd/d1c/f34 [0,4194304] 0 2024-09-21T04:15:08.980 INFO:tasks.workunit.client.1.smithi148.stdout:0/81: mkdir d5/d6/d7/d11/d19 0 2024-09-21T04:15:08.980 INFO:tasks.workunit.client.1.smithi148.stdout:0/82: fdatasync d5/d6/d7/d11/f17 0 2024-09-21T04:15:08.982 INFO:tasks.workunit.client.0.smithi110.stdout:7/188: dread dd/f18 [0,4194304] 0 2024-09-21T04:15:08.982 INFO:tasks.workunit.client.0.smithi110.stdout:7/189: readlink dd/d14/l2f 0 2024-09-21T04:15:08.982 INFO:tasks.workunit.client.1.smithi148.stdout:4/24: write d4/f6 [1522848,80071] 0 2024-09-21T04:15:08.982 INFO:tasks.workunit.client.1.smithi148.stdout:4/25: write f3 [1674039,19361] 0 2024-09-21T04:15:08.983 INFO:tasks.workunit.client.1.smithi148.stdout:7/53: unlink d5/dc/ce 0 2024-09-21T04:15:08.983 INFO:tasks.workunit.client.1.smithi148.stdout:7/54: chown d5/f6 48 1 2024-09-21T04:15:08.984 INFO:tasks.workunit.client.0.smithi110.stdout:0/314: creat d4/f5c x:0 0 0 2024-09-21T04:15:08.987 INFO:tasks.workunit.client.1.smithi148.stdout:3/31: dwrite f1 [0,4194304] 0 2024-09-21T04:15:08.987 INFO:tasks.workunit.client.1.smithi148.stdout:9/55: symlink d5/l17 0 2024-09-21T04:15:08.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/56: dread d5/fe [0,4194304] 0 2024-09-21T04:15:08.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/57: fdatasync d5/fc 0 2024-09-21T04:15:08.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/83: unlink d5/d6/d7/ca 0 2024-09-21T04:15:08.996 INFO:tasks.workunit.client.1.smithi148.stdout:4/26: mknod d4/ca 0 2024-09-21T04:15:08.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/190: unlink dd/d14/d17/l1b 0 2024-09-21T04:15:08.999 INFO:tasks.workunit.client.1.smithi148.stdout:7/55: symlink d5/d8/l11 0 2024-09-21T04:15:08.999 INFO:tasks.workunit.client.1.smithi148.stdout:7/56: write d5/f7 [161780,86543] 0 2024-09-21T04:15:08.999 INFO:tasks.workunit.client.1.smithi148.stdout:7/57: chown d5/f7 386916 1 2024-09-21T04:15:09.000 INFO:tasks.workunit.client.0.smithi110.stdout:7/191: creat dd/d14/f36 x:0 0 0 2024-09-21T04:15:09.002 INFO:tasks.workunit.client.1.smithi148.stdout:3/32: symlink d3/l9 0 2024-09-21T04:15:09.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/33: write d3/f4 [516401,89801] 0 2024-09-21T04:15:09.004 INFO:tasks.workunit.client.0.smithi110.stdout:4/258: truncate f14 2612115 0 2024-09-21T04:15:09.006 INFO:tasks.workunit.client.0.smithi110.stdout:4/259: mknod d17/d1e/c5d 0 2024-09-21T04:15:09.006 INFO:tasks.workunit.client.0.smithi110.stdout:4/260: fsync d17/d34/d3d/d20/f49 0 2024-09-21T04:15:09.006 INFO:tasks.workunit.client.0.smithi110.stdout:4/261: dread - d17/d34/d3d/f4a zero size 2024-09-21T04:15:09.007 INFO:tasks.workunit.client.1.smithi148.stdout:7/58: symlink d5/dc/l12 0 2024-09-21T04:15:09.007 INFO:tasks.workunit.client.1.smithi148.stdout:7/59: chown d5/f6 10461188 1 2024-09-21T04:15:09.007 INFO:tasks.workunit.client.0.smithi110.stdout:4/262: dread d17/d1e/f4c [0,4194304] 0 2024-09-21T04:15:09.008 INFO:tasks.workunit.client.1.smithi148.stdout:9/58: truncate d5/ff 1965368 0 2024-09-21T04:15:09.009 INFO:tasks.workunit.client.1.smithi148.stdout:7/60: truncate d5/d8/f9 88785 0 2024-09-21T04:15:09.009 INFO:tasks.workunit.client.0.smithi110.stdout:4/263: truncate d17/d1e/d21/f2e 3827128 0 2024-09-21T04:15:09.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/264: readlink d17/d1e/l44 0 2024-09-21T04:15:09.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/265: chown d17/d1e/d21 36341325 1 2024-09-21T04:15:09.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/266: write d17/d1e/f48 [757431,97448] 0 2024-09-21T04:15:09.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/267: write d17/d1e/d21/f5b [229088,117891] 0 2024-09-21T04:15:09.013 INFO:tasks.workunit.client.0.smithi110.stdout:4/268: dread d17/d34/d3d/d20/f29 [0,4194304] 0 2024-09-21T04:15:09.014 INFO:tasks.workunit.client.0.smithi110.stdout:1/386: dwrite d2/d9/d26/d3d/f3f [0,4194304] 0 2024-09-21T04:15:09.015 INFO:tasks.workunit.client.0.smithi110.stdout:4/269: getdents d17/d34/d3d 0 2024-09-21T04:15:09.018 INFO:tasks.workunit.client.0.smithi110.stdout:1/387: write d2/d9/d49/d4f/f6a [111194,97810] 0 2024-09-21T04:15:09.019 INFO:tasks.workunit.client.0.smithi110.stdout:4/270: creat d17/d34/d3d/d20/d2b/f5e x:0 0 0 2024-09-21T04:15:09.020 INFO:tasks.workunit.client.0.smithi110.stdout:4/271: rename d17/d1e/c2f to d17/d34/d3d/d20/c5f 0 2024-09-21T04:15:09.021 INFO:tasks.workunit.client.0.smithi110.stdout:4/272: fsync d17/d34/f37 0 2024-09-21T04:15:09.021 INFO:tasks.workunit.client.0.smithi110.stdout:4/273: fdatasync d17/d34/d3d/d20/f49 0 2024-09-21T04:15:09.021 INFO:tasks.workunit.client.0.smithi110.stdout:1/388: dread d2/d9/d26/d3d/f43 [0,4194304] 0 2024-09-21T04:15:09.021 INFO:tasks.workunit.client.0.smithi110.stdout:1/389: truncate d2/d9/d49/d7a/f7c 3457179 0 2024-09-21T04:15:09.022 INFO:tasks.workunit.client.0.smithi110.stdout:1/390: fdatasync d2/f4 0 2024-09-21T04:15:09.022 INFO:tasks.workunit.client.0.smithi110.stdout:1/391: symlink d2/d9/d49/d5c/l7d 0 2024-09-21T04:15:09.023 INFO:tasks.workunit.client.1.smithi148.stdout:3/34: dwrite f1 [4194304,4194304] 0 2024-09-21T04:15:09.024 INFO:tasks.workunit.client.1.smithi148.stdout:3/35: rename d3/d5/d7 to d3/d5/d8/da 0 2024-09-21T04:15:09.025 INFO:tasks.workunit.client.1.smithi148.stdout:3/36: rmdir d3 39 2024-09-21T04:15:09.025 INFO:tasks.workunit.client.1.smithi148.stdout:3/37: chown d3/l9 59 1 2024-09-21T04:15:09.025 INFO:tasks.workunit.client.0.smithi110.stdout:0/315: dwrite d4/f3b [0,4194304] 0 2024-09-21T04:15:09.026 INFO:tasks.workunit.client.0.smithi110.stdout:0/316: creat d4/dd/d10/d29/f5d x:0 0 0 2024-09-21T04:15:09.027 INFO:tasks.workunit.client.0.smithi110.stdout:8/202: dwrite d0/d7/d13/f37 [0,4194304] 0 2024-09-21T04:15:09.027 INFO:tasks.workunit.client.1.smithi148.stdout:2/92: dwrite d0/f11 [0,4194304] 0 2024-09-21T04:15:09.028 INFO:tasks.workunit.client.0.smithi110.stdout:2/66: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:09.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/27: dwrite d4/f8 [0,4194304] 0 2024-09-21T04:15:09.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/28: write d4/f9 [714446,1275] 0 2024-09-21T04:15:09.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/29: stat d4/c5 0 2024-09-21T04:15:09.030 INFO:tasks.workunit.client.0.smithi110.stdout:7/192: dwrite dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:09.034 INFO:tasks.workunit.client.1.smithi148.stdout:3/38: dread d3/f4 [0,4194304] 0 2024-09-21T04:15:09.035 INFO:tasks.workunit.client.1.smithi148.stdout:4/30: creat d4/fb x:0 0 0 2024-09-21T04:15:09.035 INFO:tasks.workunit.client.1.smithi148.stdout:4/31: creat d4/fc x:0 0 0 2024-09-21T04:15:09.037 INFO:tasks.workunit.client.1.smithi148.stdout:2/93: truncate d0/f14 2504553 0 2024-09-21T04:15:09.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/39: unlink d3/f4 0 2024-09-21T04:15:09.041 INFO:tasks.workunit.client.1.smithi148.stdout:9/59: getdents d5 0 2024-09-21T04:15:09.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/84: getdents d5/d6/d7 0 2024-09-21T04:15:09.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/85: dread - d5/d6/d7/ff zero size 2024-09-21T04:15:09.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/86: write d5/d6/d7/ff [210457,70100] 0 2024-09-21T04:15:09.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/87: creat d5/d6/f1a x:0 0 0 2024-09-21T04:15:09.043 INFO:tasks.workunit.client.1.smithi148.stdout:3/40: rmdir d3/d5/d8 39 2024-09-21T04:15:09.046 INFO:tasks.workunit.client.0.smithi110.stdout:8/203: dread d0/fb [0,4194304] 0 2024-09-21T04:15:09.046 INFO:tasks.workunit.client.1.smithi148.stdout:0/88: creat d5/d6/d7/d11/d19/f1b x:0 0 0 2024-09-21T04:15:09.046 INFO:tasks.workunit.client.1.smithi148.stdout:0/89: write d5/d6/d7/d11/d19/f1b [1042020,22728] 0 2024-09-21T04:15:09.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/90: fdatasync d5/d6/f1a 0 2024-09-21T04:15:09.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/91: write d5/d6/f1a [453925,63257] 0 2024-09-21T04:15:09.047 INFO:tasks.workunit.client.0.smithi110.stdout:8/204: symlink d0/d30/d34/d3e/l4a 0 2024-09-21T04:15:09.048 INFO:tasks.workunit.client.1.smithi148.stdout:0/92: mknod d5/d6/d7/c1c 0 2024-09-21T04:15:09.051 INFO:tasks.workunit.client.1.smithi148.stdout:0/93: rename d5/fc to d5/d6/d7/d11/f1d 0 2024-09-21T04:15:09.054 INFO:tasks.workunit.client.1.smithi148.stdout:7/61: rmdir d5 39 2024-09-21T04:15:09.055 INFO:tasks.workunit.client.1.smithi148.stdout:7/62: write f4 [4999304,20788] 0 2024-09-21T04:15:09.057 INFO:tasks.workunit.client.1.smithi148.stdout:7/63: creat d5/f13 x:0 0 0 2024-09-21T04:15:09.057 INFO:tasks.workunit.client.1.smithi148.stdout:7/64: readlink d5/dc/l12 0 2024-09-21T04:15:09.059 INFO:tasks.workunit.client.1.smithi148.stdout:7/65: mkdir d5/d14 0 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/274: rename d17/d34/d3d/d20/d2b to d17/d3e/d60 0 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/275: write d17/d1e/d21/f5b [350061,13239] 0 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/276: chown d17/f2a 164095228 1 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/277: dread - d17/f39 zero size 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/278: truncate d17/d34/d3d/d20/f49 5692904 0 2024-09-21T04:15:09.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/205: dread d0/d7/d18/f17 [4194304,4194304] 0 2024-09-21T04:15:09.063 INFO:tasks.workunit.client.0.smithi110.stdout:2/67: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:09.063 INFO:tasks.workunit.client.1.smithi148.stdout:7/66: link d5/d8/cd d5/d8/c15 0 2024-09-21T04:15:09.064 INFO:tasks.workunit.client.1.smithi148.stdout:4/32: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:09.068 INFO:tasks.workunit.client.0.smithi110.stdout:1/392: rename d2/d9/d26/c51 to d2/d9/d49/c7e 0 2024-09-21T04:15:09.069 INFO:tasks.workunit.client.1.smithi148.stdout:7/67: link f4 d5/d8/f16 0 2024-09-21T04:15:09.070 INFO:tasks.workunit.client.0.smithi110.stdout:1/393: dread d2/d9/d23/d3b/f55 [0,4194304] 0 2024-09-21T04:15:09.074 INFO:tasks.workunit.client.0.smithi110.stdout:8/206: rename d0/d7/d18 to d0/d7/d13/d4b 0 2024-09-21T04:15:09.074 INFO:tasks.workunit.client.0.smithi110.stdout:8/207: fsync d0/d7/d13/d4b/f1f 0 2024-09-21T04:15:09.074 INFO:tasks.workunit.client.1.smithi148.stdout:9/60: dwrite d5/ff [0,4194304] 0 2024-09-21T04:15:09.075 INFO:tasks.workunit.client.0.smithi110.stdout:1/394: dread d2/d9/d49/d4f/f67 [0,4194304] 0 2024-09-21T04:15:09.076 INFO:tasks.workunit.client.0.smithi110.stdout:1/395: read d2/f2e [193100,33589] 0 2024-09-21T04:15:09.076 INFO:tasks.workunit.client.0.smithi110.stdout:1/396: creat d2/d9/d23/d4e/f7f x:0 0 0 2024-09-21T04:15:09.076 INFO:tasks.workunit.client.0.smithi110.stdout:2/68: rename d2/d3/da/lb to d2/d3/d9/l10 0 2024-09-21T04:15:09.076 INFO:tasks.workunit.client.1.smithi148.stdout:4/33: read d4/fb [2215893,71500] 0 2024-09-21T04:15:09.076 INFO:tasks.workunit.client.1.smithi148.stdout:4/34: fdatasync d4/f6 0 2024-09-21T04:15:09.077 INFO:tasks.workunit.client.0.smithi110.stdout:4/279: truncate d17/d1e/f48 586619 0 2024-09-21T04:15:09.079 INFO:tasks.workunit.client.1.smithi148.stdout:3/41: rmdir d3 39 2024-09-21T04:15:09.079 INFO:tasks.workunit.client.1.smithi148.stdout:3/42: chown d3/d5/d8/da 0 1 2024-09-21T04:15:09.085 INFO:tasks.workunit.client.1.smithi148.stdout:7/68: mknod d5/d8/c17 0 2024-09-21T04:15:09.086 INFO:tasks.workunit.client.1.smithi148.stdout:2/94: dread d0/f14 [0,4194304] 0 2024-09-21T04:15:09.086 INFO:tasks.workunit.client.1.smithi148.stdout:2/95: fdatasync d0/f11 0 2024-09-21T04:15:09.086 INFO:tasks.workunit.client.0.smithi110.stdout:1/397: rename d2/d9/d23/c25 to d2/d9/d49/d5c/c80 0 2024-09-21T04:15:09.087 INFO:tasks.workunit.client.0.smithi110.stdout:5/165: sync 2024-09-21T04:15:09.087 INFO:tasks.workunit.client.0.smithi110.stdout:9/52: sync 2024-09-21T04:15:09.087 INFO:tasks.workunit.client.0.smithi110.stdout:6/105: sync 2024-09-21T04:15:09.089 INFO:tasks.workunit.client.0.smithi110.stdout:4/280: rename d17/d3e/l55 to d17/d34/d3d/d20/d43/d51/l61 0 2024-09-21T04:15:09.089 INFO:tasks.workunit.client.0.smithi110.stdout:2/69: truncate d2/f8 3419086 0 2024-09-21T04:15:09.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/61: mkdir d5/d18 0 2024-09-21T04:15:09.091 INFO:tasks.workunit.client.1.smithi148.stdout:4/35: symlink d4/ld 0 2024-09-21T04:15:09.091 INFO:tasks.workunit.client.1.smithi148.stdout:4/36: creat d4/fe x:0 0 0 2024-09-21T04:15:09.091 INFO:tasks.workunit.client.1.smithi148.stdout:4/37: fdatasync d4/f6 0 2024-09-21T04:15:09.092 INFO:tasks.workunit.client.0.smithi110.stdout:5/166: truncate d1/d7/d8/de/f24 251092 0 2024-09-21T04:15:09.092 INFO:tasks.workunit.client.0.smithi110.stdout:5/167: chown d1/d7/f13 841 1 2024-09-21T04:15:09.093 INFO:tasks.workunit.client.1.smithi148.stdout:3/43: symlink d3/d5/lb 0 2024-09-21T04:15:09.094 INFO:tasks.workunit.client.1.smithi148.stdout:3/44: truncate f1 8952542 0 2024-09-21T04:15:09.094 INFO:tasks.workunit.client.1.smithi148.stdout:3/45: fsync f1 0 2024-09-21T04:15:09.094 INFO:tasks.workunit.client.0.smithi110.stdout:6/106: symlink d4/de/l24 0 2024-09-21T04:15:09.096 INFO:tasks.workunit.client.1.smithi148.stdout:7/69: creat d5/d14/f18 x:0 0 0 2024-09-21T04:15:09.096 INFO:tasks.workunit.client.1.smithi148.stdout:7/70: readlink d5/dc/l12 0 2024-09-21T04:15:09.096 INFO:tasks.workunit.client.0.smithi110.stdout:9/53: symlink d7/db/le 0 2024-09-21T04:15:09.097 INFO:tasks.workunit.client.1.smithi148.stdout:2/96: readlink d0/la 0 2024-09-21T04:15:09.097 INFO:tasks.workunit.client.1.smithi148.stdout:2/97: stat d0/d9/dc/f12 0 2024-09-21T04:15:09.098 INFO:tasks.workunit.client.0.smithi110.stdout:4/281: mknod d17/d3e/c62 0 2024-09-21T04:15:09.099 INFO:tasks.workunit.client.1.smithi148.stdout:9/62: rename d5/c6 to d5/d18/c19 0 2024-09-21T04:15:09.100 INFO:tasks.workunit.client.0.smithi110.stdout:8/208: rmdir d0/d7/d13/d4b 39 2024-09-21T04:15:09.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/70: symlink d2/d3/da/de/l11 0 2024-09-21T04:15:09.102 INFO:tasks.workunit.client.0.smithi110.stdout:6/107: rename d4/de/c14 to d4/d12/d17/c25 0 2024-09-21T04:15:09.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/54: mkdir d7/db/df 0 2024-09-21T04:15:09.110 INFO:tasks.workunit.client.0.smithi110.stdout:4/282: symlink d17/d34/d3d/d20/d43/l63 0 2024-09-21T04:15:09.111 INFO:tasks.workunit.client.1.smithi148.stdout:3/46: rmdir d3 39 2024-09-21T04:15:09.111 INFO:tasks.workunit.client.1.smithi148.stdout:3/47: chown l2 17342 1 2024-09-21T04:15:09.112 INFO:tasks.workunit.client.0.smithi110.stdout:8/209: creat d0/d7/d13/d46/f4c x:0 0 0 2024-09-21T04:15:09.113 INFO:tasks.workunit.client.0.smithi110.stdout:2/71: write d2/d3/fd [605622,54683] 0 2024-09-21T04:15:09.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/71: unlink d5/f7 0 2024-09-21T04:15:09.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/72: dread - d5/dc/f10 zero size 2024-09-21T04:15:09.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/73: fsync d5/dc/f10 0 2024-09-21T04:15:09.115 INFO:tasks.workunit.client.0.smithi110.stdout:9/55: write d7/f8 [110215,28220] 0 2024-09-21T04:15:09.115 INFO:tasks.workunit.client.0.smithi110.stdout:5/168: dwrite d1/d7/d8/f21 [0,4194304] 0 2024-09-21T04:15:09.116 INFO:tasks.workunit.client.0.smithi110.stdout:4/283: mkdir d17/d1e/d64 0 2024-09-21T04:15:09.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/210: symlink d0/d30/l4d 0 2024-09-21T04:15:09.119 INFO:tasks.workunit.client.0.smithi110.stdout:1/398: getdents d2/d9/d49/d5c 0 2024-09-21T04:15:09.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/63: dwrite d5/ff [0,4194304] 0 2024-09-21T04:15:09.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/64: write d5/f16 [250071,35052] 0 2024-09-21T04:15:09.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/65: rename d5 to d5/d1a 22 2024-09-21T04:15:09.124 INFO:tasks.workunit.client.0.smithi110.stdout:9/56: rmdir d7 39 2024-09-21T04:15:09.124 INFO:tasks.workunit.client.0.smithi110.stdout:8/211: dread d0/d7/d13/d4b/f17 [0,4194304] 0 2024-09-21T04:15:09.124 INFO:tasks.workunit.client.0.smithi110.stdout:8/212: write d0/fb [9670491,84179] 0 2024-09-21T04:15:09.125 INFO:tasks.workunit.client.0.smithi110.stdout:6/108: dwrite d4/fa [0,4194304] 0 2024-09-21T04:15:09.126 INFO:tasks.workunit.client.0.smithi110.stdout:1/399: write d2/d9/d26/f2d [1785638,50867] 0 2024-09-21T04:15:09.127 INFO:tasks.workunit.client.0.smithi110.stdout:5/169: truncate d1/d7/d18/f28 1862599 0 2024-09-21T04:15:09.127 INFO:tasks.workunit.client.1.smithi148.stdout:2/98: dread d0/f4 [0,4194304] 0 2024-09-21T04:15:09.128 INFO:tasks.workunit.client.1.smithi148.stdout:9/66: creat d5/f1b x:0 0 0 2024-09-21T04:15:09.130 INFO:tasks.workunit.client.0.smithi110.stdout:4/284: rename fb to d17/d34/f65 0 2024-09-21T04:15:09.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/48: write f1 [6090379,98923] 0 2024-09-21T04:15:09.132 INFO:tasks.workunit.client.1.smithi148.stdout:2/99: creat d0/d9/d16/f18 x:0 0 0 2024-09-21T04:15:09.133 INFO:tasks.workunit.client.1.smithi148.stdout:9/67: symlink d5/l1c 0 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/74: dwrite d5/f6 [0,4194304] 0 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/75: fsync d5/f13 0 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/76: readlink d5/d8/l11 0 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/77: chown d5/dc 322996 1 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/78: chown d5/d8/f16 1907801 1 2024-09-21T04:15:09.135 INFO:tasks.workunit.client.1.smithi148.stdout:2/100: mknod d0/d9/dc/d13/c19 0 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/20: sync 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/13: sync 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/21: chown f0 882 1 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/14: fdatasync f0 0 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/22: chown c1 28 1 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/15: readlink l1 0 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/16: creat d3/f9 x:0 0 0 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/17: fsync d3/f8 0 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/18: dread - d3/f8 zero size 2024-09-21T04:15:09.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/19: write f0 [1161500,10930] 0 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/20: creat d3/fa x:0 0 0 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/21: write d3/f9 [803472,77713] 0 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/22: truncate f0 1790634 0 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/23: truncate d3/f7 219525 0 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/24: dread - d3/fa zero size 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/25: chown d3/c6 1773061 1 2024-09-21T04:15:09.138 INFO:tasks.workunit.client.1.smithi148.stdout:9/68: link d5/lb d5/l1d 0 2024-09-21T04:15:09.139 INFO:tasks.workunit.client.0.smithi110.stdout:6/109: rmdir d4/d12/d17 39 2024-09-21T04:15:09.140 INFO:tasks.workunit.client.0.smithi110.stdout:1/400: rmdir d2/d9/d49/d7a 39 2024-09-21T04:15:09.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/38: rmdir d4 39 2024-09-21T04:15:09.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/39: chown d4/fb 30306375 1 2024-09-21T04:15:09.142 INFO:tasks.workunit.client.1.smithi148.stdout:3/49: read f1 [3201694,54272] 0 2024-09-21T04:15:09.144 INFO:tasks.workunit.client.1.smithi148.stdout:7/79: link f2 d5/dc/f19 0 2024-09-21T04:15:09.145 INFO:tasks.workunit.client.0.smithi110.stdout:5/170: creat d1/d22/f32 x:0 0 0 2024-09-21T04:15:09.147 INFO:tasks.workunit.client.0.smithi110.stdout:4/285: mknod d17/d1e/d21/c66 0 2024-09-21T04:15:09.147 INFO:tasks.workunit.client.0.smithi110.stdout:6/110: write d4/de/f10 [4123231,33340] 0 2024-09-21T04:15:09.147 INFO:tasks.workunit.client.0.smithi110.stdout:6/111: chown d4/de/l24 0 1 2024-09-21T04:15:09.150 INFO:tasks.workunit.client.0.smithi110.stdout:9/57: mkdir d7/d10 0 2024-09-21T04:15:09.153 INFO:tasks.workunit.client.0.smithi110.stdout:1/401: rename d2/d9/d49/d4f/d53/c70 to d2/d9/d26/c81 0 2024-09-21T04:15:09.156 INFO:tasks.workunit.client.1.smithi148.stdout:5/26: dwrite d3/f7 [0,4194304] 0 2024-09-21T04:15:09.156 INFO:tasks.workunit.client.0.smithi110.stdout:8/213: dwrite d0/d7/d13/d4b/f17 [8388608,4194304] 0 2024-09-21T04:15:09.156 INFO:tasks.workunit.client.0.smithi110.stdout:8/214: write d0/d7/d13/f1e [192140,88645] 0 2024-09-21T04:15:09.157 INFO:tasks.workunit.client.1.smithi148.stdout:2/101: dwrite d0/d9/dc/f12 [0,4194304] 0 2024-09-21T04:15:09.157 INFO:tasks.workunit.client.1.smithi148.stdout:3/50: fdatasync f1 0 2024-09-21T04:15:09.159 INFO:tasks.workunit.client.0.smithi110.stdout:9/58: mkdir d7/db/d11 0 2024-09-21T04:15:09.160 INFO:tasks.workunit.client.1.smithi148.stdout:8/23: read f0 [6376726,8438] 0 2024-09-21T04:15:09.162 INFO:tasks.workunit.client.0.smithi110.stdout:6/112: rename d4/d12/c21 to d4/d12/d17/c26 0 2024-09-21T04:15:09.162 INFO:tasks.workunit.client.0.smithi110.stdout:6/113: getdents d4 0 2024-09-21T04:15:09.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/69: mkdir d5/d1e 0 2024-09-21T04:15:09.165 INFO:tasks.workunit.client.0.smithi110.stdout:1/402: unlink d2/d9/d23/d3b/f6b 0 2024-09-21T04:15:09.166 INFO:tasks.workunit.client.1.smithi148.stdout:4/40: truncate d4/f8 848452 0 2024-09-21T04:15:09.166 INFO:tasks.workunit.client.0.smithi110.stdout:8/215: dread d0/f11 [0,4194304] 0 2024-09-21T04:15:09.166 INFO:tasks.workunit.client.0.smithi110.stdout:8/216: fsync d0/d7/d13/d4b/f28 0 2024-09-21T04:15:09.166 INFO:tasks.workunit.client.0.smithi110.stdout:8/217: chown d0/d7/d13/d1c/d41/f43 5044 1 2024-09-21T04:15:09.167 INFO:tasks.workunit.client.1.smithi148.stdout:4/41: read d4/f9 [484282,111802] 0 2024-09-21T04:15:09.169 INFO:tasks.workunit.client.1.smithi148.stdout:7/80: creat d5/d8/f1a x:0 0 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:4/286: dwrite d17/d1e/f4c [0,4194304] 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/171: dwrite d1/dc/f30 [0,4194304] 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/172: dread - d1/d22/f32 zero size 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/173: fdatasync d1/d7/d18/f28 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/174: stat d1/d7/d8/de/f24 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/175: stat d1/d7/d8/de/d1e 0 2024-09-21T04:15:09.170 INFO:tasks.workunit.client.0.smithi110.stdout:5/176: creat d1/dc/f33 x:0 0 0 2024-09-21T04:15:09.175 INFO:tasks.workunit.client.0.smithi110.stdout:4/287: dread f14 [0,4194304] 0 2024-09-21T04:15:09.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/59: creat d7/d10/f12 x:0 0 0 2024-09-21T04:15:09.178 INFO:tasks.workunit.client.0.smithi110.stdout:6/114: mkdir d4/d12/d17/d27 0 2024-09-21T04:15:09.181 INFO:tasks.workunit.client.0.smithi110.stdout:2/72: truncate d2/f8 816197 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/81: dread d5/d8/f16 [0,4194304] 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/51: dwrite f1 [4194304,4194304] 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/52: stat d3/d5/d8/da 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/82: creat d5/d8/f1b x:0 0 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/83: fsync d5/f6 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/53: fsync f1 0 2024-09-21T04:15:09.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/84: stat d5/d8/f16 0 2024-09-21T04:15:09.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/85: write d5/dc/f19 [1345700,74208] 0 2024-09-21T04:15:09.184 INFO:tasks.workunit.client.1.smithi148.stdout:9/70: mkdir d5/d1f 0 2024-09-21T04:15:09.185 INFO:tasks.workunit.client.1.smithi148.stdout:9/71: read d5/fe [104413,24551] 0 2024-09-21T04:15:09.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/42: rmdir d4 39 2024-09-21T04:15:09.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/43: stat d4/fe 0 2024-09-21T04:15:09.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/44: fsync d4/f8 0 2024-09-21T04:15:09.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/102: rename d0/d9/d16/f18 to d0/d9/dc/d13/f1a 0 2024-09-21T04:15:09.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/103: creat d0/d9/f1b x:0 0 0 2024-09-21T04:15:09.191 INFO:tasks.workunit.client.1.smithi148.stdout:4/45: dread d4/fb [0,4194304] 0 2024-09-21T04:15:09.191 INFO:tasks.workunit.client.1.smithi148.stdout:4/46: read d4/f9 [398123,94504] 0 2024-09-21T04:15:09.192 INFO:tasks.workunit.client.0.smithi110.stdout:5/177: symlink d1/d7/d8/de/d1e/l34 0 2024-09-21T04:15:09.192 INFO:tasks.workunit.client.0.smithi110.stdout:5/178: creat d1/f35 x:0 0 0 2024-09-21T04:15:09.193 INFO:tasks.workunit.client.1.smithi148.stdout:2/104: write d0/d9/fe [1483794,104468] 0 2024-09-21T04:15:09.193 INFO:tasks.workunit.client.1.smithi148.stdout:2/105: fdatasync d0/f11 0 2024-09-21T04:15:09.193 INFO:tasks.workunit.client.1.smithi148.stdout:3/54: mknod d3/cc 0 2024-09-21T04:15:09.194 INFO:tasks.workunit.client.0.smithi110.stdout:4/288: symlink d17/d34/d3d/l67 0 2024-09-21T04:15:09.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/86: symlink d5/l1c 0 2024-09-21T04:15:09.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/60: dwrite d7/f8 [0,4194304] 0 2024-09-21T04:15:09.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/61: creat d7/d10/f13 x:0 0 0 2024-09-21T04:15:09.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/62: write f6 [1560910,12940] 0 2024-09-21T04:15:09.198 INFO:tasks.workunit.client.0.smithi110.stdout:3/203: sync 2024-09-21T04:15:09.199 INFO:tasks.workunit.client.0.smithi110.stdout:3/204: chown d0/l26 50448955 1 2024-09-21T04:15:09.199 INFO:tasks.workunit.client.0.smithi110.stdout:6/115: mknod d4/d12/d17/c28 0 2024-09-21T04:15:09.201 INFO:tasks.workunit.client.1.smithi148.stdout:8/24: write f0 [5192109,111086] 0 2024-09-21T04:15:09.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/25: creat f4 x:0 0 0 2024-09-21T04:15:09.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/26: chown c1 54 1 2024-09-21T04:15:09.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/27: dread - f4 zero size 2024-09-21T04:15:09.204 INFO:tasks.workunit.client.1.smithi148.stdout:9/72: mknod d5/d1f/c20 0 2024-09-21T04:15:09.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/47: symlink d4/lf 0 2024-09-21T04:15:09.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/48: readlink d4/lf 0 2024-09-21T04:15:09.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/49: truncate d4/fc 263227 0 2024-09-21T04:15:09.212 INFO:tasks.workunit.client.0.smithi110.stdout:6/116: rename d4/cd to d4/de/c29 0 2024-09-21T04:15:09.212 INFO:tasks.workunit.client.0.smithi110.stdout:6/117: chown d4/d12/d17/c25 0 1 2024-09-21T04:15:09.213 INFO:tasks.workunit.client.0.smithi110.stdout:1/403: link d2/d9/d23/f29 d2/d9/d23/f82 0 2024-09-21T04:15:09.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/27: dwrite d3/f7 [4194304,4194304] 0 2024-09-21T04:15:09.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/28: creat d3/fb x:0 0 0 2024-09-21T04:15:09.214 INFO:tasks.workunit.client.0.smithi110.stdout:3/205: dread d0/f10 [0,4194304] 0 2024-09-21T04:15:09.215 INFO:tasks.workunit.client.0.smithi110.stdout:6/118: rename d4/f8 to d4/d12/d17/f2a 0 2024-09-21T04:15:09.215 INFO:tasks.workunit.client.0.smithi110.stdout:6/119: fdatasync d4/d12/d17/f2a 0 2024-09-21T04:15:09.215 INFO:tasks.workunit.client.0.smithi110.stdout:6/120: write d4/de/f10 [3197502,89933] 0 2024-09-21T04:15:09.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/55: dread f1 [4194304,4194304] 0 2024-09-21T04:15:09.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/56: getdents d3/d5/d6 0 2024-09-21T04:15:09.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/57: rmdir d3 39 2024-09-21T04:15:09.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/58: readlink d3/d5/lb 0 2024-09-21T04:15:09.219 INFO:tasks.workunit.client.0.smithi110.stdout:6/121: rmdir d4/d12 39 2024-09-21T04:15:09.219 INFO:tasks.workunit.client.0.smithi110.stdout:1/404: rmdir d2/d9/d26/d3d 39 2024-09-21T04:15:09.221 INFO:tasks.workunit.client.1.smithi148.stdout:3/59: mknod d3/d5/cd 0 2024-09-21T04:15:09.222 INFO:tasks.workunit.client.1.smithi148.stdout:2/106: dread d0/f11 [0,4194304] 0 2024-09-21T04:15:09.222 INFO:tasks.workunit.client.0.smithi110.stdout:3/206: getdents d0/d12 0 2024-09-21T04:15:09.223 INFO:tasks.workunit.client.0.smithi110.stdout:6/122: creat d4/d12/f2b x:0 0 0 2024-09-21T04:15:09.223 INFO:tasks.workunit.client.0.smithi110.stdout:6/123: write d4/de/f11 [4611097,26476] 0 2024-09-21T04:15:09.224 INFO:tasks.workunit.client.1.smithi148.stdout:3/60: mknod d3/ce 0 2024-09-21T04:15:09.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/405: symlink d2/d9/d49/d7a/l83 0 2024-09-21T04:15:09.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/406: fsync d2/d9/d26/f4d 0 2024-09-21T04:15:09.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/407: readlink d2/d9/d23/d3b/l79 0 2024-09-21T04:15:09.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/107: link d0/d9/dc/d13/c19 d0/d9/c1c 0 2024-09-21T04:15:09.227 INFO:tasks.workunit.client.0.smithi110.stdout:3/207: creat d0/d22/f46 x:0 0 0 2024-09-21T04:15:09.229 INFO:tasks.workunit.client.1.smithi148.stdout:2/108: readlink d0/l15 0 2024-09-21T04:15:09.229 INFO:tasks.workunit.client.0.smithi110.stdout:3/208: symlink d0/d12/d2a/l47 0 2024-09-21T04:15:09.229 INFO:tasks.workunit.client.0.smithi110.stdout:3/209: readlink d0/l26 0 2024-09-21T04:15:09.230 INFO:tasks.workunit.client.1.smithi148.stdout:5/29: dread d3/f5 [0,4194304] 0 2024-09-21T04:15:09.231 INFO:tasks.workunit.client.0.smithi110.stdout:1/408: link d2/d9/d49/f77 d2/d9/d49/d5c/f84 0 2024-09-21T04:15:09.231 INFO:tasks.workunit.client.0.smithi110.stdout:1/409: chown d2/d9/d26/d3d/f58 3 1 2024-09-21T04:15:09.231 INFO:tasks.workunit.client.0.smithi110.stdout:1/410: read - d2/d9/d23/d4e/f7f zero size 2024-09-21T04:15:09.233 INFO:tasks.workunit.client.0.smithi110.stdout:1/411: dread d2/f2e [0,4194304] 0 2024-09-21T04:15:09.235 INFO:tasks.workunit.client.0.smithi110.stdout:1/412: rename d2/d9/d26/d3d/f3f to d2/d9/d49/d4f/d53/f85 0 2024-09-21T04:15:09.237 INFO:tasks.workunit.client.0.smithi110.stdout:1/413: mkdir d2/d9/d23/d3b/d5e/d86 0 2024-09-21T04:15:09.239 INFO:tasks.workunit.client.0.smithi110.stdout:9/63: truncate d7/f8 361817 0 2024-09-21T04:15:09.241 INFO:tasks.workunit.client.0.smithi110.stdout:1/414: mkdir d2/d9/d49/d4f/d87 0 2024-09-21T04:15:09.243 INFO:tasks.workunit.client.0.smithi110.stdout:9/64: rmdir d7/db 39 2024-09-21T04:15:09.244 INFO:tasks.workunit.client.0.smithi110.stdout:1/415: mkdir d2/d88 0 2024-09-21T04:15:09.245 INFO:tasks.workunit.client.0.smithi110.stdout:9/65: mkdir d7/db/d14 0 2024-09-21T04:15:09.245 INFO:tasks.workunit.client.0.smithi110.stdout:9/66: truncate f2 5101009 0 2024-09-21T04:15:09.245 INFO:tasks.workunit.client.0.smithi110.stdout:9/67: readlink d7/db/ld 0 2024-09-21T04:15:09.246 INFO:tasks.workunit.client.0.smithi110.stdout:1/416: chown d2/d9/d49/f5b 4068869 1 2024-09-21T04:15:09.247 INFO:tasks.workunit.client.0.smithi110.stdout:9/68: mknod d7/db/d11/c15 0 2024-09-21T04:15:09.247 INFO:tasks.workunit.client.0.smithi110.stdout:1/417: creat d2/d9/d23/d3b/d5e/d86/f89 x:0 0 0 2024-09-21T04:15:09.248 INFO:tasks.workunit.client.0.smithi110.stdout:1/418: chown d2/d9/d23/d3b/d5e/f5f 33947 1 2024-09-21T04:15:09.248 INFO:tasks.workunit.client.0.smithi110.stdout:1/419: fsync d2/f33 0 2024-09-21T04:15:09.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/179: dwrite d1/d7/d8/de/f24 [0,4194304] 0 2024-09-21T04:15:09.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/180: truncate d1/dc/f1d 893929 0 2024-09-21T04:15:09.252 INFO:tasks.workunit.client.1.smithi148.stdout:7/87: dwrite d5/d8/f1a [0,4194304] 0 2024-09-21T04:15:09.252 INFO:tasks.workunit.client.1.smithi148.stdout:7/88: fdatasync d5/dc/f10 0 2024-09-21T04:15:09.253 INFO:tasks.workunit.client.0.smithi110.stdout:5/181: mknod d1/d22/c36 0 2024-09-21T04:15:09.253 INFO:tasks.workunit.client.1.smithi148.stdout:3/61: dwrite f1 [4194304,4194304] 0 2024-09-21T04:15:09.255 INFO:tasks.workunit.client.0.smithi110.stdout:5/182: symlink d1/d7/d8/de/d1e/l37 0 2024-09-21T04:15:09.255 INFO:tasks.workunit.client.0.smithi110.stdout:5/183: chown d1/d7/d8/l31 7121 1 2024-09-21T04:15:09.256 INFO:tasks.workunit.client.0.smithi110.stdout:5/184: chown d1/d22 494553499 1 2024-09-21T04:15:09.256 INFO:tasks.workunit.client.1.smithi148.stdout:8/28: dread f0 [4194304,4194304] 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/29: write f4 [62880,30542] 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/30: creat f5 x:0 0 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/31: dread - f5 zero size 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/32: creat f6 x:0 0 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/33: creat f7 x:0 0 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/34: write f7 [209939,116210] 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/35: creat f8 x:0 0 0 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/36: dread - f8 zero size 2024-09-21T04:15:09.257 INFO:tasks.workunit.client.1.smithi148.stdout:8/37: creat f9 x:0 0 0 2024-09-21T04:15:09.258 INFO:tasks.workunit.client.1.smithi148.stdout:7/89: dread f0 [0,4194304] 0 2024-09-21T04:15:09.259 INFO:tasks.workunit.client.1.smithi148.stdout:4/50: dwrite d4/f8 [0,4194304] 0 2024-09-21T04:15:09.260 INFO:tasks.workunit.client.0.smithi110.stdout:5/185: dread d1/d22/f27 [0,4194304] 0 2024-09-21T04:15:09.260 INFO:tasks.workunit.client.0.smithi110.stdout:4/289: dwrite d17/d1e/d21/f5b [0,4194304] 0 2024-09-21T04:15:09.263 INFO:tasks.workunit.client.1.smithi148.stdout:3/62: dread f1 [4194304,4194304] 0 2024-09-21T04:15:09.263 INFO:tasks.workunit.client.1.smithi148.stdout:3/63: stat f1 0 2024-09-21T04:15:09.263 INFO:tasks.workunit.client.0.smithi110.stdout:6/124: dread d4/de/f10 [0,4194304] 0 2024-09-21T04:15:09.264 INFO:tasks.workunit.client.0.smithi110.stdout:5/186: rename d1/d7/d8/de/f25 to d1/f38 0 2024-09-21T04:15:09.264 INFO:tasks.workunit.client.1.smithi148.stdout:5/30: rmdir d3 39 2024-09-21T04:15:09.265 INFO:tasks.workunit.client.0.smithi110.stdout:6/125: dread d4/f6 [0,4194304] 0 2024-09-21T04:15:09.267 INFO:tasks.workunit.client.0.smithi110.stdout:2/73: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:09.268 INFO:tasks.workunit.client.1.smithi148.stdout:3/64: write f1 [6768921,21256] 0 2024-09-21T04:15:09.269 INFO:tasks.workunit.client.1.smithi148.stdout:4/51: rename d4/fc to d4/f10 0 2024-09-21T04:15:09.270 INFO:tasks.workunit.client.1.smithi148.stdout:9/73: dwrite d5/f16 [0,4194304] 0 2024-09-21T04:15:09.270 INFO:tasks.workunit.client.1.smithi148.stdout:9/74: chown d5/d1e 57824 1 2024-09-21T04:15:09.270 INFO:tasks.workunit.client.1.smithi148.stdout:2/109: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:15:09.270 INFO:tasks.workunit.client.0.smithi110.stdout:1/420: dwrite d2/d9/d49/d7a/f7c [0,4194304] 0 2024-09-21T04:15:09.270 INFO:tasks.workunit.client.0.smithi110.stdout:1/421: stat d2/d9 0 2024-09-21T04:15:09.271 INFO:tasks.workunit.client.0.smithi110.stdout:4/290: mknod d17/d34/d3d/d20/d43/c68 0 2024-09-21T04:15:09.271 INFO:tasks.workunit.client.0.smithi110.stdout:4/291: fsync d17/d34/d3d/d20/f26 0 2024-09-21T04:15:09.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/110: fdatasync d0/d9/fe 0 2024-09-21T04:15:09.273 INFO:tasks.workunit.client.1.smithi148.stdout:5/31: mkdir d3/dc 0 2024-09-21T04:15:09.273 INFO:tasks.workunit.client.1.smithi148.stdout:5/32: getdents d3/dc 0 2024-09-21T04:15:09.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/126: symlink d4/d12/l2c 0 2024-09-21T04:15:09.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/127: chown d4/de/c29 6 1 2024-09-21T04:15:09.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/90: getdents d5 0 2024-09-21T04:15:09.276 INFO:tasks.workunit.client.1.smithi148.stdout:5/33: dread f0 [0,4194304] 0 2024-09-21T04:15:09.276 INFO:tasks.workunit.client.1.smithi148.stdout:5/34: readlink l1 0 2024-09-21T04:15:09.276 INFO:tasks.workunit.client.0.smithi110.stdout:2/74: rmdir d2/d3/d9/df 0 2024-09-21T04:15:09.278 INFO:tasks.workunit.client.1.smithi148.stdout:4/52: creat d4/f11 x:0 0 0 2024-09-21T04:15:09.280 INFO:tasks.workunit.client.0.smithi110.stdout:9/69: getdents d7/db 0 2024-09-21T04:15:09.282 INFO:tasks.workunit.client.1.smithi148.stdout:7/91: read d5/d8/f16 [3644165,35690] 0 2024-09-21T04:15:09.284 INFO:tasks.workunit.client.1.smithi148.stdout:9/75: mknod d5/d1f/c21 0 2024-09-21T04:15:09.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/422: dread d2/d9/d26/f2d [0,4194304] 0 2024-09-21T04:15:09.286 INFO:tasks.workunit.client.0.smithi110.stdout:9/70: dread f3 [0,4194304] 0 2024-09-21T04:15:09.288 INFO:tasks.workunit.client.0.smithi110.stdout:4/292: dread fa [0,4194304] 0 2024-09-21T04:15:09.289 INFO:tasks.workunit.client.0.smithi110.stdout:4/293: fsync d17/f32 0 2024-09-21T04:15:09.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/53: mkdir d4/d12 0 2024-09-21T04:15:09.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/54: write d4/fb [1229720,97686] 0 2024-09-21T04:15:09.293 INFO:tasks.workunit.client.1.smithi148.stdout:7/92: creat d5/dc/f1d x:0 0 0 2024-09-21T04:15:09.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/93: chown d5 9 1 2024-09-21T04:15:09.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/94: truncate d5/dc/f1d 334721 0 2024-09-21T04:15:09.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/95: fdatasync d5/d8/f16 0 2024-09-21T04:15:09.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/96: readlink d5/l1c 0 2024-09-21T04:15:09.295 INFO:tasks.workunit.client.1.smithi148.stdout:9/76: mkdir d5/d22 0 2024-09-21T04:15:09.295 INFO:tasks.workunit.client.1.smithi148.stdout:9/77: fsync d5/f12 0 2024-09-21T04:15:09.295 INFO:tasks.workunit.client.1.smithi148.stdout:9/78: chown d5/f10 0 1 2024-09-21T04:15:09.296 INFO:tasks.workunit.client.0.smithi110.stdout:2/75: creat d2/d3/f12 x:0 0 0 2024-09-21T04:15:09.296 INFO:tasks.workunit.client.0.smithi110.stdout:2/76: chown d2/d3/d9/l10 15670391 1 2024-09-21T04:15:09.296 INFO:tasks.workunit.client.0.smithi110.stdout:2/77: fdatasync d2/f8 0 2024-09-21T04:15:09.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/35: dwrite d3/f8 [0,4194304] 0 2024-09-21T04:15:09.297 INFO:tasks.workunit.client.1.smithi148.stdout:5/36: chown d3/c6 127317759 1 2024-09-21T04:15:09.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/111: rmdir d0/d9/dc 39 2024-09-21T04:15:09.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/38: fsync f7 0 2024-09-21T04:15:09.298 INFO:tasks.workunit.client.0.smithi110.stdout:1/423: link d2/f33 d2/d9/d49/d5c/f8a 0 2024-09-21T04:15:09.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/55: mknod d4/c13 0 2024-09-21T04:15:09.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/56: stat f3 0 2024-09-21T04:15:09.302 INFO:tasks.workunit.client.1.smithi148.stdout:7/97: link d5/dc/f1d d5/d8/f1e 0 2024-09-21T04:15:09.303 INFO:tasks.workunit.client.0.smithi110.stdout:2/78: mknod d2/d3/c13 0 2024-09-21T04:15:09.305 INFO:tasks.workunit.client.1.smithi148.stdout:9/79: link d5/f12 d5/f23 0 2024-09-21T04:15:09.306 INFO:tasks.workunit.client.0.smithi110.stdout:9/71: getdents d7/d10 0 2024-09-21T04:15:09.306 INFO:tasks.workunit.client.0.smithi110.stdout:9/72: chown d7/d10/f12 26697 1 2024-09-21T04:15:09.308 INFO:tasks.workunit.client.0.smithi110.stdout:4/294: dwrite f4 [4194304,4194304] 0 2024-09-21T04:15:09.308 INFO:tasks.workunit.client.0.smithi110.stdout:6/128: dread d4/fb [8388608,4194304] 0 2024-09-21T04:15:09.308 INFO:tasks.workunit.client.0.smithi110.stdout:5/187: rename d1/d7/d8 to d1/d39 0 2024-09-21T04:15:09.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/37: creat d3/fd x:0 0 0 2024-09-21T04:15:09.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/38: chown d3/f9 0 1 2024-09-21T04:15:09.310 INFO:tasks.workunit.client.0.smithi110.stdout:4/295: rmdir d17/d34/d3d 39 2024-09-21T04:15:09.310 INFO:tasks.workunit.client.0.smithi110.stdout:4/296: stat d17/d34/d3d/l45 0 2024-09-21T04:15:09.311 INFO:tasks.workunit.client.1.smithi148.stdout:2/112: creat d0/f1d x:0 0 0 2024-09-21T04:15:09.311 INFO:tasks.workunit.client.1.smithi148.stdout:2/113: truncate d0/d9/f1b 606378 0 2024-09-21T04:15:09.312 INFO:tasks.workunit.client.0.smithi110.stdout:6/129: symlink d4/l2d 0 2024-09-21T04:15:09.314 INFO:tasks.workunit.client.1.smithi148.stdout:4/57: link c0 d4/c14 0 2024-09-21T04:15:09.315 INFO:tasks.workunit.client.0.smithi110.stdout:1/424: rename d2/f33 to d2/d9/d26/f8b 0 2024-09-21T04:15:09.315 INFO:tasks.workunit.client.0.smithi110.stdout:1/425: read - d2/d9/f6c zero size 2024-09-21T04:15:09.316 INFO:tasks.workunit.client.0.smithi110.stdout:1/426: chown d2/f48 3440126 1 2024-09-21T04:15:09.316 INFO:tasks.workunit.client.0.smithi110.stdout:1/427: write d2/d9/d26/f4d [2009123,78389] 0 2024-09-21T04:15:09.317 INFO:tasks.workunit.client.0.smithi110.stdout:4/297: symlink d17/d3e/l69 0 2024-09-21T04:15:09.317 INFO:tasks.workunit.client.1.smithi148.stdout:8/39: dwrite f6 [0,4194304] 0 2024-09-21T04:15:09.317 INFO:tasks.workunit.client.1.smithi148.stdout:8/40: read - f5 zero size 2024-09-21T04:15:09.319 INFO:tasks.workunit.client.1.smithi148.stdout:9/80: rename d5/f10 to d5/d1f/f24 0 2024-09-21T04:15:09.319 INFO:tasks.workunit.client.1.smithi148.stdout:9/81: fdatasync d5/ff 0 2024-09-21T04:15:09.320 INFO:tasks.workunit.client.0.smithi110.stdout:2/79: rename c1 to d2/d3/c14 0 2024-09-21T04:15:09.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/58: creat d4/d12/f15 x:0 0 0 2024-09-21T04:15:09.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/59: readlink d4/ld 0 2024-09-21T04:15:09.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/60: chown d4/fe 221783198 1 2024-09-21T04:15:09.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/61: stat f3 0 2024-09-21T04:15:09.322 INFO:tasks.workunit.client.0.smithi110.stdout:6/130: write d4/fa [3957938,41922] 0 2024-09-21T04:15:09.323 INFO:tasks.workunit.client.0.smithi110.stdout:6/131: stat d4/c1b 0 2024-09-21T04:15:09.323 INFO:tasks.workunit.client.0.smithi110.stdout:6/132: chown d4/de/ff 7008 1 2024-09-21T04:15:09.324 INFO:tasks.workunit.client.0.smithi110.stdout:1/428: creat d2/d9/d49/d4f/d53/f8c x:0 0 0 2024-09-21T04:15:09.324 INFO:tasks.workunit.client.0.smithi110.stdout:1/429: write d2/f4 [776322,27214] 0 2024-09-21T04:15:09.326 INFO:tasks.workunit.client.0.smithi110.stdout:4/298: mkdir d17/d1e/d6a 0 2024-09-21T04:15:09.328 INFO:tasks.workunit.client.1.smithi148.stdout:5/39: dwrite d3/fa [0,4194304] 0 2024-09-21T04:15:09.328 INFO:tasks.workunit.client.1.smithi148.stdout:5/40: fsync f0 0 2024-09-21T04:15:09.330 INFO:tasks.workunit.client.0.smithi110.stdout:5/188: rename d1/d39/l2e to d1/dc/l3a 0 2024-09-21T04:15:09.333 INFO:tasks.workunit.client.1.smithi148.stdout:9/82: mknod d5/d18/c25 0 2024-09-21T04:15:09.334 INFO:tasks.workunit.client.0.smithi110.stdout:1/430: creat d2/d9/d49/d4f/d87/f8d x:0 0 0 2024-09-21T04:15:09.334 INFO:tasks.workunit.client.0.smithi110.stdout:5/189: dread d1/f15 [0,4194304] 0 2024-09-21T04:15:09.335 INFO:tasks.workunit.client.0.smithi110.stdout:4/299: mkdir d17/d3e/d6b 0 2024-09-21T04:15:09.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/431: mkdir d2/d9/d23/d4e/d8e 0 2024-09-21T04:15:09.341 INFO:tasks.workunit.client.1.smithi148.stdout:5/41: unlink d3/c6 0 2024-09-21T04:15:09.342 INFO:tasks.workunit.client.0.smithi110.stdout:5/190: creat d1/d39/de/d1e/f3b x:0 0 0 2024-09-21T04:15:09.343 INFO:tasks.workunit.client.1.smithi148.stdout:2/114: dwrite d0/d9/f17 [0,4194304] 0 2024-09-21T04:15:09.343 INFO:tasks.workunit.client.0.smithi110.stdout:6/133: dwrite d4/d12/f2b [0,4194304] 0 2024-09-21T04:15:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/41: getdents . 0 2024-09-21T04:15:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/42: stat c1 0 2024-09-21T04:15:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/43: readlink - no filename 2024-09-21T04:15:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/44: chown c1 27396 1 2024-09-21T04:15:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:8/45: chown f5 126777910 1 2024-09-21T04:15:09.345 INFO:tasks.workunit.client.0.smithi110.stdout:0/317: sync 2024-09-21T04:15:09.346 INFO:tasks.workunit.client.0.smithi110.stdout:4/300: symlink d17/d34/d3d/d20/d43/d51/l6c 0 2024-09-21T04:15:09.347 INFO:tasks.workunit.client.0.smithi110.stdout:1/432: dread d2/d9/d26/f8b [0,4194304] 0 2024-09-21T04:15:09.348 INFO:tasks.workunit.client.1.smithi148.stdout:9/83: creat d5/d22/f26 x:0 0 0 2024-09-21T04:15:09.348 INFO:tasks.workunit.client.0.smithi110.stdout:2/80: getdents d2/d3 0 2024-09-21T04:15:09.349 INFO:tasks.workunit.client.0.smithi110.stdout:5/191: stat d1/c4 0 2024-09-21T04:15:09.351 INFO:tasks.workunit.client.1.smithi148.stdout:5/42: mknod d3/dc/ce 0 2024-09-21T04:15:09.351 INFO:tasks.workunit.client.1.smithi148.stdout:5/43: write d3/f9 [984381,21607] 0 2024-09-21T04:15:09.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/98: truncate d5/dc/f1d 58910 0 2024-09-21T04:15:09.354 INFO:tasks.workunit.client.1.smithi148.stdout:9/84: dread f0 [0,4194304] 0 2024-09-21T04:15:09.354 INFO:tasks.workunit.client.1.smithi148.stdout:9/85: read d5/fe [439520,95231] 0 2024-09-21T04:15:09.354 INFO:tasks.workunit.client.1.smithi148.stdout:9/86: chown d5/l1d 0 1 2024-09-21T04:15:09.355 INFO:tasks.workunit.client.0.smithi110.stdout:4/301: creat d17/d3e/d60/f6d x:0 0 0 2024-09-21T04:15:09.356 INFO:tasks.workunit.client.1.smithi148.stdout:6/43: sync 2024-09-21T04:15:09.356 INFO:tasks.workunit.client.1.smithi148.stdout:0/94: sync 2024-09-21T04:15:09.356 INFO:tasks.workunit.client.1.smithi148.stdout:1/78: sync 2024-09-21T04:15:09.357 INFO:tasks.workunit.client.1.smithi148.stdout:8/46: rename c1 to ca 0 2024-09-21T04:15:09.358 INFO:tasks.workunit.client.0.smithi110.stdout:2/81: mknod d2/d3/da/c15 0 2024-09-21T04:15:09.358 INFO:tasks.workunit.client.0.smithi110.stdout:2/82: truncate d2/d3/f12 752561 0 2024-09-21T04:15:09.358 INFO:tasks.workunit.client.0.smithi110.stdout:0/318: write d4/dd/fe [2249582,87381] 0 2024-09-21T04:15:09.359 INFO:tasks.workunit.client.1.smithi148.stdout:7/99: link d5/l1c d5/l1f 0 2024-09-21T04:15:09.360 INFO:tasks.workunit.client.0.smithi110.stdout:5/192: link d1/d7/f13 d1/d22/f3c 0 2024-09-21T04:15:09.361 INFO:tasks.workunit.client.0.smithi110.stdout:4/302: creat d17/d3e/d6b/f6e x:0 0 0 2024-09-21T04:15:09.362 INFO:tasks.workunit.client.1.smithi148.stdout:0/95: dread d5/d6/d7/d11/f1d [0,4194304] 0 2024-09-21T04:15:09.362 INFO:tasks.workunit.client.1.smithi148.stdout:0/96: dread - d5/d6/f16 zero size 2024-09-21T04:15:09.363 INFO:tasks.workunit.client.1.smithi148.stdout:0/97: dread - d5/d6/f16 zero size 2024-09-21T04:15:09.363 INFO:tasks.workunit.client.1.smithi148.stdout:0/98: readlink d5/d6/l14 0 2024-09-21T04:15:09.363 INFO:tasks.workunit.client.1.smithi148.stdout:1/79: unlink db/dd/c11 0 2024-09-21T04:15:09.363 INFO:tasks.workunit.client.0.smithi110.stdout:1/433: getdents d2/d9 0 2024-09-21T04:15:09.363 INFO:tasks.workunit.client.0.smithi110.stdout:6/134: dwrite d4/d12/f1d [0,4194304] 0 2024-09-21T04:15:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/44: write d3/fa [411566,62110] 0 2024-09-21T04:15:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/45: write d3/fd [986695,125228] 0 2024-09-21T04:15:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/46: chown d3/f5 29 1 2024-09-21T04:15:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:2/115: rmdir d0/d9 39 2024-09-21T04:15:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:2/116: dread - d0/d9/dc/d13/f1a zero size 2024-09-21T04:15:09.367 INFO:tasks.workunit.client.0.smithi110.stdout:0/319: link d4/dd/d10/d29/d2e/c49 d4/d47/c5e 0 2024-09-21T04:15:09.368 INFO:tasks.workunit.client.1.smithi148.stdout:1/80: dread db/ff [0,4194304] 0 2024-09-21T04:15:09.368 INFO:tasks.workunit.client.1.smithi148.stdout:1/81: dread - db/dd/f12 zero size 2024-09-21T04:15:09.369 INFO:tasks.workunit.client.1.smithi148.stdout:8/47: stat f5 0 2024-09-21T04:15:09.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/320: read d4/dd/f32 [399211,6746] 0 2024-09-21T04:15:09.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/321: truncate d4/d47/d2c/d3a/f51 826897 0 2024-09-21T04:15:09.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/322: getdents d4/dd/d10/d4f 0 2024-09-21T04:15:09.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/323: fdatasync d4/dd/d10/d29/d2e/f36 0 2024-09-21T04:15:09.371 INFO:tasks.workunit.client.0.smithi110.stdout:5/193: rename d1/dc/f30 to d1/d39/f3d 0 2024-09-21T04:15:09.371 INFO:tasks.workunit.client.0.smithi110.stdout:5/194: readlink d1/d39/de/d1e/l37 0 2024-09-21T04:15:09.371 INFO:tasks.workunit.client.0.smithi110.stdout:5/195: chown d1/f35 19925 1 2024-09-21T04:15:09.371 INFO:tasks.workunit.client.0.smithi110.stdout:5/196: truncate d1/d39/de/d1e/f3b 671565 0 2024-09-21T04:15:09.371 INFO:tasks.workunit.client.0.smithi110.stdout:5/197: fsync d1/dc/f33 0 2024-09-21T04:15:09.372 INFO:tasks.workunit.client.0.smithi110.stdout:5/198: creat d1/f3e x:0 0 0 2024-09-21T04:15:09.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/100: rename d5/d8/c15 to d5/d8/c20 0 2024-09-21T04:15:09.376 INFO:tasks.workunit.client.0.smithi110.stdout:1/434: mkdir d2/d9/d26/d3d/d8f 0 2024-09-21T04:15:09.377 INFO:tasks.workunit.client.1.smithi148.stdout:9/87: rmdir d5 39 2024-09-21T04:15:09.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/101: read d5/dc/f1d [7671,45908] 0 2024-09-21T04:15:09.382 INFO:tasks.workunit.client.1.smithi148.stdout:0/99: mkdir d5/d6/d7/d11/d19/d1e 0 2024-09-21T04:15:09.385 INFO:tasks.workunit.client.0.smithi110.stdout:0/324: truncate d4/dd/d10/f2a 3504339 0 2024-09-21T04:15:09.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/47: truncate f0 277982 0 2024-09-21T04:15:09.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/48: stat f0 0 2024-09-21T04:15:09.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/135: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:09.387 INFO:tasks.workunit.client.1.smithi148.stdout:2/117: creat d0/f1e x:0 0 0 2024-09-21T04:15:09.388 INFO:tasks.workunit.client.1.smithi148.stdout:8/48: dwrite f5 [0,4194304] 0 2024-09-21T04:15:09.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/199: truncate d1/dc/f1a 19216 0 2024-09-21T04:15:09.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/200: fsync d1/d39/de/d1e/f2f 0 2024-09-21T04:15:09.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/435: creat d2/d9/d26/d3d/d8f/f90 x:0 0 0 2024-09-21T04:15:09.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/436: truncate d2/d9/d26/f2d 2604447 0 2024-09-21T04:15:09.391 INFO:tasks.workunit.client.0.smithi110.stdout:0/325: rename d4/dd/d10/d4f to d4/dd/d10/d29/d5f 0 2024-09-21T04:15:09.392 INFO:tasks.workunit.client.1.smithi148.stdout:9/88: mknod d5/d1e/c27 0 2024-09-21T04:15:09.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/102: mknod d5/d8/c21 0 2024-09-21T04:15:09.397 INFO:tasks.workunit.client.1.smithi148.stdout:0/100: link d5/d6/d7/d11/d19/f1b d5/d6/f1f 0 2024-09-21T04:15:09.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/326: write d4/f3b [3927492,19056] 0 2024-09-21T04:15:09.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/327: fsync d4/d47/f40 0 2024-09-21T04:15:09.398 INFO:tasks.workunit.client.0.smithi110.stdout:0/328: chown d4/dd/d10/d29/d5f 78468 1 2024-09-21T04:15:09.398 INFO:tasks.workunit.client.0.smithi110.stdout:0/329: dread d4/f56 [0,4194304] 0 2024-09-21T04:15:09.398 INFO:tasks.workunit.client.0.smithi110.stdout:6/136: mkdir d4/d12/d17/d2e 0 2024-09-21T04:15:09.399 INFO:tasks.workunit.client.1.smithi148.stdout:5/49: unlink d3/f5 0 2024-09-21T04:15:09.400 INFO:tasks.workunit.client.0.smithi110.stdout:5/201: rmdir d1/d39/de 39 2024-09-21T04:15:09.400 INFO:tasks.workunit.client.0.smithi110.stdout:5/202: chown d1/f6 4 1 2024-09-21T04:15:09.401 INFO:tasks.workunit.client.1.smithi148.stdout:2/118: creat d0/f1f x:0 0 0 2024-09-21T04:15:09.402 INFO:tasks.workunit.client.0.smithi110.stdout:1/437: truncate d2/d9/d49/f5b 2587463 0 2024-09-21T04:15:09.406 INFO:tasks.workunit.client.1.smithi148.stdout:1/82: getdents db 0 2024-09-21T04:15:09.407 INFO:tasks.workunit.client.0.smithi110.stdout:0/330: mkdir d4/dd/d60 0 2024-09-21T04:15:09.409 INFO:tasks.workunit.client.0.smithi110.stdout:6/137: unlink d4/de/l22 0 2024-09-21T04:15:09.410 INFO:tasks.workunit.client.1.smithi148.stdout:9/89: rename d5/d18 to d5/d22/d28 0 2024-09-21T04:15:09.411 INFO:tasks.workunit.client.0.smithi110.stdout:5/203: mkdir d1/d7/d18/d3f 0 2024-09-21T04:15:09.412 INFO:tasks.workunit.client.0.smithi110.stdout:5/204: write d1/f6 [482919,41694] 0 2024-09-21T04:15:09.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/103: truncate d5/f6 3055228 0 2024-09-21T04:15:09.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/104: creat d5/d8/f22 x:0 0 0 2024-09-21T04:15:09.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/105: write d5/dc/f19 [2808444,38498] 0 2024-09-21T04:15:09.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/106: chown d5/l1f 162 1 2024-09-21T04:15:09.414 INFO:tasks.workunit.client.0.smithi110.stdout:1/438: mkdir d2/d9/d49/d5c/d91 0 2024-09-21T04:15:09.414 INFO:tasks.workunit.client.0.smithi110.stdout:1/439: stat d2/d9/d26/c72 0 2024-09-21T04:15:09.415 INFO:tasks.workunit.client.1.smithi148.stdout:0/101: link d5/fe d5/f20 0 2024-09-21T04:15:09.415 INFO:tasks.workunit.client.1.smithi148.stdout:0/102: creat d5/d6/d7/d11/d19/f21 x:0 0 0 2024-09-21T04:15:09.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/331: symlink d4/dd/d10/d29/d45/l61 0 2024-09-21T04:15:09.417 INFO:tasks.workunit.client.1.smithi148.stdout:5/50: link l1 d3/lf 0 2024-09-21T04:15:09.418 INFO:tasks.workunit.client.0.smithi110.stdout:6/138: creat d4/d12/d17/d27/f2f x:0 0 0 2024-09-21T04:15:09.419 INFO:tasks.workunit.client.0.smithi110.stdout:5/205: rename d1/d39/l31 to d1/d7/d18/d3f/l40 0 2024-09-21T04:15:09.419 INFO:tasks.workunit.client.0.smithi110.stdout:5/206: stat d1/d7/f13 0 2024-09-21T04:15:09.420 INFO:tasks.workunit.client.1.smithi148.stdout:1/83: creat db/f1b x:0 0 0 2024-09-21T04:15:09.423 INFO:tasks.workunit.client.1.smithi148.stdout:9/90: rename d5/d22/f26 to d5/d1e/f29 0 2024-09-21T04:15:09.423 INFO:tasks.workunit.client.1.smithi148.stdout:9/91: write d5/d1f/f24 [604959,19849] 0 2024-09-21T04:15:09.423 INFO:tasks.workunit.client.0.smithi110.stdout:0/332: stat d4/l4d 0 2024-09-21T04:15:09.423 INFO:tasks.workunit.client.0.smithi110.stdout:0/333: readlink d4/dd/d10/d29/d45/l48 0 2024-09-21T04:15:09.423 INFO:tasks.workunit.client.0.smithi110.stdout:0/334: read d4/f3b [1908850,79795] 0 2024-09-21T04:15:09.425 INFO:tasks.workunit.client.0.smithi110.stdout:6/139: truncate d4/f7 2487725 0 2024-09-21T04:15:09.428 INFO:tasks.workunit.client.1.smithi148.stdout:7/107: mkdir d5/dc/d23 0 2024-09-21T04:15:09.430 INFO:tasks.workunit.client.0.smithi110.stdout:5/207: write d1/d39/f2d [4132769,104589] 0 2024-09-21T04:15:09.439 INFO:tasks.workunit.client.1.smithi148.stdout:5/51: mknod d3/c10 0 2024-09-21T04:15:09.441 INFO:tasks.workunit.client.1.smithi148.stdout:2/119: mknod d0/c20 0 2024-09-21T04:15:09.444 INFO:tasks.workunit.client.0.smithi110.stdout:0/335: rename d4/dd/d10/d29/d2e to d4/d1c/d62 0 2024-09-21T04:15:09.445 INFO:tasks.workunit.client.1.smithi148.stdout:7/108: mkdir d5/dc/d24 0 2024-09-21T04:15:09.447 INFO:tasks.workunit.client.0.smithi110.stdout:0/336: symlink d4/d47/l63 0 2024-09-21T04:15:09.447 INFO:tasks.workunit.client.0.smithi110.stdout:0/337: write d4/d47/d2c/d3a/f43 [417606,54139] 0 2024-09-21T04:15:09.447 INFO:tasks.workunit.client.0.smithi110.stdout:0/338: dread - d4/f57 zero size 2024-09-21T04:15:09.448 INFO:tasks.workunit.client.0.smithi110.stdout:0/339: chown d4/d1c/d62/l3f 75 1 2024-09-21T04:15:09.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/120: creat d0/d9/dc/d13/f21 x:0 0 0 2024-09-21T04:15:09.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/121: readlink d0/l15 0 2024-09-21T04:15:09.448 INFO:tasks.workunit.client.0.smithi110.stdout:0/340: link d4/c33 d4/d1c/c64 0 2024-09-21T04:15:09.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/122: mknod d0/d9/d16/c22 0 2024-09-21T04:15:09.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/123: chown d0/c20 127 1 2024-09-21T04:15:09.450 INFO:tasks.workunit.client.0.smithi110.stdout:0/341: unlink d4/dd/l16 0 2024-09-21T04:15:09.451 INFO:tasks.workunit.client.1.smithi148.stdout:2/124: creat d0/f23 x:0 0 0 2024-09-21T04:15:09.451 INFO:tasks.workunit.client.0.smithi110.stdout:0/342: unlink d4/dd/d10/d29/d45/c55 0 2024-09-21T04:15:09.452 INFO:tasks.workunit.client.0.smithi110.stdout:0/343: write d4/dd/d10/f50 [648809,9478] 0 2024-09-21T04:15:09.452 INFO:tasks.workunit.client.1.smithi148.stdout:2/125: creat d0/d9/dc/f24 x:0 0 0 2024-09-21T04:15:09.453 INFO:tasks.workunit.client.0.smithi110.stdout:0/344: rmdir d4/d47/d2c/d3a 39 2024-09-21T04:15:09.453 INFO:tasks.workunit.client.0.smithi110.stdout:0/345: chown d4 19 1 2024-09-21T04:15:09.454 INFO:tasks.workunit.client.1.smithi148.stdout:2/126: mkdir d0/d9/dc/d13/d25 0 2024-09-21T04:15:09.456 INFO:tasks.workunit.client.0.smithi110.stdout:1/440: dwrite d2/d9/d23/f38 [0,4194304] 0 2024-09-21T04:15:09.457 INFO:tasks.workunit.client.1.smithi148.stdout:1/84: dwrite f6 [4194304,4194304] 0 2024-09-21T04:15:09.470 INFO:tasks.workunit.client.0.smithi110.stdout:0/346: write d4/f1f [2137527,112036] 0 2024-09-21T04:15:09.470 INFO:tasks.workunit.client.0.smithi110.stdout:0/347: write d4/f1f [3131258,116535] 0 2024-09-21T04:15:09.470 INFO:tasks.workunit.client.0.smithi110.stdout:0/348: stat d4/f1f 0 2024-09-21T04:15:09.470 INFO:tasks.workunit.client.1.smithi148.stdout:2/127: dread d0/ff [0,4194304] 0 2024-09-21T04:15:09.471 INFO:tasks.workunit.client.1.smithi148.stdout:2/128: truncate d0/f1d 655383 0 2024-09-21T04:15:09.471 INFO:tasks.workunit.client.1.smithi148.stdout:2/129: creat d0/d9/dc/f26 x:0 0 0 2024-09-21T04:15:09.471 INFO:tasks.workunit.client.1.smithi148.stdout:9/92: dwrite d5/f23 [0,4194304] 0 2024-09-21T04:15:09.473 INFO:tasks.workunit.client.0.smithi110.stdout:6/140: dwrite d4/fa [0,4194304] 0 2024-09-21T04:15:09.474 INFO:tasks.workunit.client.0.smithi110.stdout:6/141: chown d4/d12/d17/d27/f2f 1043686961 1 2024-09-21T04:15:09.474 INFO:tasks.workunit.client.0.smithi110.stdout:6/142: write f3 [2670858,51278] 0 2024-09-21T04:15:09.475 INFO:tasks.workunit.client.1.smithi148.stdout:9/93: dread d5/d1f/f24 [0,4194304] 0 2024-09-21T04:15:09.476 INFO:tasks.workunit.client.0.smithi110.stdout:6/143: unlink d4/de/l24 0 2024-09-21T04:15:09.478 INFO:tasks.workunit.client.0.smithi110.stdout:6/144: truncate d4/de/ff 2399466 0 2024-09-21T04:15:09.478 INFO:tasks.workunit.client.0.smithi110.stdout:6/145: chown d4/d12 690 1 2024-09-21T04:15:09.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/146: creat d4/d12/d17/d2e/f30 x:0 0 0 2024-09-21T04:15:09.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/147: write d4/f6 [5112535,33296] 0 2024-09-21T04:15:09.480 INFO:tasks.workunit.client.1.smithi148.stdout:0/103: rmdir d5/d6 39 2024-09-21T04:15:09.490 INFO:tasks.workunit.client.1.smithi148.stdout:0/104: dread d5/d6/d7/d11/f17 [4194304,4194304] 0 2024-09-21T04:15:09.490 INFO:tasks.workunit.client.1.smithi148.stdout:0/105: mkdir d5/d6/d7/d22 0 2024-09-21T04:15:09.496 INFO:tasks.workunit.client.1.smithi148.stdout:7/109: dwrite d5/f13 [0,4194304] 0 2024-09-21T04:15:09.496 INFO:tasks.workunit.client.1.smithi148.stdout:7/110: readlink d5/l1f 0 2024-09-21T04:15:09.497 INFO:tasks.workunit.client.0.smithi110.stdout:7/193: sync 2024-09-21T04:15:09.498 INFO:tasks.workunit.client.0.smithi110.stdout:8/218: sync 2024-09-21T04:15:09.499 INFO:tasks.workunit.client.0.smithi110.stdout:8/219: dread d0/d7/d13/f1e [0,4194304] 0 2024-09-21T04:15:09.501 INFO:tasks.workunit.client.0.smithi110.stdout:7/194: link dd/d14/c2d dd/d1c/c37 0 2024-09-21T04:15:09.501 INFO:tasks.workunit.client.0.smithi110.stdout:1/441: dwrite d2/d9/d26/d3d/f43 [0,4194304] 0 2024-09-21T04:15:09.501 INFO:tasks.workunit.client.0.smithi110.stdout:1/442: write d2/f45 [371789,52814] 0 2024-09-21T04:15:09.501 INFO:tasks.workunit.client.1.smithi148.stdout:5/52: dwrite d3/fb [0,4194304] 0 2024-09-21T04:15:09.501 INFO:tasks.workunit.client.1.smithi148.stdout:5/53: fsync d3/f7 0 2024-09-21T04:15:09.503 INFO:tasks.workunit.client.0.smithi110.stdout:8/220: creat d0/d7/d13/d4b/f4e x:0 0 0 2024-09-21T04:15:09.504 INFO:tasks.workunit.client.0.smithi110.stdout:5/208: dwrite d1/d22/f3c [4194304,4194304] 0 2024-09-21T04:15:09.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/85: dwrite db/dd/f12 [0,4194304] 0 2024-09-21T04:15:09.506 INFO:tasks.workunit.client.0.smithi110.stdout:1/443: dread d2/d9/d49/d5c/f8a [0,4194304] 0 2024-09-21T04:15:09.506 INFO:tasks.workunit.client.0.smithi110.stdout:1/444: dread - d2/d9/d26/d3d/d8f/f90 zero size 2024-09-21T04:15:09.506 INFO:tasks.workunit.client.0.smithi110.stdout:1/445: truncate d2/d9/d23/d4e/f61 4269307 0 2024-09-21T04:15:09.507 INFO:tasks.workunit.client.1.smithi148.stdout:2/130: dwrite d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:09.509 INFO:tasks.workunit.client.0.smithi110.stdout:8/221: truncate d0/d7/d13/f37 805910 0 2024-09-21T04:15:09.513 INFO:tasks.workunit.client.0.smithi110.stdout:0/349: dwrite d4/dd/d10/f1a [4194304,4194304] 0 2024-09-21T04:15:09.513 INFO:tasks.workunit.client.0.smithi110.stdout:1/446: dread d2/d9/d26/f2d [0,4194304] 0 2024-09-21T04:15:09.514 INFO:tasks.workunit.client.1.smithi148.stdout:1/86: readlink db/l17 0 2024-09-21T04:15:09.516 INFO:tasks.workunit.client.0.smithi110.stdout:8/222: mknod d0/d7/c4f 0 2024-09-21T04:15:09.516 INFO:tasks.workunit.client.0.smithi110.stdout:8/223: write d0/d7/fd [6152126,56635] 0 2024-09-21T04:15:09.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/447: mknod d2/d9/d49/d4f/c92 0 2024-09-21T04:15:09.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/224: creat d0/d7/d13/d1c/f50 x:0 0 0 2024-09-21T04:15:09.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/225: dread - d0/d7/d13/d1c/d41/f43 zero size 2024-09-21T04:15:09.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/226: write d0/d7/f20 [4237651,48192] 0 2024-09-21T04:15:09.524 INFO:tasks.workunit.client.1.smithi148.stdout:2/131: mkdir d0/d9/dc/d13/d27 0 2024-09-21T04:15:09.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/132: rename d0/d9/dc/d13/d25 to d0/d9/dc/d13/d25/d28 22 2024-09-21T04:15:09.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/133: getdents d0/d9/dc/d13/d27 0 2024-09-21T04:15:09.525 INFO:tasks.workunit.client.1.smithi148.stdout:0/106: dwrite d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:09.526 INFO:tasks.workunit.client.0.smithi110.stdout:1/448: mkdir d2/d9/d23/d3b/d93 0 2024-09-21T04:15:09.527 INFO:tasks.workunit.client.1.smithi148.stdout:5/54: getdents d3 0 2024-09-21T04:15:09.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/107: write d5/fe [316876,85092] 0 2024-09-21T04:15:09.531 INFO:tasks.workunit.client.1.smithi148.stdout:2/134: truncate d0/d9/dc/f12 2261614 0 2024-09-21T04:15:09.531 INFO:tasks.workunit.client.1.smithi148.stdout:2/135: write d0/f1f [969005,43687] 0 2024-09-21T04:15:09.534 INFO:tasks.workunit.client.1.smithi148.stdout:5/55: creat d3/f11 x:0 0 0 2024-09-21T04:15:09.535 INFO:tasks.workunit.client.1.smithi148.stdout:1/87: truncate f6 2894069 0 2024-09-21T04:15:09.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/108: creat d5/d6/d7/d11/d19/f23 x:0 0 0 2024-09-21T04:15:09.540 INFO:tasks.workunit.client.1.smithi148.stdout:0/109: creat d5/d6/d7/d22/f24 x:0 0 0 2024-09-21T04:15:09.546 INFO:tasks.workunit.client.0.smithi110.stdout:7/195: dwrite dd/d14/d17/f2b [0,4194304] 0 2024-09-21T04:15:09.548 INFO:tasks.workunit.client.1.smithi148.stdout:7/111: dwrite d5/dc/f1d [0,4194304] 0 2024-09-21T04:15:09.548 INFO:tasks.workunit.client.1.smithi148.stdout:7/112: write d5/d8/f9 [151453,52230] 0 2024-09-21T04:15:09.548 INFO:tasks.workunit.client.0.smithi110.stdout:0/350: dwrite d4/d1c/d62/f39 [0,4194304] 0 2024-09-21T04:15:09.549 INFO:tasks.workunit.client.0.smithi110.stdout:0/351: dread d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:09.549 INFO:tasks.workunit.client.0.smithi110.stdout:0/352: readlink d4/l4d 0 2024-09-21T04:15:09.549 INFO:tasks.workunit.client.0.smithi110.stdout:0/353: dread - d4/dd/d10/f41 zero size 2024-09-21T04:15:09.549 INFO:tasks.workunit.client.0.smithi110.stdout:0/354: write d4/dd/f38 [1016556,103917] 0 2024-09-21T04:15:09.550 INFO:tasks.workunit.client.0.smithi110.stdout:1/449: dwrite d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:09.550 INFO:tasks.workunit.client.1.smithi148.stdout:5/56: write d3/fb [1240802,103367] 0 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:3/65: sync 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:3/66: fdatasync f1 0 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:4/62: sync 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:3/67: fsync f1 0 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:3/68: read f1 [7646408,103783] 0 2024-09-21T04:15:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:9/94: dwrite d5/f23 [4194304,4194304] 0 2024-09-21T04:15:09.552 INFO:tasks.workunit.client.0.smithi110.stdout:8/227: dwrite d0/fb [4194304,4194304] 0 2024-09-21T04:15:09.552 INFO:tasks.workunit.client.0.smithi110.stdout:8/228: fsync d0/f3d 0 2024-09-21T04:15:09.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/355: mknod d4/d47/d2c/c65 0 2024-09-21T04:15:09.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/356: chown d4/f1f 240136409 1 2024-09-21T04:15:09.558 INFO:tasks.workunit.client.0.smithi110.stdout:0/357: truncate d4/f3b 3387593 0 2024-09-21T04:15:09.562 INFO:tasks.workunit.client.1.smithi148.stdout:4/63: mkdir d4/d12/d16 0 2024-09-21T04:15:09.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/69: symlink d3/lf 0 2024-09-21T04:15:09.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/70: chown d3/lf 191921 1 2024-09-21T04:15:09.563 INFO:tasks.workunit.client.1.smithi148.stdout:9/95: unlink d5/c15 0 2024-09-21T04:15:09.564 INFO:tasks.workunit.client.1.smithi148.stdout:7/113: getdents d5/d14 0 2024-09-21T04:15:09.566 INFO:tasks.workunit.client.1.smithi148.stdout:9/96: unlink d5/d1f/c21 0 2024-09-21T04:15:09.566 INFO:tasks.workunit.client.1.smithi148.stdout:9/97: chown d5/f1b 3 1 2024-09-21T04:15:09.567 INFO:tasks.workunit.client.0.smithi110.stdout:6/148: write d4/de/ff [848698,115676] 0 2024-09-21T04:15:09.567 INFO:tasks.workunit.client.1.smithi148.stdout:7/114: rmdir d5/dc/d23 0 2024-09-21T04:15:09.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/136: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:15:09.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/137: read - d0/d9/dc/d13/f21 zero size 2024-09-21T04:15:09.569 INFO:tasks.workunit.client.1.smithi148.stdout:9/98: unlink d5/d22/d28/c19 0 2024-09-21T04:15:09.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/57: dread d3/fd [0,4194304] 0 2024-09-21T04:15:09.571 INFO:tasks.workunit.client.1.smithi148.stdout:0/110: dwrite d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:09.572 INFO:tasks.workunit.client.1.smithi148.stdout:0/111: write d5/d6/f1f [1873917,48415] 0 2024-09-21T04:15:09.572 INFO:tasks.workunit.client.1.smithi148.stdout:7/115: unlink f0 0 2024-09-21T04:15:09.572 INFO:tasks.workunit.client.0.smithi110.stdout:1/450: read f1 [449310,48268] 0 2024-09-21T04:15:09.573 INFO:tasks.workunit.client.0.smithi110.stdout:6/149: rename d4/d12/d16/c23 to d4/d12/d16/c31 0 2024-09-21T04:15:09.573 INFO:tasks.workunit.client.0.smithi110.stdout:6/150: dread - d4/d12/d17/d27/f2f zero size 2024-09-21T04:15:09.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/138: creat d0/d9/f29 x:0 0 0 2024-09-21T04:15:09.575 INFO:tasks.workunit.client.1.smithi148.stdout:1/88: dread f6 [0,4194304] 0 2024-09-21T04:15:09.576 INFO:tasks.workunit.client.0.smithi110.stdout:1/451: link d2/d9/f6c d2/f94 0 2024-09-21T04:15:09.578 INFO:tasks.workunit.client.1.smithi148.stdout:9/99: unlink d5/f12 0 2024-09-21T04:15:09.578 INFO:tasks.workunit.client.0.smithi110.stdout:8/229: dwrite d0/d7/d13/d4b/f35 [0,4194304] 0 2024-09-21T04:15:09.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/58: readlink d3/lf 0 2024-09-21T04:15:09.581 INFO:tasks.workunit.client.1.smithi148.stdout:0/112: unlink d5/d6/d7/d11/d19/f23 0 2024-09-21T04:15:09.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/139: creat d0/d9/dc/d13/d25/f2a x:0 0 0 2024-09-21T04:15:09.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/89: write db/ff [4759890,17577] 0 2024-09-21T04:15:09.584 INFO:tasks.workunit.client.1.smithi148.stdout:9/100: truncate d5/fd 763745 0 2024-09-21T04:15:09.584 INFO:tasks.workunit.client.1.smithi148.stdout:9/101: chown d5/l17 4511561 1 2024-09-21T04:15:09.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/59: creat d3/f12 x:0 0 0 2024-09-21T04:15:09.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/113: symlink d5/l25 0 2024-09-21T04:15:09.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/114: chown d5/d6/f1f 43 1 2024-09-21T04:15:09.590 INFO:tasks.workunit.client.1.smithi148.stdout:9/102: rename d5/lb to d5/d1e/l2a 0 2024-09-21T04:15:09.590 INFO:tasks.workunit.client.1.smithi148.stdout:5/60: write d3/f8 [4589160,32372] 0 2024-09-21T04:15:09.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/90: link db/c16 db/dd/c1c 0 2024-09-21T04:15:09.592 INFO:tasks.workunit.client.1.smithi148.stdout:5/61: dread f0 [0,4194304] 0 2024-09-21T04:15:09.593 INFO:tasks.workunit.client.1.smithi148.stdout:0/115: rename d5/fe to d5/d6/d7/d11/d19/f26 0 2024-09-21T04:15:09.594 INFO:tasks.workunit.client.1.smithi148.stdout:9/103: mkdir d5/d2b 0 2024-09-21T04:15:09.598 INFO:tasks.workunit.client.1.smithi148.stdout:0/116: rename d5/d6/f1a to d5/d6/d7/d11/d19/f27 0 2024-09-21T04:15:09.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/104: creat d5/d22/d28/f2c x:0 0 0 2024-09-21T04:15:09.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/105: stat d5/l17 0 2024-09-21T04:15:09.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/106: readlink d5/l7 0 2024-09-21T04:15:09.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/117: mknod d5/d6/d7/d18/c28 0 2024-09-21T04:15:09.600 INFO:tasks.workunit.client.1.smithi148.stdout:9/107: link f0 d5/d1e/f2d 0 2024-09-21T04:15:09.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/49: sync 2024-09-21T04:15:09.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/50: readlink - no filename 2024-09-21T04:15:09.601 INFO:tasks.workunit.client.1.smithi148.stdout:6/44: sync 2024-09-21T04:15:09.601 INFO:tasks.workunit.client.1.smithi148.stdout:8/51: read - f9 zero size 2024-09-21T04:15:09.602 INFO:tasks.workunit.client.1.smithi148.stdout:6/45: creat d0/d5/dc/fd x:0 0 0 2024-09-21T04:15:09.602 INFO:tasks.workunit.client.1.smithi148.stdout:6/46: truncate d0/d5/fb 659368 0 2024-09-21T04:15:09.604 INFO:tasks.workunit.client.1.smithi148.stdout:6/47: rmdir d0/d5/d7 0 2024-09-21T04:15:09.610 INFO:tasks.workunit.client.1.smithi148.stdout:6/48: write d0/f1 [815207,48824] 0 2024-09-21T04:15:09.611 INFO:tasks.workunit.client.1.smithi148.stdout:6/49: write d0/d5/dc/fd [243159,6745] 0 2024-09-21T04:15:09.611 INFO:tasks.workunit.client.1.smithi148.stdout:6/50: write d0/d5/dc/fd [894483,63436] 0 2024-09-21T04:15:09.620 INFO:tasks.workunit.client.1.smithi148.stdout:0/118: dread d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:09.620 INFO:tasks.workunit.client.1.smithi148.stdout:0/119: dread - d5/d6/d7/d11/d19/f21 zero size 2024-09-21T04:15:09.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/120: creat d5/d6/d7/d11/f29 x:0 0 0 2024-09-21T04:15:09.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/121: read d5/d6/d7/d11/d19/f26 [247873,122979] 0 2024-09-21T04:15:09.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/122: mkdir d5/d6/d2a 0 2024-09-21T04:15:09.623 INFO:tasks.workunit.client.1.smithi148.stdout:0/123: dread - d5/d6/d7/d11/d19/f21 zero size 2024-09-21T04:15:09.631 INFO:tasks.workunit.client.0.smithi110.stdout:9/73: sync 2024-09-21T04:15:09.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/210: sync 2024-09-21T04:15:09.633 INFO:tasks.workunit.client.0.smithi110.stdout:9/74: rename d7/db/le to d7/db/d14/l16 0 2024-09-21T04:15:09.634 INFO:tasks.workunit.client.0.smithi110.stdout:6/151: dread d4/f13 [0,4194304] 0 2024-09-21T04:15:09.634 INFO:tasks.workunit.client.0.smithi110.stdout:6/152: write d4/d12/f1d [1753791,2150] 0 2024-09-21T04:15:09.635 INFO:tasks.workunit.client.0.smithi110.stdout:0/358: dwrite d4/f58 [0,4194304] 0 2024-09-21T04:15:09.635 INFO:tasks.workunit.client.0.smithi110.stdout:9/75: mkdir d7/db/d11/d17 0 2024-09-21T04:15:09.635 INFO:tasks.workunit.client.1.smithi148.stdout:4/64: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:09.635 INFO:tasks.workunit.client.1.smithi148.stdout:2/140: dwrite d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:09.635 INFO:tasks.workunit.client.1.smithi148.stdout:3/71: dwrite f1 [8388608,4194304] 0 2024-09-21T04:15:09.636 INFO:tasks.workunit.client.1.smithi148.stdout:1/91: dwrite db/dd/f13 [0,4194304] 0 2024-09-21T04:15:09.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/108: dwrite d5/f16 [4194304,4194304] 0 2024-09-21T04:15:09.638 INFO:tasks.workunit.client.0.smithi110.stdout:0/359: mkdir d4/dd/d53/d66 0 2024-09-21T04:15:09.640 INFO:tasks.workunit.client.1.smithi148.stdout:6/51: dread d0/d5/dc/fd [0,4194304] 0 2024-09-21T04:15:09.641 INFO:tasks.workunit.client.0.smithi110.stdout:9/76: truncate f3 2992157 0 2024-09-21T04:15:09.641 INFO:tasks.workunit.client.0.smithi110.stdout:9/77: dread - d7/d10/f13 zero size 2024-09-21T04:15:09.641 INFO:tasks.workunit.client.0.smithi110.stdout:9/78: readlink d7/db/ld 0 2024-09-21T04:15:09.643 INFO:tasks.workunit.client.1.smithi148.stdout:1/92: write f6 [494885,20724] 0 2024-09-21T04:15:09.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/62: dread d3/fb [0,4194304] 0 2024-09-21T04:15:09.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/63: dread - d3/f12 zero size 2024-09-21T04:15:09.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/65: read d4/f8 [2815210,66536] 0 2024-09-21T04:15:09.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/66: getdents d4/d12/d16 0 2024-09-21T04:15:09.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/67: fdatasync f3 0 2024-09-21T04:15:09.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/68: creat d4/f17 x:0 0 0 2024-09-21T04:15:09.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/69: write d4/f9 [344,98971] 0 2024-09-21T04:15:09.648 INFO:tasks.workunit.client.0.smithi110.stdout:0/360: mknod d4/dd/d53/d66/c67 0 2024-09-21T04:15:09.649 INFO:tasks.workunit.client.0.smithi110.stdout:0/361: truncate d4/dd/f1b 5075923 0 2024-09-21T04:15:09.655 INFO:tasks.workunit.client.0.smithi110.stdout:9/79: mkdir d7/d18 0 2024-09-21T04:15:09.655 INFO:tasks.workunit.client.0.smithi110.stdout:9/80: fsync d7/d10/f12 0 2024-09-21T04:15:09.655 INFO:tasks.workunit.client.0.smithi110.stdout:9/81: rename d7/db/d14 to d7/db/d14/d19 22 2024-09-21T04:15:09.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/93: dread db/dd/f12 [0,4194304] 0 2024-09-21T04:15:09.660 INFO:tasks.workunit.client.1.smithi148.stdout:2/141: mkdir d0/d2b 0 2024-09-21T04:15:09.660 INFO:tasks.workunit.client.1.smithi148.stdout:2/142: chown d0/c20 30 1 2024-09-21T04:15:09.663 INFO:tasks.workunit.client.1.smithi148.stdout:9/109: mkdir d5/d22/d2e 0 2024-09-21T04:15:09.664 INFO:tasks.workunit.client.1.smithi148.stdout:3/72: dwrite f1 [0,4194304] 0 2024-09-21T04:15:09.666 INFO:tasks.workunit.client.0.smithi110.stdout:0/362: dread d4/dd/d10/d29/d45/f5b [0,4194304] 0 2024-09-21T04:15:09.666 INFO:tasks.workunit.client.0.smithi110.stdout:6/153: dwrite d4/f6 [4194304,4194304] 0 2024-09-21T04:15:09.666 INFO:tasks.workunit.client.0.smithi110.stdout:0/363: write d4/dd/d53/f5a [532363,6335] 0 2024-09-21T04:15:09.669 INFO:tasks.workunit.client.0.smithi110.stdout:0/364: symlink d4/d1c/d62/l68 0 2024-09-21T04:15:09.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/70: mknod d4/d12/c18 0 2024-09-21T04:15:09.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/71: write d4/f11 [555855,23987] 0 2024-09-21T04:15:09.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/72: write d4/f11 [761430,7320] 0 2024-09-21T04:15:09.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/73: chown d4 21165 1 2024-09-21T04:15:09.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/143: rename d0/d9/dc/d13/d27 to d0/d9/dc/d13/d2c 0 2024-09-21T04:15:09.675 INFO:tasks.workunit.client.0.smithi110.stdout:9/82: getdents d7/db 0 2024-09-21T04:15:09.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/73: rename d3/lf to d3/d5/d8/da/l10 0 2024-09-21T04:15:09.677 INFO:tasks.workunit.client.0.smithi110.stdout:9/83: mknod d7/db/df/c1a 0 2024-09-21T04:15:09.677 INFO:tasks.workunit.client.0.smithi110.stdout:9/84: getdents d7/db 0 2024-09-21T04:15:09.678 INFO:tasks.workunit.client.1.smithi148.stdout:5/64: symlink d3/l13 0 2024-09-21T04:15:09.678 INFO:tasks.workunit.client.1.smithi148.stdout:5/65: fdatasync d3/f8 0 2024-09-21T04:15:09.679 INFO:tasks.workunit.client.1.smithi148.stdout:4/74: creat d4/d12/d16/f19 x:0 0 0 2024-09-21T04:15:09.680 INFO:tasks.workunit.client.0.smithi110.stdout:9/85: rename d7/db/d11/d17 to d7/d10/d1b 0 2024-09-21T04:15:09.680 INFO:tasks.workunit.client.0.smithi110.stdout:9/86: dread - d7/d10/f12 zero size 2024-09-21T04:15:09.680 INFO:tasks.workunit.client.0.smithi110.stdout:9/87: chown d7/db 106912031 1 2024-09-21T04:15:09.682 INFO:tasks.workunit.client.1.smithi148.stdout:1/94: dwrite db/dd/f13 [0,4194304] 0 2024-09-21T04:15:09.682 INFO:tasks.workunit.client.1.smithi148.stdout:6/52: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:15:09.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/144: mkdir d0/d9/dc/d13/d25/d2d 0 2024-09-21T04:15:09.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/145: dread - d0/f23 zero size 2024-09-21T04:15:09.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/146: truncate d0/d9/dc/d13/f1a 139349 0 2024-09-21T04:15:09.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/110: mkdir d5/d22/d28/d2f 0 2024-09-21T04:15:09.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/111: creat d5/d22/d28/f30 x:0 0 0 2024-09-21T04:15:09.685 INFO:tasks.workunit.client.1.smithi148.stdout:3/74: getdents d3/d5/d8 0 2024-09-21T04:15:09.687 INFO:tasks.workunit.client.0.smithi110.stdout:9/88: read d7/f8 [43842,103893] 0 2024-09-21T04:15:09.688 INFO:tasks.workunit.client.1.smithi148.stdout:5/66: write d3/f9 [395520,56485] 0 2024-09-21T04:15:09.688 INFO:tasks.workunit.client.1.smithi148.stdout:5/67: fsync d3/fb 0 2024-09-21T04:15:09.692 INFO:tasks.workunit.client.1.smithi148.stdout:3/75: creat d3/d5/d8/da/f11 x:0 0 0 2024-09-21T04:15:09.692 INFO:tasks.workunit.client.1.smithi148.stdout:3/76: read - d3/d5/d8/da/f11 zero size 2024-09-21T04:15:09.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/95: symlink db/l1d 0 2024-09-21T04:15:09.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/96: fsync db/dd/f13 0 2024-09-21T04:15:09.696 INFO:tasks.workunit.client.0.smithi110.stdout:0/365: dread d4/f58 [0,4194304] 0 2024-09-21T04:15:09.697 INFO:tasks.workunit.client.1.smithi148.stdout:6/53: read d0/f1 [1714202,76754] 0 2024-09-21T04:15:09.697 INFO:tasks.workunit.client.1.smithi148.stdout:6/54: chown d0/d5 6 1 2024-09-21T04:15:09.698 INFO:tasks.workunit.client.1.smithi148.stdout:6/55: write d0/f1 [1118950,44535] 0 2024-09-21T04:15:09.698 INFO:tasks.workunit.client.1.smithi148.stdout:6/56: chown d0/d5/fb 0 1 2024-09-21T04:15:09.698 INFO:tasks.workunit.client.0.smithi110.stdout:6/154: dread d4/d12/f1d [0,4194304] 0 2024-09-21T04:15:09.699 INFO:tasks.workunit.client.1.smithi148.stdout:5/68: mkdir d3/d14 0 2024-09-21T04:15:09.699 INFO:tasks.workunit.client.1.smithi148.stdout:5/69: write d3/f11 [841059,15318] 0 2024-09-21T04:15:09.699 INFO:tasks.workunit.client.1.smithi148.stdout:5/70: chown d3/fb 4 1 2024-09-21T04:15:09.699 INFO:tasks.workunit.client.0.smithi110.stdout:9/89: dread d7/f8 [0,4194304] 0 2024-09-21T04:15:09.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/90: dread d7/f8 [0,4194304] 0 2024-09-21T04:15:09.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/91: chown d7/db 24583 1 2024-09-21T04:15:09.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/92: readlink d7/db/lc 0 2024-09-21T04:15:09.702 INFO:tasks.workunit.client.0.smithi110.stdout:0/366: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:09.704 INFO:tasks.workunit.client.1.smithi148.stdout:9/112: link d5/c14 d5/d2b/c31 0 2024-09-21T04:15:09.705 INFO:tasks.workunit.client.1.smithi148.stdout:6/57: symlink d0/d8/le 0 2024-09-21T04:15:09.705 INFO:tasks.workunit.client.1.smithi148.stdout:6/58: chown d0/d5/ca 186530333 1 2024-09-21T04:15:09.706 INFO:tasks.workunit.client.0.smithi110.stdout:0/367: mkdir d4/d1c/d62/d69 0 2024-09-21T04:15:09.707 INFO:tasks.workunit.client.0.smithi110.stdout:0/368: stat d4/d47 0 2024-09-21T04:15:09.710 INFO:tasks.workunit.client.0.smithi110.stdout:9/93: write f2 [495454,107138] 0 2024-09-21T04:15:09.710 INFO:tasks.workunit.client.0.smithi110.stdout:9/94: write d7/f8 [60888,76211] 0 2024-09-21T04:15:09.710 INFO:tasks.workunit.client.1.smithi148.stdout:4/75: dwrite d4/f8 [4194304,4194304] 0 2024-09-21T04:15:09.710 INFO:tasks.workunit.client.1.smithi148.stdout:4/76: fsync d4/f6 0 2024-09-21T04:15:09.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/97: symlink db/l1e 0 2024-09-21T04:15:09.711 INFO:tasks.workunit.client.1.smithi148.stdout:2/147: dwrite d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:09.711 INFO:tasks.workunit.client.1.smithi148.stdout:2/148: chown d0/d9/d16 586 1 2024-09-21T04:15:09.711 INFO:tasks.workunit.client.1.smithi148.stdout:2/149: chown d0/d9/dc/f12 14368 1 2024-09-21T04:15:09.712 INFO:tasks.workunit.client.0.smithi110.stdout:0/369: symlink d4/d47/l6a 0 2024-09-21T04:15:09.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/155: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:09.714 INFO:tasks.workunit.client.1.smithi148.stdout:9/113: readlink d5/l17 0 2024-09-21T04:15:09.717 INFO:tasks.workunit.client.1.smithi148.stdout:5/71: creat d3/dc/f15 x:0 0 0 2024-09-21T04:15:09.719 INFO:tasks.workunit.client.0.smithi110.stdout:9/95: rmdir d7/d10/d1b 0 2024-09-21T04:15:09.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/370: write d4/d1c/d62/f39 [3324805,52619] 0 2024-09-21T04:15:09.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/371: readlink d4/dd/d10/d29/d45/l61 0 2024-09-21T04:15:09.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/372: readlink d4/d47/l6a 0 2024-09-21T04:15:09.721 INFO:tasks.workunit.client.0.smithi110.stdout:6/156: rmdir d4/d12/d16 39 2024-09-21T04:15:09.722 INFO:tasks.workunit.client.0.smithi110.stdout:0/373: fsync d4/dd/d53/f5a 0 2024-09-21T04:15:09.722 INFO:tasks.workunit.client.1.smithi148.stdout:5/72: fdatasync d3/f9 0 2024-09-21T04:15:09.722 INFO:tasks.workunit.client.1.smithi148.stdout:6/59: dwrite d0/d5/fb [0,4194304] 0 2024-09-21T04:15:09.722 INFO:tasks.workunit.client.1.smithi148.stdout:6/60: chown d0/d5/dc 1785256 1 2024-09-21T04:15:09.722 INFO:tasks.workunit.client.1.smithi148.stdout:6/61: creat d0/d5/ff x:0 0 0 2024-09-21T04:15:09.723 INFO:tasks.workunit.client.0.smithi110.stdout:9/96: creat d7/db/f1c x:0 0 0 2024-09-21T04:15:09.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/77: rename d3/d5/d8 to d3/d5/d6/d12 0 2024-09-21T04:15:09.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/78: write d3/d5/d6/d12/da/f11 [430967,122698] 0 2024-09-21T04:15:09.730 INFO:tasks.workunit.client.0.smithi110.stdout:6/157: link d4/d12/l2c d4/l32 0 2024-09-21T04:15:09.734 INFO:tasks.workunit.client.0.smithi110.stdout:0/374: unlink d4/dd/d10/d29/d45/l61 0 2024-09-21T04:15:09.734 INFO:tasks.workunit.client.0.smithi110.stdout:0/375: creat d4/dd/d10/d29/f6b x:0 0 0 2024-09-21T04:15:09.736 INFO:tasks.workunit.client.0.smithi110.stdout:9/97: unlink c5 0 2024-09-21T04:15:09.737 INFO:tasks.workunit.client.1.smithi148.stdout:5/73: dwrite f0 [0,4194304] 0 2024-09-21T04:15:09.738 INFO:tasks.workunit.client.1.smithi148.stdout:4/77: link d4/l7 d4/d12/l1a 0 2024-09-21T04:15:09.738 INFO:tasks.workunit.client.1.smithi148.stdout:4/78: chown d4/f11 375869 1 2024-09-21T04:15:09.739 INFO:tasks.workunit.client.1.smithi148.stdout:4/79: truncate d4/d12/d16/f19 604939 0 2024-09-21T04:15:09.739 INFO:tasks.workunit.client.0.smithi110.stdout:6/158: truncate f3 524596 0 2024-09-21T04:15:09.740 INFO:tasks.workunit.client.1.smithi148.stdout:1/98: write db/f15 [3584051,2463] 0 2024-09-21T04:15:09.743 INFO:tasks.workunit.client.0.smithi110.stdout:0/376: creat d4/d47/d2c/d3a/f6c x:0 0 0 2024-09-21T04:15:09.745 INFO:tasks.workunit.client.0.smithi110.stdout:9/98: mknod d7/db/c1d 0 2024-09-21T04:15:09.745 INFO:tasks.workunit.client.0.smithi110.stdout:9/99: stat l4 0 2024-09-21T04:15:09.747 INFO:tasks.workunit.client.0.smithi110.stdout:6/159: mkdir d4/d12/d17/d33 0 2024-09-21T04:15:09.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/99: write db/dd/f13 [3551717,114548] 0 2024-09-21T04:15:09.749 INFO:tasks.workunit.client.1.smithi148.stdout:2/150: creat d0/d9/dc/d13/d25/d2d/f2e x:0 0 0 2024-09-21T04:15:09.750 INFO:tasks.workunit.client.0.smithi110.stdout:0/377: symlink d4/d1c/l6d 0 2024-09-21T04:15:09.751 INFO:tasks.workunit.client.1.smithi148.stdout:9/114: rmdir d5/d22/d2e 0 2024-09-21T04:15:09.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/100: dread db/ff [4194304,4194304] 0 2024-09-21T04:15:09.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/116: sync 2024-09-21T04:15:09.754 INFO:tasks.workunit.client.1.smithi148.stdout:8/52: sync 2024-09-21T04:15:09.756 INFO:tasks.workunit.client.0.smithi110.stdout:0/378: mknod d4/d1c/c6e 0 2024-09-21T04:15:09.757 INFO:tasks.workunit.client.1.smithi148.stdout:3/79: mkdir d3/d13 0 2024-09-21T04:15:09.761 INFO:tasks.workunit.client.0.smithi110.stdout:6/160: dread d4/de/f11 [0,4194304] 0 2024-09-21T04:15:09.761 INFO:tasks.workunit.client.1.smithi148.stdout:9/115: chown d5/d1e/c27 1362331836 1 2024-09-21T04:15:09.762 INFO:tasks.workunit.client.0.smithi110.stdout:0/379: dread d4/dd/f38 [0,4194304] 0 2024-09-21T04:15:09.763 INFO:tasks.workunit.client.1.smithi148.stdout:1/101: creat db/f1f x:0 0 0 2024-09-21T04:15:09.763 INFO:tasks.workunit.client.0.smithi110.stdout:6/161: rename d4/d12/d17/d2e/f30 to d4/d12/f34 0 2024-09-21T04:15:09.764 INFO:tasks.workunit.client.0.smithi110.stdout:6/162: chown d4/de/c29 3147 1 2024-09-21T04:15:09.764 INFO:tasks.workunit.client.1.smithi148.stdout:8/53: dread - f8 zero size 2024-09-21T04:15:09.764 INFO:tasks.workunit.client.0.smithi110.stdout:2/83: sync 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/380: symlink d4/d1c/d62/d69/l6f 0 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/381: truncate d4/d47/d2c/d3a/f51 1841354 0 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/382: write d4/d47/f40 [721536,129874] 0 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/383: creat d4/dd/d10/f70 x:0 0 0 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/384: fdatasync d4/f31 0 2024-09-21T04:15:09.766 INFO:tasks.workunit.client.0.smithi110.stdout:0/385: dread - d4/f57 zero size 2024-09-21T04:15:09.767 INFO:tasks.workunit.client.0.smithi110.stdout:0/386: truncate d4/d47/d2c/d3a/f43 1041368 0 2024-09-21T04:15:09.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/117: rmdir d5 39 2024-09-21T04:15:09.767 INFO:tasks.workunit.client.0.smithi110.stdout:6/163: symlink d4/d12/d17/l35 0 2024-09-21T04:15:09.768 INFO:tasks.workunit.client.1.smithi148.stdout:2/151: mkdir d0/d2f 0 2024-09-21T04:15:09.772 INFO:tasks.workunit.client.1.smithi148.stdout:9/116: fdatasync d5/d1f/f24 0 2024-09-21T04:15:09.775 INFO:tasks.workunit.client.0.smithi110.stdout:6/164: rename d4/d12/d17/l18 to d4/d12/d16/l36 0 2024-09-21T04:15:09.775 INFO:tasks.workunit.client.0.smithi110.stdout:2/84: write d2/f8 [3450462,21185] 0 2024-09-21T04:15:09.775 INFO:tasks.workunit.client.0.smithi110.stdout:2/85: chown d2/d3/da/de/l11 457 1 2024-09-21T04:15:09.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/102: chown db/lc 528344663 1 2024-09-21T04:15:09.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/152: rename d0/d9/dc/f26 to d0/d2f/f30 0 2024-09-21T04:15:09.779 INFO:tasks.workunit.client.0.smithi110.stdout:6/165: mknod d4/d12/d17/d27/c37 0 2024-09-21T04:15:09.781 INFO:tasks.workunit.client.1.smithi148.stdout:9/117: mkdir d5/d2b/d32 0 2024-09-21T04:15:09.783 INFO:tasks.workunit.client.1.smithi148.stdout:9/118: link d5/d2b/c31 d5/d1e/c33 0 2024-09-21T04:15:09.796 INFO:tasks.workunit.client.0.smithi110.stdout:6/166: read d4/d12/f2b [544310,130096] 0 2024-09-21T04:15:09.797 INFO:tasks.workunit.client.0.smithi110.stdout:6/167: mkdir d4/d12/d17/d2e/d38 0 2024-09-21T04:15:09.797 INFO:tasks.workunit.client.0.smithi110.stdout:6/168: stat d4/fa 0 2024-09-21T04:15:09.815 INFO:tasks.workunit.client.1.smithi148.stdout:4/80: dwrite d4/f8 [4194304,4194304] 0 2024-09-21T04:15:09.817 INFO:tasks.workunit.client.1.smithi148.stdout:4/81: getdents d4 0 2024-09-21T04:15:09.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/82: chown d4/fb 917920 1 2024-09-21T04:15:09.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/83: creat d4/d12/f1b x:0 0 0 2024-09-21T04:15:09.818 INFO:tasks.workunit.client.1.smithi148.stdout:4/84: symlink d4/d12/l1c 0 2024-09-21T04:15:09.821 INFO:tasks.workunit.client.1.smithi148.stdout:8/54: dwrite f9 [0,4194304] 0 2024-09-21T04:15:09.821 INFO:tasks.workunit.client.1.smithi148.stdout:2/153: dwrite d0/f1d [0,4194304] 0 2024-09-21T04:15:09.822 INFO:tasks.workunit.client.1.smithi148.stdout:9/119: dwrite d5/d22/d28/f2c [0,4194304] 0 2024-09-21T04:15:09.823 INFO:tasks.workunit.client.1.smithi148.stdout:8/55: truncate f0 7937186 0 2024-09-21T04:15:09.824 INFO:tasks.workunit.client.0.smithi110.stdout:2/86: dwrite d2/d3/f12 [0,4194304] 0 2024-09-21T04:15:09.824 INFO:tasks.workunit.client.0.smithi110.stdout:6/169: dwrite d4/fa [4194304,4194304] 0 2024-09-21T04:15:09.824 INFO:tasks.workunit.client.0.smithi110.stdout:6/170: write d4/d12/f34 [338553,32674] 0 2024-09-21T04:15:09.826 INFO:tasks.workunit.client.1.smithi148.stdout:2/154: symlink d0/d9/dc/l31 0 2024-09-21T04:15:09.827 INFO:tasks.workunit.client.1.smithi148.stdout:2/155: getdents d0/d2b 0 2024-09-21T04:15:09.827 INFO:tasks.workunit.client.0.smithi110.stdout:6/171: write d4/f7 [1779343,77512] 0 2024-09-21T04:15:09.828 INFO:tasks.workunit.client.0.smithi110.stdout:2/87: creat d2/f16 x:0 0 0 2024-09-21T04:15:09.829 INFO:tasks.workunit.client.1.smithi148.stdout:7/118: dwrite d5/dc/f19 [4194304,4194304] 0 2024-09-21T04:15:09.831 INFO:tasks.workunit.client.0.smithi110.stdout:6/172: link d4/l1a d4/d12/d17/d33/l39 0 2024-09-21T04:15:09.834 INFO:tasks.workunit.client.1.smithi148.stdout:2/156: creat d0/d2b/f32 x:0 0 0 2024-09-21T04:15:09.836 INFO:tasks.workunit.client.0.smithi110.stdout:6/173: link d4/c1b d4/d12/d17/d27/c3a 0 2024-09-21T04:15:09.836 INFO:tasks.workunit.client.1.smithi148.stdout:4/85: getdents d4/d12 0 2024-09-21T04:15:09.837 INFO:tasks.workunit.client.1.smithi148.stdout:7/119: mkdir d5/d14/d25 0 2024-09-21T04:15:09.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/88: rename d2/d3/da to d2/d3/d9/d17 0 2024-09-21T04:15:09.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/89: readlink d2/d3/d9/l10 0 2024-09-21T04:15:09.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/90: chown d2/d3/f12 93626 1 2024-09-21T04:15:09.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/120: getdents d5/d1f 0 2024-09-21T04:15:09.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/121: truncate d5/d1f/f24 814117 0 2024-09-21T04:15:09.841 INFO:tasks.workunit.client.1.smithi148.stdout:8/56: truncate f6 596501 0 2024-09-21T04:15:09.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/86: symlink d4/d12/d16/l1d 0 2024-09-21T04:15:09.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/87: truncate d4/f10 822551 0 2024-09-21T04:15:09.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/122: truncate d5/f1b 661604 0 2024-09-21T04:15:09.845 INFO:tasks.workunit.client.1.smithi148.stdout:8/57: write f7 [1172277,109762] 0 2024-09-21T04:15:09.845 INFO:tasks.workunit.client.1.smithi148.stdout:2/157: rmdir d0/d9/dc/d13/d25/d2d 39 2024-09-21T04:15:09.848 INFO:tasks.workunit.client.0.smithi110.stdout:6/174: dread d4/de/f1f [0,4194304] 0 2024-09-21T04:15:09.849 INFO:tasks.workunit.client.1.smithi148.stdout:4/88: rename f3 to d4/d12/f1e 0 2024-09-21T04:15:09.849 INFO:tasks.workunit.client.1.smithi148.stdout:4/89: dread d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:09.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/90: chown d4/c13 1466 1 2024-09-21T04:15:09.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/91: creat d4/d12/d16/f1f x:0 0 0 2024-09-21T04:15:09.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/91: dread d2/d3/f12 [0,4194304] 0 2024-09-21T04:15:09.851 INFO:tasks.workunit.client.1.smithi148.stdout:7/120: dwrite d5/d8/f1b [0,4194304] 0 2024-09-21T04:15:09.853 INFO:tasks.workunit.client.0.smithi110.stdout:6/175: symlink d4/d12/d17/d27/l3b 0 2024-09-21T04:15:09.855 INFO:tasks.workunit.client.1.smithi148.stdout:8/58: getdents . 0 2024-09-21T04:15:09.855 INFO:tasks.workunit.client.1.smithi148.stdout:8/59: getdents . 0 2024-09-21T04:15:09.856 INFO:tasks.workunit.client.0.smithi110.stdout:2/92: truncate d2/d3/fd 301446 0 2024-09-21T04:15:09.856 INFO:tasks.workunit.client.0.smithi110.stdout:2/93: dread - d2/f16 zero size 2024-09-21T04:15:09.857 INFO:tasks.workunit.client.1.smithi148.stdout:2/158: creat d0/d9/dc/d13/d2c/f33 x:0 0 0 2024-09-21T04:15:09.857 INFO:tasks.workunit.client.1.smithi148.stdout:2/159: chown d0/d9/f1b 692313 1 2024-09-21T04:15:09.858 INFO:tasks.workunit.client.1.smithi148.stdout:2/160: write d0/f1f [696372,79413] 0 2024-09-21T04:15:09.859 INFO:tasks.workunit.client.0.smithi110.stdout:2/94: mkdir d2/d3/d9/d17/de/d18 0 2024-09-21T04:15:09.862 INFO:tasks.workunit.client.0.smithi110.stdout:0/387: dwrite d4/dd/d10/f70 [0,4194304] 0 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.0.smithi110.stdout:6/176: dread d4/de/f11 [0,4194304] 0 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.0.smithi110.stdout:6/177: chown d4/fa 4 1 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.0.smithi110.stdout:6/178: write d4/d12/f2b [2092970,18338] 0 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/80: dwrite f1 [4194304,4194304] 0 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/81: write d3/d5/d6/d12/da/f11 [1135476,115953] 0 2024-09-21T04:15:09.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/82: chown d3/d5/d6/d12 3991007 1 2024-09-21T04:15:09.864 INFO:tasks.workunit.client.0.smithi110.stdout:0/388: dread d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:09.864 INFO:tasks.workunit.client.0.smithi110.stdout:0/389: rename d4/d47/d2c to d4/d47/d2c/d3a/d71 22 2024-09-21T04:15:09.864 INFO:tasks.workunit.client.0.smithi110.stdout:2/95: mkdir d2/d3/d19 0 2024-09-21T04:15:09.864 INFO:tasks.workunit.client.0.smithi110.stdout:2/96: fsync d2/d3/f12 0 2024-09-21T04:15:09.865 INFO:tasks.workunit.client.1.smithi148.stdout:1/103: dwrite f6 [0,4194304] 0 2024-09-21T04:15:09.865 INFO:tasks.workunit.client.1.smithi148.stdout:1/104: chown db/l1e 409658236 1 2024-09-21T04:15:09.865 INFO:tasks.workunit.client.1.smithi148.stdout:1/105: creat db/dd/f20 x:0 0 0 2024-09-21T04:15:09.865 INFO:tasks.workunit.client.1.smithi148.stdout:1/106: dread - db/dd/f20 zero size 2024-09-21T04:15:09.865 INFO:tasks.workunit.client.1.smithi148.stdout:9/123: mknod d5/c34 0 2024-09-21T04:15:09.872 INFO:tasks.workunit.client.1.smithi148.stdout:7/121: mknod d5/d8/c26 0 2024-09-21T04:15:09.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/122: fdatasync d5/d8/f1a 0 2024-09-21T04:15:09.873 INFO:tasks.workunit.client.0.smithi110.stdout:6/179: mkdir d4/d3c 0 2024-09-21T04:15:09.873 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:09 smithi148 ceph-mon[34957]: pgmap v269: 65 pgs: 65 active+clean; 499 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 41 MiB/s wr, 645 op/s 2024-09-21T04:15:09.874 INFO:tasks.workunit.client.0.smithi110.stdout:6/180: truncate d4/de/f10 168797 0 2024-09-21T04:15:09.876 INFO:tasks.workunit.client.0.smithi110.stdout:6/181: creat d4/f3d x:0 0 0 2024-09-21T04:15:09.878 INFO:tasks.workunit.client.0.smithi110.stdout:6/182: mkdir d4/d12/d17/d33/d3e 0 2024-09-21T04:15:09.878 INFO:tasks.workunit.client.0.smithi110.stdout:6/183: fsync f3 0 2024-09-21T04:15:09.880 INFO:tasks.workunit.client.1.smithi148.stdout:8/60: creat fb x:0 0 0 2024-09-21T04:15:09.882 INFO:tasks.workunit.client.1.smithi148.stdout:2/161: symlink d0/d9/d16/l34 0 2024-09-21T04:15:09.883 INFO:tasks.workunit.client.1.smithi148.stdout:4/92: getdents d4/d12/d16 0 2024-09-21T04:15:09.883 INFO:tasks.workunit.client.1.smithi148.stdout:4/93: write d4/d12/f15 [523744,61954] 0 2024-09-21T04:15:09.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/94: write d4/f10 [298298,44052] 0 2024-09-21T04:15:09.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/95: fdatasync d4/d12/d16/f1f 0 2024-09-21T04:15:09.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/96: write d4/f6 [2430711,129782] 0 2024-09-21T04:15:09.885 INFO:tasks.workunit.client.1.smithi148.stdout:4/97: chown d4/f9 401396648 1 2024-09-21T04:15:09.885 INFO:tasks.workunit.client.1.smithi148.stdout:4/98: stat d4/c5 0 2024-09-21T04:15:09.885 INFO:tasks.workunit.client.1.smithi148.stdout:4/99: write d4/d12/f1e [2388052,82369] 0 2024-09-21T04:15:09.885 INFO:tasks.workunit.client.1.smithi148.stdout:3/83: symlink d3/d5/d6/l14 0 2024-09-21T04:15:09.886 INFO:tasks.workunit.client.1.smithi148.stdout:9/124: mknod d5/d1f/c35 0 2024-09-21T04:15:09.887 INFO:tasks.workunit.client.1.smithi148.stdout:7/123: unlink d5/d14/f18 0 2024-09-21T04:15:09.888 INFO:tasks.workunit.client.1.smithi148.stdout:7/124: truncate d5/d8/ff 81488 0 2024-09-21T04:15:09.888 INFO:tasks.workunit.client.1.smithi148.stdout:2/162: creat d0/d2f/f35 x:0 0 0 2024-09-21T04:15:09.890 INFO:tasks.workunit.client.1.smithi148.stdout:4/100: unlink d4/d12/f1e 0 2024-09-21T04:15:09.891 INFO:tasks.workunit.client.1.smithi148.stdout:9/125: creat d5/f36 x:0 0 0 2024-09-21T04:15:09.892 INFO:tasks.workunit.client.1.smithi148.stdout:8/61: mkdir dc 0 2024-09-21T04:15:09.892 INFO:tasks.workunit.client.1.smithi148.stdout:8/62: getdents dc 0 2024-09-21T04:15:09.893 INFO:tasks.workunit.client.1.smithi148.stdout:1/107: dwrite db/dd/f20 [0,4194304] 0 2024-09-21T04:15:09.894 INFO:tasks.workunit.client.1.smithi148.stdout:7/125: getdents d5/dc 0 2024-09-21T04:15:09.894 INFO:tasks.workunit.client.1.smithi148.stdout:7/126: chown d5/d14/d25 534 1 2024-09-21T04:15:09.894 INFO:tasks.workunit.client.1.smithi148.stdout:8/63: fsync f6 0 2024-09-21T04:15:09.897 INFO:tasks.workunit.client.1.smithi148.stdout:2/163: dread d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:09.897 INFO:tasks.workunit.client.1.smithi148.stdout:2/164: fsync d0/d9/f1b 0 2024-09-21T04:15:09.899 INFO:tasks.workunit.client.1.smithi148.stdout:9/126: write d5/d22/d28/f2c [217491,110160] 0 2024-09-21T04:15:09.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/165: truncate d0/d9/f17 2854303 0 2024-09-21T04:15:09.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/166: chown d0/d9/dc/d13/d25/d2d/f2e 522 1 2024-09-21T04:15:09.901 INFO:tasks.workunit.client.1.smithi148.stdout:8/64: unlink f4 0 2024-09-21T04:15:09.901 INFO:tasks.workunit.client.1.smithi148.stdout:8/65: write f6 [1335892,24122] 0 2024-09-21T04:15:09.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/127: dread f0 [0,4194304] 0 2024-09-21T04:15:09.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/128: write d5/f36 [243015,95182] 0 2024-09-21T04:15:09.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/390: dwrite d4/f56 [0,4194304] 0 2024-09-21T04:15:09.903 INFO:tasks.workunit.client.1.smithi148.stdout:8/66: chown f7 3 1 2024-09-21T04:15:09.903 INFO:tasks.workunit.client.0.smithi110.stdout:2/97: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:15:09.903 INFO:tasks.workunit.client.0.smithi110.stdout:2/98: readlink d2/d3/d9/d17/de/l11 0 2024-09-21T04:15:09.904 INFO:tasks.workunit.client.1.smithi148.stdout:9/129: mkdir d5/d1e/d37 0 2024-09-21T04:15:09.905 INFO:tasks.workunit.client.1.smithi148.stdout:1/108: dread db/dd/f20 [0,4194304] 0 2024-09-21T04:15:09.905 INFO:tasks.workunit.client.1.smithi148.stdout:1/109: write db/dd/f13 [4263466,17980] 0 2024-09-21T04:15:09.905 INFO:tasks.workunit.client.0.smithi110.stdout:6/184: dwrite d4/d12/d17/d27/f2f [0,4194304] 0 2024-09-21T04:15:09.905 INFO:tasks.workunit.client.0.smithi110.stdout:6/185: write d4/fc [88775,64021] 0 2024-09-21T04:15:09.906 INFO:tasks.workunit.client.0.smithi110.stdout:6/186: chown d4/d12/d16/c31 0 1 2024-09-21T04:15:09.906 INFO:tasks.workunit.client.0.smithi110.stdout:6/187: chown d4/c1b 3229352 1 2024-09-21T04:15:09.906 INFO:tasks.workunit.client.0.smithi110.stdout:6/188: stat d4/d12/d17/d27/f2f 0 2024-09-21T04:15:09.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/130: mkdir d5/d22/d28/d38 0 2024-09-21T04:15:09.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/131: fdatasync d5/d1e/f29 0 2024-09-21T04:15:09.907 INFO:tasks.workunit.client.0.smithi110.stdout:2/99: fsync d2/f8 0 2024-09-21T04:15:09.907 INFO:tasks.workunit.client.1.smithi148.stdout:8/67: unlink f0 0 2024-09-21T04:15:09.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/110: mkdir db/dd/d21 0 2024-09-21T04:15:09.911 INFO:tasks.workunit.client.1.smithi148.stdout:7/127: dread d5/f13 [0,4194304] 0 2024-09-21T04:15:09.913 INFO:tasks.workunit.client.1.smithi148.stdout:8/68: chown ca 163 1 2024-09-21T04:15:09.913 INFO:tasks.workunit.client.1.smithi148.stdout:8/69: getdents dc 0 2024-09-21T04:15:09.914 INFO:tasks.workunit.client.1.smithi148.stdout:2/167: dread d0/f1f [0,4194304] 0 2024-09-21T04:15:09.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:09 smithi110 ceph-mon[27723]: pgmap v269: 65 pgs: 65 active+clean; 499 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 41 MiB/s wr, 645 op/s 2024-09-21T04:15:09.917 INFO:tasks.workunit.client.1.smithi148.stdout:9/132: mkdir d5/d39 0 2024-09-21T04:15:09.917 INFO:tasks.workunit.client.1.smithi148.stdout:9/133: write d5/d22/d28/f30 [11283,50452] 0 2024-09-21T04:15:09.919 INFO:tasks.workunit.client.1.smithi148.stdout:8/70: dread - f2 zero size 2024-09-21T04:15:09.921 INFO:tasks.workunit.client.1.smithi148.stdout:2/168: link d0/d9/l10 d0/d9/l36 0 2024-09-21T04:15:09.923 INFO:tasks.workunit.client.1.smithi148.stdout:3/84: truncate f1 549895 0 2024-09-21T04:15:09.924 INFO:tasks.workunit.client.1.smithi148.stdout:1/111: rename db/l17 to db/l22 0 2024-09-21T04:15:09.926 INFO:tasks.workunit.client.1.smithi148.stdout:8/71: mknod dc/cd 0 2024-09-21T04:15:09.926 INFO:tasks.workunit.client.1.smithi148.stdout:4/101: dwrite d4/f10 [0,4194304] 0 2024-09-21T04:15:09.926 INFO:tasks.workunit.client.1.smithi148.stdout:8/72: write f7 [258873,37349] 0 2024-09-21T04:15:09.927 INFO:tasks.workunit.client.1.smithi148.stdout:2/169: creat d0/d9/d16/f37 x:0 0 0 2024-09-21T04:15:09.928 INFO:tasks.workunit.client.1.smithi148.stdout:4/102: mknod d4/c20 0 2024-09-21T04:15:09.928 INFO:tasks.workunit.client.1.smithi148.stdout:4/103: stat d4/d12 0 2024-09-21T04:15:09.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/134: read f1 [2474118,74093] 0 2024-09-21T04:15:09.931 INFO:tasks.workunit.client.0.smithi110.stdout:6/189: symlink d4/d3c/l3f 0 2024-09-21T04:15:09.932 INFO:tasks.workunit.client.1.smithi148.stdout:2/170: unlink d0/d9/l10 0 2024-09-21T04:15:09.932 INFO:tasks.workunit.client.1.smithi148.stdout:2/171: chown d0/d9/dc 1438 1 2024-09-21T04:15:09.932 INFO:tasks.workunit.client.1.smithi148.stdout:0/124: sync 2024-09-21T04:15:09.933 INFO:tasks.workunit.client.1.smithi148.stdout:4/104: link d4/ld d4/d12/l21 0 2024-09-21T04:15:09.933 INFO:tasks.workunit.client.1.smithi148.stdout:1/112: dread f6 [0,4194304] 0 2024-09-21T04:15:09.933 INFO:tasks.workunit.client.1.smithi148.stdout:1/113: creat db/dd/f23 x:0 0 0 2024-09-21T04:15:09.934 INFO:tasks.workunit.client.1.smithi148.stdout:9/135: dread d5/d1f/f24 [0,4194304] 0 2024-09-21T04:15:09.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/172: unlink d0/d9/dc/d13/c19 0 2024-09-21T04:15:09.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/125: mkdir d5/d6/d2a/d2b 0 2024-09-21T04:15:09.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/126: fdatasync d5/d6/d7/d11/d19/f27 0 2024-09-21T04:15:09.936 INFO:tasks.workunit.client.1.smithi148.stdout:4/105: mkdir d4/d12/d22 0 2024-09-21T04:15:09.937 INFO:tasks.workunit.client.0.smithi110.stdout:6/190: read d4/d12/d17/d27/f2f [4043805,41910] 0 2024-09-21T04:15:09.939 INFO:tasks.workunit.client.0.smithi110.stdout:6/191: dread d4/de/f10 [0,4194304] 0 2024-09-21T04:15:09.939 INFO:tasks.workunit.client.0.smithi110.stdout:6/192: write d4/fc [177710,1717] 0 2024-09-21T04:15:09.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/136: symlink d5/d39/l3a 0 2024-09-21T04:15:09.941 INFO:tasks.workunit.client.0.smithi110.stdout:2/100: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:15:09.942 INFO:tasks.workunit.client.1.smithi148.stdout:2/173: mkdir d0/d9/d38 0 2024-09-21T04:15:09.942 INFO:tasks.workunit.client.1.smithi148.stdout:7/128: dwrite d5/d8/f22 [0,4194304] 0 2024-09-21T04:15:09.944 INFO:tasks.workunit.client.1.smithi148.stdout:0/127: creat d5/d6/d2a/d2b/f2c x:0 0 0 2024-09-21T04:15:09.944 INFO:tasks.workunit.client.1.smithi148.stdout:0/128: read d5/d6/d7/ff [224936,11235] 0 2024-09-21T04:15:09.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/106: mkdir d4/d23 0 2024-09-21T04:15:09.946 INFO:tasks.workunit.client.1.smithi148.stdout:4/107: chown d4/d12/f1b 89757 1 2024-09-21T04:15:09.947 INFO:tasks.workunit.client.0.smithi110.stdout:2/101: rename d2/d3/d9/d17 to d2/d1a 0 2024-09-21T04:15:09.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/85: dread d3/d5/d6/d12/da/f11 [0,4194304] 0 2024-09-21T04:15:09.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/86: write d3/d5/d6/d12/da/f11 [1760565,55838] 0 2024-09-21T04:15:09.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/87: write d3/d5/d6/d12/da/f11 [2295723,122282] 0 2024-09-21T04:15:09.949 INFO:tasks.workunit.client.1.smithi148.stdout:7/129: dread f4 [0,4194304] 0 2024-09-21T04:15:09.949 INFO:tasks.workunit.client.1.smithi148.stdout:1/114: mkdir db/d24 0 2024-09-21T04:15:09.949 INFO:tasks.workunit.client.0.smithi110.stdout:6/193: getdents d4/d12/d17/d2e 0 2024-09-21T04:15:09.949 INFO:tasks.workunit.client.0.smithi110.stdout:6/194: write d4/fb [1451004,40644] 0 2024-09-21T04:15:09.950 INFO:tasks.workunit.client.0.smithi110.stdout:6/195: write d4/d12/f34 [850615,82989] 0 2024-09-21T04:15:09.950 INFO:tasks.workunit.client.0.smithi110.stdout:6/196: fsync d4/de/f11 0 2024-09-21T04:15:09.950 INFO:tasks.workunit.client.0.smithi110.stdout:6/197: write d4/fb [3533372,48605] 0 2024-09-21T04:15:09.952 INFO:tasks.workunit.client.0.smithi110.stdout:2/102: creat d2/d1a/de/f1b x:0 0 0 2024-09-21T04:15:09.953 INFO:tasks.workunit.client.0.smithi110.stdout:2/103: write d2/d3/f12 [4585515,130878] 0 2024-09-21T04:15:09.953 INFO:tasks.workunit.client.0.smithi110.stdout:2/104: read d2/f8 [3022602,120906] 0 2024-09-21T04:15:09.953 INFO:tasks.workunit.client.0.smithi110.stdout:2/105: creat d2/f1c x:0 0 0 2024-09-21T04:15:09.953 INFO:tasks.workunit.client.0.smithi110.stdout:2/106: readlink d2/d3/d9/l10 0 2024-09-21T04:15:09.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/174: rename d0/d2f/f30 to d0/d9/f39 0 2024-09-21T04:15:09.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/175: dread - d0/d2b/f32 zero size 2024-09-21T04:15:09.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/176: fdatasync d0/d9/d16/f37 0 2024-09-21T04:15:09.957 INFO:tasks.workunit.client.0.smithi110.stdout:2/107: rmdir d2/d1a/de 39 2024-09-21T04:15:09.957 INFO:tasks.workunit.client.1.smithi148.stdout:7/130: write d5/d8/f1b [345313,12743] 0 2024-09-21T04:15:09.957 INFO:tasks.workunit.client.1.smithi148.stdout:7/131: stat d5/f6 0 2024-09-21T04:15:09.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/108: unlink d4/c20 0 2024-09-21T04:15:09.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/109: stat d4/d23 0 2024-09-21T04:15:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:6/62: sync 2024-09-21T04:15:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/177: read d0/ff [805492,48865] 0 2024-09-21T04:15:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:5/74: sync 2024-09-21T04:15:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/178: truncate d0/d9/f39 26184 0 2024-09-21T04:15:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/179: chown d0/l15 431 1 2024-09-21T04:15:09.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/88: getdents d3/d5/d6 0 2024-09-21T04:15:09.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/89: stat d3/l9 0 2024-09-21T04:15:09.961 INFO:tasks.workunit.client.0.smithi110.stdout:4/303: sync 2024-09-21T04:15:09.961 INFO:tasks.workunit.client.0.smithi110.stdout:5/209: sync 2024-09-21T04:15:09.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/108: creat d2/d1a/de/d18/f1d x:0 0 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:8/73: getdents dc 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:6/63: dread d0/d5/dc/fd [0,4194304] 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:1/115: creat db/d24/f25 x:0 0 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:1/116: stat db/l1e 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:1/117: creat db/d24/f26 x:0 0 0 2024-09-21T04:15:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:1/118: dread - db/dd/f23 zero size 2024-09-21T04:15:09.965 INFO:tasks.workunit.client.0.smithi110.stdout:6/198: rename d4/d12/d17/d27 to d4/d12/d17/d2e/d40 0 2024-09-21T04:15:09.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/137: link d5/fc d5/d1e/d37/f3b 0 2024-09-21T04:15:09.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/138: chown d5/d22/d28/c25 995 1 2024-09-21T04:15:09.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/139: write f1 [3916052,34866] 0 2024-09-21T04:15:09.968 INFO:tasks.workunit.client.0.smithi110.stdout:4/304: mknod d17/d3e/d60/c6f 0 2024-09-21T04:15:09.968 INFO:tasks.workunit.client.0.smithi110.stdout:4/305: getdents d17/d1e/d64 0 2024-09-21T04:15:09.970 INFO:tasks.workunit.client.1.smithi148.stdout:7/132: mknod d5/c27 0 2024-09-21T04:15:09.970 INFO:tasks.workunit.client.1.smithi148.stdout:7/133: write d5/dc/f1d [4353046,99641] 0 2024-09-21T04:15:09.971 INFO:tasks.workunit.client.0.smithi110.stdout:5/210: creat d1/d7/d18/f41 x:0 0 0 2024-09-21T04:15:09.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/75: truncate d3/f8 1762540 0 2024-09-21T04:15:09.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/76: rename d3/dc to d3/dc/d16 22 2024-09-21T04:15:09.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/77: creat d3/dc/f17 x:0 0 0 2024-09-21T04:15:09.974 INFO:tasks.workunit.client.0.smithi110.stdout:2/109: mknod d2/d3/c1e 0 2024-09-21T04:15:09.975 INFO:tasks.workunit.client.1.smithi148.stdout:3/90: mkdir d3/d5/d6/d12/d15 0 2024-09-21T04:15:09.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/140: read d5/d1e/d37/f3b [349569,724] 0 2024-09-21T04:15:09.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/141: write d5/d1f/f24 [998881,13810] 0 2024-09-21T04:15:09.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/306: link d17/d34/d3d/d20/d43/d51/l6c d17/d34/d3d/l70 0 2024-09-21T04:15:09.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/307: chown d17/d34/d3d/d20/d43/c68 410 1 2024-09-21T04:15:09.978 INFO:tasks.workunit.client.1.smithi148.stdout:3/91: read f1 [83123,103240] 0 2024-09-21T04:15:09.978 INFO:tasks.workunit.client.1.smithi148.stdout:8/74: creat dc/fe x:0 0 0 2024-09-21T04:15:09.980 INFO:tasks.workunit.client.0.smithi110.stdout:5/211: symlink d1/d39/l42 0 2024-09-21T04:15:09.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/212: dread d1/dc/f1d [0,4194304] 0 2024-09-21T04:15:09.982 INFO:tasks.workunit.client.1.smithi148.stdout:6/64: creat d0/d8/f10 x:0 0 0 2024-09-21T04:15:09.983 INFO:tasks.workunit.client.0.smithi110.stdout:4/308: mknod d17/d34/d3d/d20/c71 0 2024-09-21T04:15:09.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/119: stat db/l18 0 2024-09-21T04:15:09.992 INFO:tasks.workunit.client.0.smithi110.stdout:2/110: read d2/f16 [629342,22195] 0 2024-09-21T04:15:09.995 INFO:tasks.workunit.client.0.smithi110.stdout:5/213: dread d1/d39/f2d [4194304,4194304] 0 2024-09-21T04:15:09.996 INFO:tasks.workunit.client.1.smithi148.stdout:0/129: dwrite f4 [8388608,4194304] 0 2024-09-21T04:15:09.997 INFO:tasks.workunit.client.1.smithi148.stdout:4/110: dwrite d4/f17 [0,4194304] 0 2024-09-21T04:15:09.997 INFO:tasks.workunit.client.1.smithi148.stdout:4/111: write d4/f8 [2949966,81647] 0 2024-09-21T04:15:09.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/112: write d4/d12/f1b [131543,73504] 0 2024-09-21T04:15:09.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/113: read d4/d12/f1b [176038,123484] 0 2024-09-21T04:15:09.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/114: write d4/f6 [2932336,74268] 0 2024-09-21T04:15:09.999 INFO:tasks.workunit.client.1.smithi148.stdout:5/78: mknod d3/d14/c18 0 2024-09-21T04:15:10.000 INFO:tasks.workunit.client.0.smithi110.stdout:4/309: dread d17/f2a [0,4194304] 0 2024-09-21T04:15:10.000 INFO:tasks.workunit.client.0.smithi110.stdout:4/310: chown d17/d1e/d6a 0 1 2024-09-21T04:15:10.001 INFO:tasks.workunit.client.1.smithi148.stdout:9/142: getdents d5/d1f 0 2024-09-21T04:15:10.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/214: read d1/d39/f2d [8079533,46228] 0 2024-09-21T04:15:10.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/75: unlink ca 0 2024-09-21T04:15:10.004 INFO:tasks.workunit.client.0.smithi110.stdout:2/111: rename d2/d3/d9 to d2/d1a/d1f 0 2024-09-21T04:15:10.004 INFO:tasks.workunit.client.0.smithi110.stdout:2/112: chown d2/d1a/d1f/l10 1851442946 1 2024-09-21T04:15:10.005 INFO:tasks.workunit.client.0.smithi110.stdout:2/113: write d2/d1a/de/f1b [157832,12754] 0 2024-09-21T04:15:10.005 INFO:tasks.workunit.client.1.smithi148.stdout:7/134: fsync d5/d8/f1b 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/65: link d0/d5/ca d0/d8/c11 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/92: dwrite f1 [0,4194304] 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/66: write d0/d8/f10 [337785,81077] 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/67: chown d0/d5/dc 2 1 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/68: creat d0/d5/f12 x:0 0 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/69: fsync d0/d5/fb 0 2024-09-21T04:15:10.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/70: chown d0/d5/ff 957776541 1 2024-09-21T04:15:10.008 INFO:tasks.workunit.client.1.smithi148.stdout:6/71: write d0/d5/f12 [44811,28087] 0 2024-09-21T04:15:10.008 INFO:tasks.workunit.client.1.smithi148.stdout:1/120: creat db/f27 x:0 0 0 2024-09-21T04:15:10.008 INFO:tasks.workunit.client.1.smithi148.stdout:7/135: dread d5/d8/f9 [0,4194304] 0 2024-09-21T04:15:10.014 INFO:tasks.workunit.client.1.smithi148.stdout:2/180: dwrite d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:10.014 INFO:tasks.workunit.client.1.smithi148.stdout:2/181: read d0/f1f [108446,85074] 0 2024-09-21T04:15:10.019 INFO:tasks.workunit.client.1.smithi148.stdout:0/130: rename d5/d6/d7/d11/d19/f21 to d5/d6/d7/d18/f2d 0 2024-09-21T04:15:10.019 INFO:tasks.workunit.client.1.smithi148.stdout:0/131: readlink d5/d6/l14 0 2024-09-21T04:15:10.021 INFO:tasks.workunit.client.1.smithi148.stdout:3/93: write f1 [3128331,110156] 0 2024-09-21T04:15:10.022 INFO:tasks.workunit.client.1.smithi148.stdout:4/115: dwrite d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:10.022 INFO:tasks.workunit.client.1.smithi148.stdout:4/116: chown d4/d12/f15 95322 1 2024-09-21T04:15:10.029 INFO:tasks.workunit.client.1.smithi148.stdout:8/76: symlink dc/lf 0 2024-09-21T04:15:10.029 INFO:tasks.workunit.client.1.smithi148.stdout:8/77: creat dc/f10 x:0 0 0 2024-09-21T04:15:10.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/79: dwrite d3/f12 [0,4194304] 0 2024-09-21T04:15:10.037 INFO:tasks.workunit.client.1.smithi148.stdout:1/121: mknod db/dd/c28 0 2024-09-21T04:15:10.038 INFO:tasks.workunit.client.0.smithi110.stdout:5/215: creat d1/d39/de/f43 x:0 0 0 2024-09-21T04:15:10.038 INFO:tasks.workunit.client.0.smithi110.stdout:5/216: chown d1/d39/f3d 1356 1 2024-09-21T04:15:10.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/114: link d2/d3/fd d2/d1a/f20 0 2024-09-21T04:15:10.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/115: stat d2/d1a/d1f 0 2024-09-21T04:15:10.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/132: mknod d5/d6/d7/d11/c2e 0 2024-09-21T04:15:10.044 INFO:tasks.workunit.client.1.smithi148.stdout:3/94: fsync d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:10.044 INFO:tasks.workunit.client.1.smithi148.stdout:3/95: write f1 [2679683,104462] 0 2024-09-21T04:15:10.045 INFO:tasks.workunit.client.1.smithi148.stdout:9/143: dwrite d5/d22/d28/f30 [0,4194304] 0 2024-09-21T04:15:10.045 INFO:tasks.workunit.client.0.smithi110.stdout:5/217: rename d1/d39/de/f43 to d1/d22/f44 0 2024-09-21T04:15:10.047 INFO:tasks.workunit.client.0.smithi110.stdout:5/218: creat d1/d39/f45 x:0 0 0 2024-09-21T04:15:10.051 INFO:tasks.workunit.client.0.smithi110.stdout:5/219: link d1/f38 d1/d39/de/f46 0 2024-09-21T04:15:10.052 INFO:tasks.workunit.client.1.smithi148.stdout:8/78: mknod dc/c11 0 2024-09-21T04:15:10.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/72: rmdir d0/d5/dc 39 2024-09-21T04:15:10.059 INFO:tasks.workunit.client.1.smithi148.stdout:1/122: mknod db/dd/c29 0 2024-09-21T04:15:10.062 INFO:tasks.workunit.client.1.smithi148.stdout:3/96: mknod d3/d13/c16 0 2024-09-21T04:15:10.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/79: symlink dc/l12 0 2024-09-21T04:15:10.065 INFO:tasks.workunit.client.1.smithi148.stdout:6/73: mknod d0/d5/c13 0 2024-09-21T04:15:10.067 INFO:tasks.workunit.client.1.smithi148.stdout:1/123: mkdir db/d24/d2a 0 2024-09-21T04:15:10.067 INFO:tasks.workunit.client.1.smithi148.stdout:1/124: chown c2 1563 1 2024-09-21T04:15:10.067 INFO:tasks.workunit.client.1.smithi148.stdout:1/125: creat db/d24/f2b x:0 0 0 2024-09-21T04:15:10.069 INFO:tasks.workunit.client.1.smithi148.stdout:3/97: rmdir d3/d5/d6/d12 39 2024-09-21T04:15:10.070 INFO:tasks.workunit.client.1.smithi148.stdout:5/80: dread d3/fb [0,4194304] 0 2024-09-21T04:15:10.072 INFO:tasks.workunit.client.0.smithi110.stdout:2/116: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:15:10.073 INFO:tasks.workunit.client.1.smithi148.stdout:0/133: dwrite d5/d6/d7/d11/f17 [0,4194304] 0 2024-09-21T04:15:10.077 INFO:tasks.workunit.client.1.smithi148.stdout:7/136: dwrite f2 [0,4194304] 0 2024-09-21T04:15:10.078 INFO:tasks.workunit.client.1.smithi148.stdout:8/80: mkdir dc/d13 0 2024-09-21T04:15:10.081 INFO:tasks.workunit.client.0.smithi110.stdout:5/220: dwrite d1/d39/f2d [0,4194304] 0 2024-09-21T04:15:10.081 INFO:tasks.workunit.client.0.smithi110.stdout:5/221: dread - d1/f3e zero size 2024-09-21T04:15:10.082 INFO:tasks.workunit.client.0.smithi110.stdout:4/311: dwrite d17/d1e/f48 [0,4194304] 0 2024-09-21T04:15:10.082 INFO:tasks.workunit.client.0.smithi110.stdout:4/312: write d17/d3e/d6b/f6e [597150,28870] 0 2024-09-21T04:15:10.082 INFO:tasks.workunit.client.0.smithi110.stdout:4/313: dread - d17/d3e/d60/f5e zero size 2024-09-21T04:15:10.082 INFO:tasks.workunit.client.0.smithi110.stdout:4/314: write d17/d1e/d21/f2e [4859580,38916] 0 2024-09-21T04:15:10.083 INFO:tasks.workunit.client.1.smithi148.stdout:8/81: dread f6 [0,4194304] 0 2024-09-21T04:15:10.083 INFO:tasks.workunit.client.0.smithi110.stdout:5/222: mknod d1/dc/c47 0 2024-09-21T04:15:10.084 INFO:tasks.workunit.client.1.smithi148.stdout:2/182: dwrite d0/f23 [0,4194304] 0 2024-09-21T04:15:10.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/98: creat d3/d5/d6/f17 x:0 0 0 2024-09-21T04:15:10.087 INFO:tasks.workunit.client.0.smithi110.stdout:4/315: dread d17/d3e/f40 [0,4194304] 0 2024-09-21T04:15:10.088 INFO:tasks.workunit.client.1.smithi148.stdout:5/81: link d3/dc/f17 d3/dc/f19 0 2024-09-21T04:15:10.090 INFO:tasks.workunit.client.1.smithi148.stdout:0/134: creat d5/d6/d7/f2f x:0 0 0 2024-09-21T04:15:10.091 INFO:tasks.workunit.client.0.smithi110.stdout:4/316: truncate f5 2561794 0 2024-09-21T04:15:10.091 INFO:tasks.workunit.client.0.smithi110.stdout:4/317: read - d17/d34/d3d/d57/f5a zero size 2024-09-21T04:15:10.091 INFO:tasks.workunit.client.0.smithi110.stdout:2/117: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:15:10.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/318: symlink d17/d1e/l72 0 2024-09-21T04:15:10.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/319: chown d17/f27 2 1 2024-09-21T04:15:10.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/118: stat d2/d1a/d1f/l10 0 2024-09-21T04:15:10.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/119: fsync d2/d3/f12 0 2024-09-21T04:15:10.097 INFO:tasks.workunit.client.0.smithi110.stdout:4/320: symlink d17/d34/d3d/d20/l73 0 2024-09-21T04:15:10.102 INFO:tasks.workunit.client.1.smithi148.stdout:2/183: getdents d0 0 2024-09-21T04:15:10.102 INFO:tasks.workunit.client.0.smithi110.stdout:4/321: read d17/f2a [3701229,42003] 0 2024-09-21T04:15:10.105 INFO:tasks.workunit.client.1.smithi148.stdout:8/82: write f5 [1646561,80271] 0 2024-09-21T04:15:10.105 INFO:tasks.workunit.client.1.smithi148.stdout:8/83: creat dc/f14 x:0 0 0 2024-09-21T04:15:10.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/99: mknod d3/d5/c18 0 2024-09-21T04:15:10.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/100: truncate d3/d5/d6/d12/da/f11 3449640 0 2024-09-21T04:15:10.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/101: dread - d3/d5/d6/f17 zero size 2024-09-21T04:15:10.108 INFO:tasks.workunit.client.1.smithi148.stdout:5/82: mkdir d3/d14/d1a 0 2024-09-21T04:15:10.109 INFO:tasks.workunit.client.1.smithi148.stdout:0/135: mknod d5/d6/d7/d18/c30 0 2024-09-21T04:15:10.110 INFO:tasks.workunit.client.1.smithi148.stdout:2/184: creat d0/d9/dc/d13/d2c/f3a x:0 0 0 2024-09-21T04:15:10.111 INFO:tasks.workunit.client.1.smithi148.stdout:2/185: creat d0/d9/dc/d13/f3b x:0 0 0 2024-09-21T04:15:10.113 INFO:tasks.workunit.client.1.smithi148.stdout:8/84: symlink dc/l15 0 2024-09-21T04:15:10.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/136: dread d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:10.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/137: dread - d5/d6/f16 zero size 2024-09-21T04:15:10.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/138: creat d5/d6/d7/f31 x:0 0 0 2024-09-21T04:15:10.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/102: mkdir d3/d5/d6/d12/da/d19 0 2024-09-21T04:15:10.114 INFO:tasks.workunit.client.1.smithi148.stdout:5/83: creat d3/f1b x:0 0 0 2024-09-21T04:15:10.114 INFO:tasks.workunit.client.1.smithi148.stdout:5/84: write d3/dc/f19 [360980,17605] 0 2024-09-21T04:15:10.115 INFO:tasks.workunit.client.1.smithi148.stdout:2/186: stat d0/d9/d16/f37 0 2024-09-21T04:15:10.117 INFO:tasks.workunit.client.1.smithi148.stdout:8/85: rename dc/lf to dc/l16 0 2024-09-21T04:15:10.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/139: creat d5/d6/d7/d18/f32 x:0 0 0 2024-09-21T04:15:10.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/140: fdatasync d5/d6/d7/d11/f1d 0 2024-09-21T04:15:10.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/103: link d3/d5/d6/f17 d3/d5/d6/d12/f1a 0 2024-09-21T04:15:10.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/104: fsync d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:10.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/105: fdatasync f1 0 2024-09-21T04:15:10.122 INFO:tasks.workunit.client.1.smithi148.stdout:8/86: link f7 dc/f17 0 2024-09-21T04:15:10.122 INFO:tasks.workunit.client.1.smithi148.stdout:8/87: dread - f2 zero size 2024-09-21T04:15:10.122 INFO:tasks.workunit.client.1.smithi148.stdout:8/88: readlink dc/l15 0 2024-09-21T04:15:10.122 INFO:tasks.workunit.client.1.smithi148.stdout:8/89: creat dc/f18 x:0 0 0 2024-09-21T04:15:10.122 INFO:tasks.workunit.client.1.smithi148.stdout:8/90: chown dc/f18 30096512 1 2024-09-21T04:15:10.123 INFO:tasks.workunit.client.1.smithi148.stdout:0/141: dread d5/d6/f1f [0,4194304] 0 2024-09-21T04:15:10.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/106: symlink d3/d5/d6/d12/da/l1b 0 2024-09-21T04:15:10.125 INFO:tasks.workunit.client.1.smithi148.stdout:9/144: truncate d5/d22/d28/f30 3938673 0 2024-09-21T04:15:10.126 INFO:tasks.workunit.client.1.smithi148.stdout:2/187: write d0/f1d [642656,88755] 0 2024-09-21T04:15:10.126 INFO:tasks.workunit.client.1.smithi148.stdout:2/188: chown d0/d9/dc/f24 60 1 2024-09-21T04:15:10.126 INFO:tasks.workunit.client.1.smithi148.stdout:2/189: readlink d0/d9/dc/l31 0 2024-09-21T04:15:10.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/223: rename d1/f6 to d1/d39/de/f48 0 2024-09-21T04:15:10.138 INFO:tasks.workunit.client.0.smithi110.stdout:4/322: rename f8 to d17/d34/f74 0 2024-09-21T04:15:10.139 INFO:tasks.workunit.client.0.smithi110.stdout:5/224: dread d1/dc/f1a [0,4194304] 0 2024-09-21T04:15:10.140 INFO:tasks.workunit.client.0.smithi110.stdout:5/225: mkdir d1/d49 0 2024-09-21T04:15:10.141 INFO:tasks.workunit.client.0.smithi110.stdout:5/226: mkdir d1/d39/de/d1e/d4a 0 2024-09-21T04:15:10.142 INFO:tasks.workunit.client.0.smithi110.stdout:5/227: write d1/d22/f27 [375503,80030] 0 2024-09-21T04:15:10.143 INFO:tasks.workunit.client.0.smithi110.stdout:5/228: symlink d1/d39/l4b 0 2024-09-21T04:15:10.143 INFO:tasks.workunit.client.1.smithi148.stdout:9/145: dread d5/f23 [0,4194304] 0 2024-09-21T04:15:10.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/107: dwrite d3/d5/d6/d12/f1a [0,4194304] 0 2024-09-21T04:15:10.144 INFO:tasks.workunit.client.0.smithi110.stdout:5/229: creat d1/f4c x:0 0 0 2024-09-21T04:15:10.147 INFO:tasks.workunit.client.0.smithi110.stdout:2/120: truncate d2/f16 1468647 0 2024-09-21T04:15:10.147 INFO:tasks.workunit.client.0.smithi110.stdout:2/121: write d2/d3/fd [480152,83341] 0 2024-09-21T04:15:10.147 INFO:tasks.workunit.client.0.smithi110.stdout:2/122: readlink d2/d1a/de/l11 0 2024-09-21T04:15:10.148 INFO:tasks.workunit.client.1.smithi148.stdout:7/137: truncate f2 2128013 0 2024-09-21T04:15:10.155 INFO:tasks.workunit.client.1.smithi148.stdout:5/85: rmdir d3 39 2024-09-21T04:15:10.156 INFO:tasks.workunit.client.0.smithi110.stdout:4/323: write d17/d1e/f4c [3729257,30163] 0 2024-09-21T04:15:10.158 INFO:tasks.workunit.client.1.smithi148.stdout:5/86: creat d3/d14/d1a/f1c x:0 0 0 2024-09-21T04:15:10.158 INFO:tasks.workunit.client.1.smithi148.stdout:5/87: write d3/dc/f19 [1346976,111244] 0 2024-09-21T04:15:10.158 INFO:tasks.workunit.client.1.smithi148.stdout:3/108: dread d3/d5/d6/d12/f1a [0,4194304] 0 2024-09-21T04:15:10.158 INFO:tasks.workunit.client.1.smithi148.stdout:3/109: chown d3/d5/lb 3895 1 2024-09-21T04:15:10.158 INFO:tasks.workunit.client.0.smithi110.stdout:4/324: mknod d17/d34/d3d/d20/d43/d51/d52/c75 0 2024-09-21T04:15:10.161 INFO:tasks.workunit.client.1.smithi148.stdout:7/138: read d5/d8/f1e [2179816,54896] 0 2024-09-21T04:15:10.161 INFO:tasks.workunit.client.1.smithi148.stdout:3/110: fsync d3/d5/d6/f17 0 2024-09-21T04:15:10.162 INFO:tasks.workunit.client.1.smithi148.stdout:1/126: rename db/l1e to db/dd/l2c 0 2024-09-21T04:15:10.163 INFO:tasks.workunit.client.1.smithi148.stdout:2/190: dwrite d0/d9/dc/d13/d25/d2d/f2e [0,4194304] 0 2024-09-21T04:15:10.164 INFO:tasks.workunit.client.1.smithi148.stdout:8/91: rename dc/f10 to dc/d13/f19 0 2024-09-21T04:15:10.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/88: read d3/f12 [3869763,64764] 0 2024-09-21T04:15:10.166 INFO:tasks.workunit.client.1.smithi148.stdout:1/127: symlink db/d24/d2a/l2d 0 2024-09-21T04:15:10.169 INFO:tasks.workunit.client.1.smithi148.stdout:2/191: symlink d0/d9/dc/d13/d2c/l3c 0 2024-09-21T04:15:10.169 INFO:tasks.workunit.client.1.smithi148.stdout:2/192: fdatasync d0/d9/dc/d13/f3b 0 2024-09-21T04:15:10.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/89: mknod d3/d14/d1a/c1d 0 2024-09-21T04:15:10.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/90: read d3/f9 [447633,46544] 0 2024-09-21T04:15:10.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/91: chown d3 113 1 2024-09-21T04:15:10.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/92: chown d3/f9 49038 1 2024-09-21T04:15:10.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/93: fdatasync d3/f1b 0 2024-09-21T04:15:10.172 INFO:tasks.workunit.client.1.smithi148.stdout:1/128: unlink db/dd/c28 0 2024-09-21T04:15:10.173 INFO:tasks.workunit.client.0.smithi110.stdout:2/123: dwrite d2/d3/f12 [4194304,4194304] 0 2024-09-21T04:15:10.173 INFO:tasks.workunit.client.0.smithi110.stdout:2/124: truncate d2/d3/fd 1028535 0 2024-09-21T04:15:10.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/193: mknod d0/d9/d16/c3d 0 2024-09-21T04:15:10.175 INFO:tasks.workunit.client.1.smithi148.stdout:7/139: write d5/d8/f1a [1128839,44408] 0 2024-09-21T04:15:10.176 INFO:tasks.workunit.client.1.smithi148.stdout:5/94: unlink d3/f11 0 2024-09-21T04:15:10.176 INFO:tasks.workunit.client.1.smithi148.stdout:5/95: truncate d3/f1b 1038001 0 2024-09-21T04:15:10.176 INFO:tasks.workunit.client.0.smithi110.stdout:2/125: getdents d2/d1a/de/d18 0 2024-09-21T04:15:10.176 INFO:tasks.workunit.client.0.smithi110.stdout:2/126: chown d2/d1a/d1f 1383 1 2024-09-21T04:15:10.176 INFO:tasks.workunit.client.0.smithi110.stdout:2/127: chown d2/d3/c1e 1 1 2024-09-21T04:15:10.177 INFO:tasks.workunit.client.0.smithi110.stdout:2/128: write d2/d1a/f20 [406195,45439] 0 2024-09-21T04:15:10.177 INFO:tasks.workunit.client.1.smithi148.stdout:1/129: mkdir db/d24/d2e 0 2024-09-21T04:15:10.177 INFO:tasks.workunit.client.1.smithi148.stdout:1/130: fsync f6 0 2024-09-21T04:15:10.178 INFO:tasks.workunit.client.0.smithi110.stdout:5/230: rename d1/d39/de to d1/d49/d4d 0 2024-09-21T04:15:10.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/146: dwrite d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:10.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/147: fsync d5/d1f/f24 0 2024-09-21T04:15:10.180 INFO:tasks.workunit.client.1.smithi148.stdout:2/194: symlink d0/d2f/l3e 0 2024-09-21T04:15:10.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/92: dwrite f6 [0,4194304] 0 2024-09-21T04:15:10.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/93: chown dc/fe 0 1 2024-09-21T04:15:10.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/94: fsync dc/f18 0 2024-09-21T04:15:10.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/95: creat dc/f1a x:0 0 0 2024-09-21T04:15:10.183 INFO:tasks.workunit.client.0.smithi110.stdout:4/325: rename d17/d34/d3d/l70 to d17/d1e/d21/l76 0 2024-09-21T04:15:10.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/231: creat d1/d7/f4e x:0 0 0 2024-09-21T04:15:10.186 INFO:tasks.workunit.client.0.smithi110.stdout:5/232: write d1/f4c [290440,57447] 0 2024-09-21T04:15:10.186 INFO:tasks.workunit.client.0.smithi110.stdout:5/233: creat d1/d22/f4f x:0 0 0 2024-09-21T04:15:10.186 INFO:tasks.workunit.client.0.smithi110.stdout:5/234: truncate d1/dc/f1d 1873999 0 2024-09-21T04:15:10.186 INFO:tasks.workunit.client.0.smithi110.stdout:5/235: chown d1/d39/f45 373 1 2024-09-21T04:15:10.188 INFO:tasks.workunit.client.1.smithi148.stdout:2/195: mknod d0/d9/dc/d13/d25/d2d/c3f 0 2024-09-21T04:15:10.189 INFO:tasks.workunit.client.0.smithi110.stdout:5/236: link d1/d39/f2d d1/d39/f50 0 2024-09-21T04:15:10.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/140: dwrite f4 [4194304,4194304] 0 2024-09-21T04:15:10.194 INFO:tasks.workunit.client.1.smithi148.stdout:7/141: mkdir d5/dc/d28 0 2024-09-21T04:15:10.194 INFO:tasks.workunit.client.1.smithi148.stdout:7/142: fsync d5/d8/f1a 0 2024-09-21T04:15:10.196 INFO:tasks.workunit.client.1.smithi148.stdout:2/196: mkdir d0/d9/d40 0 2024-09-21T04:15:10.198 INFO:tasks.workunit.client.1.smithi148.stdout:2/197: link d0/d9/dc/d13/d25/d2d/f2e d0/d9/d38/f41 0 2024-09-21T04:15:10.199 INFO:tasks.workunit.client.1.smithi148.stdout:2/198: dread - d0/d9/dc/d13/d2c/f33 zero size 2024-09-21T04:15:10.202 INFO:tasks.workunit.client.0.smithi110.stdout:4/326: write d17/d34/f65 [1968669,113057] 0 2024-09-21T04:15:10.202 INFO:tasks.workunit.client.0.smithi110.stdout:4/327: chown d17/f1a 0 1 2024-09-21T04:15:10.206 INFO:tasks.workunit.client.1.smithi148.stdout:8/96: dread f9 [0,4194304] 0 2024-09-21T04:15:10.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/96: dread d3/fa [0,4194304] 0 2024-09-21T04:15:10.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/143: dread d5/d8/ff [0,4194304] 0 2024-09-21T04:15:10.208 INFO:tasks.workunit.client.0.smithi110.stdout:5/237: dwrite d1/f35 [0,4194304] 0 2024-09-21T04:15:10.208 INFO:tasks.workunit.client.0.smithi110.stdout:5/238: dread - d1/d22/f4f zero size 2024-09-21T04:15:10.209 INFO:tasks.workunit.client.1.smithi148.stdout:8/97: mknod dc/c1b 0 2024-09-21T04:15:10.209 INFO:tasks.workunit.client.1.smithi148.stdout:8/98: chown fb 0 1 2024-09-21T04:15:10.210 INFO:tasks.workunit.client.1.smithi148.stdout:5/97: mkdir d3/dc/d1e 0 2024-09-21T04:15:10.211 INFO:tasks.workunit.client.1.smithi148.stdout:7/144: creat d5/dc/d28/f29 x:0 0 0 2024-09-21T04:15:10.213 INFO:tasks.workunit.client.1.smithi148.stdout:8/99: creat dc/d13/f1c x:0 0 0 2024-09-21T04:15:10.213 INFO:tasks.workunit.client.1.smithi148.stdout:8/100: dread - dc/d13/f19 zero size 2024-09-21T04:15:10.213 INFO:tasks.workunit.client.1.smithi148.stdout:8/101: creat dc/f1d x:0 0 0 2024-09-21T04:15:10.213 INFO:tasks.workunit.client.0.smithi110.stdout:5/239: mkdir d1/d22/d51 0 2024-09-21T04:15:10.215 INFO:tasks.workunit.client.1.smithi148.stdout:4/117: sync 2024-09-21T04:15:10.215 INFO:tasks.workunit.client.1.smithi148.stdout:5/98: unlink d3/d14/d1a/c1d 0 2024-09-21T04:15:10.215 INFO:tasks.workunit.client.1.smithi148.stdout:5/99: chown d3/dc 5 1 2024-09-21T04:15:10.217 INFO:tasks.workunit.client.1.smithi148.stdout:4/118: symlink d4/d12/d22/l24 0 2024-09-21T04:15:10.223 INFO:tasks.workunit.client.1.smithi148.stdout:4/119: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:10.223 INFO:tasks.workunit.client.1.smithi148.stdout:4/120: fsync d4/f9 0 2024-09-21T04:15:10.223 INFO:tasks.workunit.client.1.smithi148.stdout:4/121: write d4/f6 [4045420,95760] 0 2024-09-21T04:15:10.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/199: write d0/d9/dc/fd [2116369,124710] 0 2024-09-21T04:15:10.227 INFO:tasks.workunit.client.1.smithi148.stdout:5/100: write f0 [3987324,4135] 0 2024-09-21T04:15:10.227 INFO:tasks.workunit.client.1.smithi148.stdout:5/101: write f0 [3739898,109768] 0 2024-09-21T04:15:10.227 INFO:tasks.workunit.client.0.smithi110.stdout:5/240: dread d1/d7/d18/f28 [0,4194304] 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.0.smithi110.stdout:5/241: readlink d1/d49/d4d/l1f 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/129: dwrite d2/d3/fd [0,4194304] 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/130: stat d2/d3 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.1.smithi148.stdout:2/200: truncate d0/d9/f17 3885291 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.1.smithi148.stdout:2/201: fdatasync d0/d9/dc/d13/d25/d2d/f2e 0 2024-09-21T04:15:10.228 INFO:tasks.workunit.client.1.smithi148.stdout:2/202: chown d0/d9/dc/d13/d2c/l3c 607860 1 2024-09-21T04:15:10.230 INFO:tasks.workunit.client.0.smithi110.stdout:5/242: unlink d1/d22/f32 0 2024-09-21T04:15:10.230 INFO:tasks.workunit.client.0.smithi110.stdout:5/243: dread - d1/d7/d18/f41 zero size 2024-09-21T04:15:10.233 INFO:tasks.workunit.client.1.smithi148.stdout:5/102: rename d3/dc/d1e to d3/d14/d1f 0 2024-09-21T04:15:10.233 INFO:tasks.workunit.client.1.smithi148.stdout:5/103: write d3/d14/d1a/f1c [830382,96446] 0 2024-09-21T04:15:10.234 INFO:tasks.workunit.client.0.smithi110.stdout:2/131: creat d2/d1a/de/f21 x:0 0 0 2024-09-21T04:15:10.235 INFO:tasks.workunit.client.0.smithi110.stdout:2/132: creat d2/d1a/de/f22 x:0 0 0 2024-09-21T04:15:10.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/203: mknod d0/d9/d16/c42 0 2024-09-21T04:15:10.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/204: truncate d0/d2b/f32 165870 0 2024-09-21T04:15:10.237 INFO:tasks.workunit.client.0.smithi110.stdout:5/244: symlink d1/d49/l52 0 2024-09-21T04:15:10.244 INFO:tasks.workunit.client.1.smithi148.stdout:2/205: dread d0/f23 [0,4194304] 0 2024-09-21T04:15:10.244 INFO:tasks.workunit.client.1.smithi148.stdout:2/206: stat d0/c20 0 2024-09-21T04:15:10.245 INFO:tasks.workunit.client.1.smithi148.stdout:2/207: link d0/d9/dc/d13/d2c/f3a d0/d9/d38/f43 0 2024-09-21T04:15:10.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/145: dwrite d5/d8/f1a [0,4194304] 0 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.0.smithi110.stdout:4/328: dwrite d17/f2a [0,4194304] 0 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.0.smithi110.stdout:4/329: fsync d17/d34/d3d/d20/f26 0 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/102: dwrite fb [0,4194304] 0 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/103: fdatasync dc/f18 0 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/104: chown dc/d13/f1c 7 1 2024-09-21T04:15:10.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/104: dwrite d3/f1b [0,4194304] 0 2024-09-21T04:15:10.254 INFO:tasks.workunit.client.1.smithi148.stdout:7/146: mknod d5/c2a 0 2024-09-21T04:15:10.255 INFO:tasks.workunit.client.1.smithi148.stdout:7/147: read d5/d8/ff [3850,103620] 0 2024-09-21T04:15:10.261 INFO:tasks.workunit.client.1.smithi148.stdout:4/122: truncate d4/f6 3760500 0 2024-09-21T04:15:10.263 INFO:tasks.workunit.client.1.smithi148.stdout:5/105: mkdir d3/d14/d1f/d20 0 2024-09-21T04:15:10.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/148: rename d5/dc/f1d to d5/dc/d24/f2b 0 2024-09-21T04:15:10.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/149: write d5/dc/f19 [2745220,8246] 0 2024-09-21T04:15:10.266 INFO:tasks.workunit.client.0.smithi110.stdout:4/330: dwrite d17/d34/d3d/d20/f29 [4194304,4194304] 0 2024-09-21T04:15:10.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/123: symlink d4/l25 0 2024-09-21T04:15:10.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/124: chown d4/d12/l1a 1894815856 1 2024-09-21T04:15:10.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/125: write d4/d12/d16/f19 [1044278,128611] 0 2024-09-21T04:15:10.268 INFO:tasks.workunit.client.1.smithi148.stdout:5/106: symlink d3/d14/d1f/d20/l21 0 2024-09-21T04:15:10.269 INFO:tasks.workunit.client.0.smithi110.stdout:4/331: mkdir d17/d1e/d77 0 2024-09-21T04:15:10.270 INFO:tasks.workunit.client.0.smithi110.stdout:9/100: sync 2024-09-21T04:15:10.270 INFO:tasks.workunit.client.0.smithi110.stdout:1/452: sync 2024-09-21T04:15:10.270 INFO:tasks.workunit.client.0.smithi110.stdout:3/211: sync 2024-09-21T04:15:10.270 INFO:tasks.workunit.client.0.smithi110.stdout:7/196: sync 2024-09-21T04:15:10.270 INFO:tasks.workunit.client.0.smithi110.stdout:3/212: readlink d0/d22/l2e 0 2024-09-21T04:15:10.271 INFO:tasks.workunit.client.0.smithi110.stdout:3/213: write d0/d12/f23 [561250,82995] 0 2024-09-21T04:15:10.271 INFO:tasks.workunit.client.0.smithi110.stdout:1/453: creat d2/d9/d23/d3b/d5e/d86/f95 x:0 0 0 2024-09-21T04:15:10.271 INFO:tasks.workunit.client.0.smithi110.stdout:1/454: fsync d2/d9/d49/d4f/d87/f8d 0 2024-09-21T04:15:10.271 INFO:tasks.workunit.client.1.smithi148.stdout:7/150: mknod d5/dc/d28/c2c 0 2024-09-21T04:15:10.271 INFO:tasks.workunit.client.1.smithi148.stdout:7/151: dread - d5/dc/f10 zero size 2024-09-21T04:15:10.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/208: getdents d0/d9/d38 0 2024-09-21T04:15:10.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/209: dread - d0/d9/dc/d13/d2c/f3a zero size 2024-09-21T04:15:10.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/455: dread d2/d9/f1a [0,4194304] 0 2024-09-21T04:15:10.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/456: chown d2/lb 2 1 2024-09-21T04:15:10.272 INFO:tasks.workunit.client.0.smithi110.stdout:4/332: rename d17/d34/d3d/d20/l4b to d17/d34/l78 0 2024-09-21T04:15:10.273 INFO:tasks.workunit.client.1.smithi148.stdout:4/126: rename d4/d12/d22 to d4/d12/d16/d26 0 2024-09-21T04:15:10.274 INFO:tasks.workunit.client.0.smithi110.stdout:9/101: symlink d7/db/l1e 0 2024-09-21T04:15:10.275 INFO:tasks.workunit.client.0.smithi110.stdout:1/457: dread d2/f45 [0,4194304] 0 2024-09-21T04:15:10.275 INFO:tasks.workunit.client.0.smithi110.stdout:4/333: dread d17/d3e/d6b/f6e [0,4194304] 0 2024-09-21T04:15:10.275 INFO:tasks.workunit.client.0.smithi110.stdout:4/334: creat d17/d1e/f79 x:0 0 0 2024-09-21T04:15:10.276 INFO:tasks.workunit.client.0.smithi110.stdout:8/230: sync 2024-09-21T04:15:10.278 INFO:tasks.workunit.client.1.smithi148.stdout:7/152: write d5/f13 [1953719,24168] 0 2024-09-21T04:15:10.281 INFO:tasks.workunit.client.0.smithi110.stdout:7/197: rename dd/d14/f28 to dd/d1c/f38 0 2024-09-21T04:15:10.281 INFO:tasks.workunit.client.0.smithi110.stdout:7/198: readlink dd/d14/d17/l29 0 2024-09-21T04:15:10.282 INFO:tasks.workunit.client.0.smithi110.stdout:9/102: mkdir d7/db/d1f 0 2024-09-21T04:15:10.282 INFO:tasks.workunit.client.0.smithi110.stdout:9/103: creat d7/db/f20 x:0 0 0 2024-09-21T04:15:10.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/335: dread d17/d1e/d21/f5b [0,4194304] 0 2024-09-21T04:15:10.283 INFO:tasks.workunit.client.1.smithi148.stdout:5/107: rename l1 to d3/dc/l22 0 2024-09-21T04:15:10.283 INFO:tasks.workunit.client.1.smithi148.stdout:5/108: readlink d3/lf 0 2024-09-21T04:15:10.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/458: creat d2/d9/d49/d5c/d91/f96 x:0 0 0 2024-09-21T04:15:10.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/459: write d2/f4 [400228,63764] 0 2024-09-21T04:15:10.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/460: dread - d2/d9/d49/f77 zero size 2024-09-21T04:15:10.285 INFO:tasks.workunit.client.1.smithi148.stdout:4/127: unlink d4/ld 0 2024-09-21T04:15:10.285 INFO:tasks.workunit.client.1.smithi148.stdout:4/128: chown d4/f17 6896952 1 2024-09-21T04:15:10.285 INFO:tasks.workunit.client.1.smithi148.stdout:4/129: dread - d4/fe zero size 2024-09-21T04:15:10.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/210: getdents d0/d9/dc/d13/d2c 0 2024-09-21T04:15:10.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/231: getdents d0/d30/d34/d3e/d32 0 2024-09-21T04:15:10.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/153: mknod d5/dc/d24/c2d 0 2024-09-21T04:15:10.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/154: chown d5/d14 11240220 1 2024-09-21T04:15:10.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/214: getdents d0/d12/d16/d1a 0 2024-09-21T04:15:10.291 INFO:tasks.workunit.client.1.smithi148.stdout:5/109: creat d3/d14/d1f/f23 x:0 0 0 2024-09-21T04:15:10.292 INFO:tasks.workunit.client.0.smithi110.stdout:7/199: mkdir dd/d1c/d39 0 2024-09-21T04:15:10.293 INFO:tasks.workunit.client.1.smithi148.stdout:4/130: truncate d4/fb 990939 0 2024-09-21T04:15:10.294 INFO:tasks.workunit.client.1.smithi148.stdout:2/211: chown d0/d9/fe 449145995 1 2024-09-21T04:15:10.294 INFO:tasks.workunit.client.1.smithi148.stdout:6/74: sync 2024-09-21T04:15:10.296 INFO:tasks.workunit.client.0.smithi110.stdout:9/104: rmdir d7/d18 0 2024-09-21T04:15:10.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/232: write d0/d7/d13/d4b/f1f [1031643,60722] 0 2024-09-21T04:15:10.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/233: creat d0/d7/d13/d46/f51 x:0 0 0 2024-09-21T04:15:10.301 INFO:tasks.workunit.client.0.smithi110.stdout:4/336: mkdir d17/d34/d3d/d20/d43/d51/d7a 0 2024-09-21T04:15:10.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/131: mknod d4/d12/c27 0 2024-09-21T04:15:10.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/132: fdatasync d4/f8 0 2024-09-21T04:15:10.305 INFO:tasks.workunit.client.0.smithi110.stdout:7/200: creat dd/d14/d17/f3a x:0 0 0 2024-09-21T04:15:10.305 INFO:tasks.workunit.client.0.smithi110.stdout:7/201: truncate dd/f1a 874634 0 2024-09-21T04:15:10.305 INFO:tasks.workunit.client.0.smithi110.stdout:3/215: dread d0/d12/d16/d1a/f39 [0,4194304] 0 2024-09-21T04:15:10.305 INFO:tasks.workunit.client.0.smithi110.stdout:3/216: truncate d0/d22/f46 37607 0 2024-09-21T04:15:10.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/155: dwrite d5/d8/f1a [0,4194304] 0 2024-09-21T04:15:10.307 INFO:tasks.workunit.client.0.smithi110.stdout:9/105: symlink d7/db/d14/l21 0 2024-09-21T04:15:10.307 INFO:tasks.workunit.client.0.smithi110.stdout:9/106: truncate d7/db/f20 344879 0 2024-09-21T04:15:10.309 INFO:tasks.workunit.client.1.smithi148.stdout:5/110: mkdir d3/d24 0 2024-09-21T04:15:10.309 INFO:tasks.workunit.client.1.smithi148.stdout:5/111: rename d3 to d3/d14/d25 22 2024-09-21T04:15:10.313 INFO:tasks.workunit.client.1.smithi148.stdout:4/133: truncate d4/f8 409724 0 2024-09-21T04:15:10.314 INFO:tasks.workunit.client.1.smithi148.stdout:4/134: fdatasync d4/f8 0 2024-09-21T04:15:10.314 INFO:tasks.workunit.client.1.smithi148.stdout:4/135: write d4/f9 [318405,26118] 0 2024-09-21T04:15:10.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/234: mkdir d0/d7/d23/d52 0 2024-09-21T04:15:10.316 INFO:tasks.workunit.client.1.smithi148.stdout:6/75: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:15:10.317 INFO:tasks.workunit.client.1.smithi148.stdout:6/76: chown d0/d5/fb 1032582941 1 2024-09-21T04:15:10.317 INFO:tasks.workunit.client.1.smithi148.stdout:5/112: mkdir d3/d14/d1f/d26 0 2024-09-21T04:15:10.317 INFO:tasks.workunit.client.1.smithi148.stdout:5/113: creat d3/d14/d1a/f27 x:0 0 0 2024-09-21T04:15:10.319 INFO:tasks.workunit.client.0.smithi110.stdout:7/202: truncate dd/d14/f22 93363 0 2024-09-21T04:15:10.321 INFO:tasks.workunit.client.1.smithi148.stdout:6/77: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:15:10.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/136: write d4/d12/f1b [909123,19623] 0 2024-09-21T04:15:10.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/137: chown d4/f10 106164 1 2024-09-21T04:15:10.323 INFO:tasks.workunit.client.0.smithi110.stdout:3/217: rmdir d0/d3a 0 2024-09-21T04:15:10.324 INFO:tasks.workunit.client.1.smithi148.stdout:7/156: dwrite d5/dc/d28/f29 [0,4194304] 0 2024-09-21T04:15:10.325 INFO:tasks.workunit.client.0.smithi110.stdout:9/107: symlink d7/db/d14/l22 0 2024-09-21T04:15:10.326 INFO:tasks.workunit.client.1.smithi148.stdout:6/78: dread d0/d5/dc/fd [0,4194304] 0 2024-09-21T04:15:10.326 INFO:tasks.workunit.client.1.smithi148.stdout:6/79: dread - d0/d5/ff zero size 2024-09-21T04:15:10.327 INFO:tasks.workunit.client.0.smithi110.stdout:8/235: symlink d0/d7/d13/d1c/l53 0 2024-09-21T04:15:10.327 INFO:tasks.workunit.client.0.smithi110.stdout:8/236: dread - d0/d7/d13/d46/f4c zero size 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:7/157: dread d5/d8/f1e [0,4194304] 0 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:7/158: stat d5/d8/c21 0 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:5/114: creat d3/d14/d1f/f28 x:0 0 0 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/80: dread d0/d5/f12 [0,4194304] 0 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/81: chown d0/d8 4268317 1 2024-09-21T04:15:10.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/82: fsync d0/d5/ff 0 2024-09-21T04:15:10.332 INFO:tasks.workunit.client.0.smithi110.stdout:3/218: creat d0/d12/d3c/f48 x:0 0 0 2024-09-21T04:15:10.333 INFO:tasks.workunit.client.0.smithi110.stdout:9/108: creat d7/db/d1f/f23 x:0 0 0 2024-09-21T04:15:10.334 INFO:tasks.workunit.client.0.smithi110.stdout:8/237: dread d0/d7/d23/f33 [0,4194304] 0 2024-09-21T04:15:10.337 INFO:tasks.workunit.client.1.smithi148.stdout:7/159: dread f4 [4194304,4194304] 0 2024-09-21T04:15:10.341 INFO:tasks.workunit.client.0.smithi110.stdout:3/219: symlink d0/d12/d16/d1a/l49 0 2024-09-21T04:15:10.341 INFO:tasks.workunit.client.0.smithi110.stdout:3/220: creat d0/d12/d16/f4a x:0 0 0 2024-09-21T04:15:10.341 INFO:tasks.workunit.client.0.smithi110.stdout:3/221: write d0/d12/d16/f4a [576022,69391] 0 2024-09-21T04:15:10.342 INFO:tasks.workunit.client.0.smithi110.stdout:7/203: rename dd/d14/f22 to dd/d1c/d39/f3b 0 2024-09-21T04:15:10.344 INFO:tasks.workunit.client.0.smithi110.stdout:3/222: dread d0/f6 [0,4194304] 0 2024-09-21T04:15:10.344 INFO:tasks.workunit.client.0.smithi110.stdout:8/238: mkdir d0/d30/d54 0 2024-09-21T04:15:10.344 INFO:tasks.workunit.client.0.smithi110.stdout:8/239: read - d0/d7/d13/d1c/f50 zero size 2024-09-21T04:15:10.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/115: creat d3/d14/f29 x:0 0 0 2024-09-21T04:15:10.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/116: chown d3/d14/d1f/d20 66309602 1 2024-09-21T04:15:10.347 INFO:tasks.workunit.client.0.smithi110.stdout:3/223: symlink d0/l4b 0 2024-09-21T04:15:10.347 INFO:tasks.workunit.client.0.smithi110.stdout:3/224: write d0/d12/f32 [186857,73072] 0 2024-09-21T04:15:10.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/160: write d5/f6 [55190,73597] 0 2024-09-21T04:15:10.348 INFO:tasks.workunit.client.1.smithi148.stdout:7/161: rename d5/dc to d5/dc/d24/d2e 22 2024-09-21T04:15:10.349 INFO:tasks.workunit.client.0.smithi110.stdout:8/240: mknod d0/d7/d13/d1c/c55 0 2024-09-21T04:15:10.349 INFO:tasks.workunit.client.0.smithi110.stdout:8/241: fsync d0/f11 0 2024-09-21T04:15:10.350 INFO:tasks.workunit.client.1.smithi148.stdout:6/83: dwrite d0/f1 [4194304,4194304] 0 2024-09-21T04:15:10.351 INFO:tasks.workunit.client.1.smithi148.stdout:6/84: chown d0/d8/c9 1475 1 2024-09-21T04:15:10.351 INFO:tasks.workunit.client.1.smithi148.stdout:6/85: chown d0/d5/dc/fd 1275 1 2024-09-21T04:15:10.351 INFO:tasks.workunit.client.1.smithi148.stdout:6/86: truncate d0/d5/ff 209929 0 2024-09-21T04:15:10.351 INFO:tasks.workunit.client.1.smithi148.stdout:6/87: fsync d0/d5/dc/fd 0 2024-09-21T04:15:10.351 INFO:tasks.workunit.client.0.smithi110.stdout:8/242: symlink d0/d7/d13/d1c/d41/l56 0 2024-09-21T04:15:10.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/162: getdents d5/dc/d28 0 2024-09-21T04:15:10.355 INFO:tasks.workunit.client.0.smithi110.stdout:8/243: creat d0/d7/d13/f57 x:0 0 0 2024-09-21T04:15:10.357 INFO:tasks.workunit.client.0.smithi110.stdout:3/225: dread d0/f7 [0,4194304] 0 2024-09-21T04:15:10.359 INFO:tasks.workunit.client.1.smithi148.stdout:7/163: mknod d5/d14/c2f 0 2024-09-21T04:15:10.359 INFO:tasks.workunit.client.1.smithi148.stdout:7/164: write d5/d8/f22 [4225453,59979] 0 2024-09-21T04:15:10.360 INFO:tasks.workunit.client.1.smithi148.stdout:7/165: mknod d5/dc/d24/c30 0 2024-09-21T04:15:10.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/226: dread d0/f10 [0,4194304] 0 2024-09-21T04:15:10.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/227: link d0/d12/c17 d0/d12/d2a/c4c 0 2024-09-21T04:15:10.365 INFO:tasks.workunit.client.0.smithi110.stdout:9/109: dwrite d7/d10/f13 [0,4194304] 0 2024-09-21T04:15:10.365 INFO:tasks.workunit.client.0.smithi110.stdout:9/110: dread - d7/db/f1c zero size 2024-09-21T04:15:10.366 INFO:tasks.workunit.client.0.smithi110.stdout:9/111: chown d7/c9 0 1 2024-09-21T04:15:10.366 INFO:tasks.workunit.client.0.smithi110.stdout:9/112: fsync f3 0 2024-09-21T04:15:10.366 INFO:tasks.workunit.client.0.smithi110.stdout:9/113: chown d7/db/ld 1 1 2024-09-21T04:15:10.367 INFO:tasks.workunit.client.0.smithi110.stdout:3/228: link d0/fe d0/d12/d2a/f4d 0 2024-09-21T04:15:10.368 INFO:tasks.workunit.client.0.smithi110.stdout:3/229: getdents d0/d12 0 2024-09-21T04:15:10.368 INFO:tasks.workunit.client.0.smithi110.stdout:3/230: readlink d0/ld 0 2024-09-21T04:15:10.369 INFO:tasks.workunit.client.0.smithi110.stdout:9/114: mkdir d7/d10/d24 0 2024-09-21T04:15:10.369 INFO:tasks.workunit.client.0.smithi110.stdout:3/231: creat d0/d12/d16/d1a/f4e x:0 0 0 2024-09-21T04:15:10.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/115: rename d7/db/df to d7/db/d1f/d25 0 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.1.smithi148.stdout:0/142: sync 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.1.smithi148.stdout:1/131: sync 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/166: dread d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.1.smithi148.stdout:3/111: sync 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/167: readlink d5/l1c 0 2024-09-21T04:15:10.372 INFO:tasks.workunit.client.0.smithi110.stdout:3/232: rmdir d0/d22 39 2024-09-21T04:15:10.373 INFO:tasks.workunit.client.0.smithi110.stdout:9/116: mkdir d7/d26 0 2024-09-21T04:15:10.374 INFO:tasks.workunit.client.1.smithi148.stdout:1/132: link c2 db/dd/c2f 0 2024-09-21T04:15:10.375 INFO:tasks.workunit.client.1.smithi148.stdout:0/143: symlink d5/d6/d7/l33 0 2024-09-21T04:15:10.375 INFO:tasks.workunit.client.1.smithi148.stdout:0/144: creat d5/d6/d7/d11/f34 x:0 0 0 2024-09-21T04:15:10.375 INFO:tasks.workunit.client.1.smithi148.stdout:0/145: truncate d5/d6/d7/d11/d19/f26 1109032 0 2024-09-21T04:15:10.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/117: symlink d7/d10/d24/l27 0 2024-09-21T04:15:10.376 INFO:tasks.workunit.client.0.smithi110.stdout:9/118: readlink d7/db/d14/l22 0 2024-09-21T04:15:10.376 INFO:tasks.workunit.client.1.smithi148.stdout:6/88: dwrite d0/d5/ff [0,4194304] 0 2024-09-21T04:15:10.376 INFO:tasks.workunit.client.1.smithi148.stdout:6/89: creat d0/d5/f14 x:0 0 0 2024-09-21T04:15:10.376 INFO:tasks.workunit.client.0.smithi110.stdout:7/204: dwrite dd/d1c/f34 [4194304,4194304] 0 2024-09-21T04:15:10.376 INFO:tasks.workunit.client.1.smithi148.stdout:3/112: symlink d3/d5/d6/d12/l1c 0 2024-09-21T04:15:10.377 INFO:tasks.workunit.client.1.smithi148.stdout:7/168: creat d5/dc/d24/f31 x:0 0 0 2024-09-21T04:15:10.378 INFO:tasks.workunit.client.0.smithi110.stdout:9/119: link d7/db/d11/c15 d7/d10/d24/c28 0 2024-09-21T04:15:10.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/133: rename c5 to db/dd/d21/c30 0 2024-09-21T04:15:10.381 INFO:tasks.workunit.client.1.smithi148.stdout:3/113: dread d3/d5/d6/d12/f1a [0,4194304] 0 2024-09-21T04:15:10.383 INFO:tasks.workunit.client.0.smithi110.stdout:7/205: link dd/d1c/d39/f3b dd/d1c/d39/f3c 0 2024-09-21T04:15:10.385 INFO:tasks.workunit.client.0.smithi110.stdout:8/244: dwrite d0/d7/d13/d4b/f40 [0,4194304] 0 2024-09-21T04:15:10.385 INFO:tasks.workunit.client.0.smithi110.stdout:8/245: fsync d0/d30/f31 0 2024-09-21T04:15:10.386 INFO:tasks.workunit.client.1.smithi148.stdout:6/90: write d0/f1 [4436404,57739] 0 2024-09-21T04:15:10.387 INFO:tasks.workunit.client.1.smithi148.stdout:5/117: dwrite d3/f9 [0,4194304] 0 2024-09-21T04:15:10.390 INFO:tasks.workunit.client.0.smithi110.stdout:7/206: dread dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:10.399 INFO:tasks.workunit.client.0.smithi110.stdout:3/233: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:15:10.401 INFO:tasks.workunit.client.1.smithi148.stdout:1/134: dwrite db/dd/f13 [0,4194304] 0 2024-09-21T04:15:10.467 INFO:tasks.workunit.client.0.smithi110.stdout:9/120: symlink d7/db/d1f/d25/l29 0 2024-09-21T04:15:10.470 INFO:tasks.workunit.client.1.smithi148.stdout:7/169: symlink d5/l32 0 2024-09-21T04:15:10.470 INFO:tasks.workunit.client.1.smithi148.stdout:7/170: stat d5/dc/d28/f29 0 2024-09-21T04:15:10.471 INFO:tasks.workunit.client.1.smithi148.stdout:7/171: truncate d5/d8/f9 370952 0 2024-09-21T04:15:10.471 INFO:tasks.workunit.client.1.smithi148.stdout:7/172: dread - d5/dc/f10 zero size 2024-09-21T04:15:10.471 INFO:tasks.workunit.client.0.smithi110.stdout:8/246: creat d0/d30/f58 x:0 0 0 2024-09-21T04:15:10.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/146: rename d5/f20 to d5/d6/d2a/d2b/f35 0 2024-09-21T04:15:10.478 INFO:tasks.workunit.client.1.smithi148.stdout:3/114: creat d3/d5/d6/d12/da/f1d x:0 0 0 2024-09-21T04:15:10.483 INFO:tasks.workunit.client.0.smithi110.stdout:8/247: dwrite d0/d7/d13/d4b/f45 [0,4194304] 0 2024-09-21T04:15:10.484 INFO:tasks.workunit.client.0.smithi110.stdout:7/207: mknod dd/c3d 0 2024-09-21T04:15:10.484 INFO:tasks.workunit.client.0.smithi110.stdout:7/208: stat dd/d14/f36 0 2024-09-21T04:15:10.484 INFO:tasks.workunit.client.0.smithi110.stdout:7/209: fsync dd/d14/d17/f3a 0 2024-09-21T04:15:10.486 INFO:tasks.workunit.client.0.smithi110.stdout:3/234: getdents d0/d22/d29 0 2024-09-21T04:15:10.486 INFO:tasks.workunit.client.0.smithi110.stdout:3/235: chown d0/d12/d2a/f31 15 1 2024-09-21T04:15:10.486 INFO:tasks.workunit.client.0.smithi110.stdout:3/236: creat d0/d12/d3c/f4f x:0 0 0 2024-09-21T04:15:10.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/91: unlink d0/d5/dc/fd 0 2024-09-21T04:15:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/118: symlink d3/d24/l2a 0 2024-09-21T04:15:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/119: write d3/dc/f17 [2137146,20894] 0 2024-09-21T04:15:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/120: chown d3/d14/d1f/f28 17223151 1 2024-09-21T04:15:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:6/92: read d0/d8/f10 [33754,9588] 0 2024-09-21T04:15:10.491 INFO:tasks.workunit.client.1.smithi148.stdout:0/147: dwrite d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:10.492 INFO:tasks.workunit.client.0.smithi110.stdout:9/121: rename d7/db/f1c to d7/d10/d24/f2a 0 2024-09-21T04:15:10.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/173: dwrite d5/d8/ff [0,4194304] 0 2024-09-21T04:15:10.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/174: write d5/d8/f9 [1315500,70280] 0 2024-09-21T04:15:10.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/175: readlink d5/d8/l11 0 2024-09-21T04:15:10.494 INFO:tasks.workunit.client.1.smithi148.stdout:1/135: symlink db/dd/d21/l31 0 2024-09-21T04:15:10.497 INFO:tasks.workunit.client.0.smithi110.stdout:3/237: link d0/l26 d0/d22/d29/l50 0 2024-09-21T04:15:10.498 INFO:tasks.workunit.client.0.smithi110.stdout:9/122: rename d7/db/d14/l22 to d7/d10/d24/l2b 0 2024-09-21T04:15:10.499 INFO:tasks.workunit.client.0.smithi110.stdout:9/123: dread - d7/d10/d24/f2a zero size 2024-09-21T04:15:10.499 INFO:tasks.workunit.client.0.smithi110.stdout:3/238: mkdir d0/d51 0 2024-09-21T04:15:10.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/124: creat d7/d26/f2c x:0 0 0 2024-09-21T04:15:10.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/125: fdatasync d7/db/f20 0 2024-09-21T04:15:10.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/126: creat d7/d26/f2d x:0 0 0 2024-09-21T04:15:10.502 INFO:tasks.workunit.client.1.smithi148.stdout:5/121: symlink d3/d14/d1a/l2b 0 2024-09-21T04:15:10.503 INFO:tasks.workunit.client.1.smithi148.stdout:0/148: dread f4 [4194304,4194304] 0 2024-09-21T04:15:10.503 INFO:tasks.workunit.client.0.smithi110.stdout:9/127: dread f2 [4194304,4194304] 0 2024-09-21T04:15:10.503 INFO:tasks.workunit.client.0.smithi110.stdout:8/248: dwrite d0/d7/d13/d46/f51 [0,4194304] 0 2024-09-21T04:15:10.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/239: creat d0/d12/f52 x:0 0 0 2024-09-21T04:15:10.507 INFO:tasks.workunit.client.0.smithi110.stdout:9/128: truncate f6 1089409 0 2024-09-21T04:15:10.509 INFO:tasks.workunit.client.1.smithi148.stdout:1/136: rename db/dd/d21/c30 to db/dd/c32 0 2024-09-21T04:15:10.509 INFO:tasks.workunit.client.1.smithi148.stdout:1/137: creat db/d24/f33 x:0 0 0 2024-09-21T04:15:10.512 INFO:tasks.workunit.client.0.smithi110.stdout:8/249: link d0/d7/d13/d1c/f50 d0/d30/d54/f59 0 2024-09-21T04:15:10.512 INFO:tasks.workunit.client.0.smithi110.stdout:8/250: stat d0/d7/d13/d4b/l25 0 2024-09-21T04:15:10.512 INFO:tasks.workunit.client.0.smithi110.stdout:8/251: readlink d0/l15 0 2024-09-21T04:15:10.513 INFO:tasks.workunit.client.1.smithi148.stdout:5/122: dread d3/f9 [0,4194304] 0 2024-09-21T04:15:10.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/240: creat d0/d12/d2a/f53 x:0 0 0 2024-09-21T04:15:10.514 INFO:tasks.workunit.client.1.smithi148.stdout:1/138: symlink db/dd/l34 0 2024-09-21T04:15:10.516 INFO:tasks.workunit.client.0.smithi110.stdout:0/391: sync 2024-09-21T04:15:10.516 INFO:tasks.workunit.client.0.smithi110.stdout:6/199: sync 2024-09-21T04:15:10.516 INFO:tasks.workunit.client.1.smithi148.stdout:1/139: symlink db/d24/d2e/l35 0 2024-09-21T04:15:10.517 INFO:tasks.workunit.client.0.smithi110.stdout:0/392: unlink d4/d1c/d62/d69/l6f 0 2024-09-21T04:15:10.522 INFO:tasks.workunit.client.1.smithi148.stdout:8/105: sync 2024-09-21T04:15:10.522 INFO:tasks.workunit.client.1.smithi148.stdout:9/148: sync 2024-09-21T04:15:10.522 INFO:tasks.workunit.client.1.smithi148.stdout:1/140: dread db/dd/f20 [0,4194304] 0 2024-09-21T04:15:10.523 INFO:tasks.workunit.client.1.smithi148.stdout:8/106: mknod dc/c1e 0 2024-09-21T04:15:10.524 INFO:tasks.workunit.client.1.smithi148.stdout:5/123: write d3/f8 [1119303,30247] 0 2024-09-21T04:15:10.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/149: creat d5/d22/f3c x:0 0 0 2024-09-21T04:15:10.526 INFO:tasks.workunit.client.1.smithi148.stdout:8/107: rename dc/c1e to dc/c1f 0 2024-09-21T04:15:10.527 INFO:tasks.workunit.client.1.smithi148.stdout:5/124: getdents d3/d14/d1f 0 2024-09-21T04:15:10.527 INFO:tasks.workunit.client.1.smithi148.stdout:8/108: fdatasync dc/f14 0 2024-09-21T04:15:10.528 INFO:tasks.workunit.client.1.smithi148.stdout:5/125: mknod d3/d14/d1f/c2c 0 2024-09-21T04:15:10.528 INFO:tasks.workunit.client.1.smithi148.stdout:5/126: creat d3/d14/f2d x:0 0 0 2024-09-21T04:15:10.528 INFO:tasks.workunit.client.1.smithi148.stdout:8/109: symlink dc/d13/l20 0 2024-09-21T04:15:10.528 INFO:tasks.workunit.client.1.smithi148.stdout:5/127: rmdir d3/d14/d1f 39 2024-09-21T04:15:10.529 INFO:tasks.workunit.client.1.smithi148.stdout:5/128: chown f0 3 1 2024-09-21T04:15:10.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/110: truncate f8 277527 0 2024-09-21T04:15:10.530 INFO:tasks.workunit.client.1.smithi148.stdout:5/129: truncate d3/f12 4003523 0 2024-09-21T04:15:10.531 INFO:tasks.workunit.client.0.smithi110.stdout:7/210: dwrite dd/d14/d17/f2b [0,4194304] 0 2024-09-21T04:15:10.532 INFO:tasks.workunit.client.0.smithi110.stdout:7/211: creat dd/f3e x:0 0 0 2024-09-21T04:15:10.532 INFO:tasks.workunit.client.0.smithi110.stdout:7/212: rename dd/d14/d17 to dd/d14/d17/d3f 22 2024-09-21T04:15:10.533 INFO:tasks.workunit.client.1.smithi148.stdout:5/130: mkdir d3/d14/d1f/d26/d2e 0 2024-09-21T04:15:10.533 INFO:tasks.workunit.client.1.smithi148.stdout:9/150: dread d5/d1e/f2d [0,4194304] 0 2024-09-21T04:15:10.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/213: symlink dd/d14/d17/l40 0 2024-09-21T04:15:10.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/214: fdatasync dd/f2e 0 2024-09-21T04:15:10.536 INFO:tasks.workunit.client.1.smithi148.stdout:9/151: link d5/l17 d5/d39/l3d 0 2024-09-21T04:15:10.537 INFO:tasks.workunit.client.0.smithi110.stdout:7/215: rmdir dd/d1c/d39 39 2024-09-21T04:15:10.537 INFO:tasks.workunit.client.1.smithi148.stdout:6/93: dwrite d0/d5/f12 [0,4194304] 0 2024-09-21T04:15:10.538 INFO:tasks.workunit.client.1.smithi148.stdout:9/152: truncate d5/f1b 327356 0 2024-09-21T04:15:10.541 INFO:tasks.workunit.client.1.smithi148.stdout:4/138: sync 2024-09-21T04:15:10.542 INFO:tasks.workunit.client.1.smithi148.stdout:2/212: sync 2024-09-21T04:15:10.542 INFO:tasks.workunit.client.1.smithi148.stdout:6/94: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:15:10.543 INFO:tasks.workunit.client.1.smithi148.stdout:9/153: chown d5/f16 6261 1 2024-09-21T04:15:10.544 INFO:tasks.workunit.client.1.smithi148.stdout:4/139: symlink d4/d23/l28 0 2024-09-21T04:15:10.544 INFO:tasks.workunit.client.1.smithi148.stdout:4/140: truncate d4/f8 1386143 0 2024-09-21T04:15:10.545 INFO:tasks.workunit.client.0.smithi110.stdout:0/393: dread d4/fb [0,4194304] 0 2024-09-21T04:15:10.545 INFO:tasks.workunit.client.1.smithi148.stdout:9/154: creat d5/d1f/f3e x:0 0 0 2024-09-21T04:15:10.547 INFO:tasks.workunit.client.0.smithi110.stdout:0/394: mkdir d4/dd/d60/d72 0 2024-09-21T04:15:10.549 INFO:tasks.workunit.client.1.smithi148.stdout:9/155: link d5/f36 d5/d22/f3f 0 2024-09-21T04:15:10.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/141: mkdir d4/d12/d16/d29 0 2024-09-21T04:15:10.550 INFO:tasks.workunit.client.0.smithi110.stdout:0/395: mknod d4/d1c/c73 0 2024-09-21T04:15:10.550 INFO:tasks.workunit.client.0.smithi110.stdout:9/129: dwrite d7/db/d1f/f23 [0,4194304] 0 2024-09-21T04:15:10.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/156: mknod d5/d22/d28/d38/c40 0 2024-09-21T04:15:10.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/157: creat d5/d22/d28/f41 x:0 0 0 2024-09-21T04:15:10.551 INFO:tasks.workunit.client.1.smithi148.stdout:9/158: dread - d5/d22/d28/f41 zero size 2024-09-21T04:15:10.551 INFO:tasks.workunit.client.1.smithi148.stdout:9/159: truncate d5/d22/d28/f41 967580 0 2024-09-21T04:15:10.551 INFO:tasks.workunit.client.1.smithi148.stdout:4/142: mkdir d4/d23/d2a 0 2024-09-21T04:15:10.551 INFO:tasks.workunit.client.0.smithi110.stdout:0/396: link d4/dd/d10/d29/f6b d4/dd/d53/d66/f74 0 2024-09-21T04:15:10.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/397: chown d4/dd/d10/f2a 11104 1 2024-09-21T04:15:10.554 INFO:tasks.workunit.client.1.smithi148.stdout:9/160: creat d5/f42 x:0 0 0 2024-09-21T04:15:10.554 INFO:tasks.workunit.client.0.smithi110.stdout:9/130: truncate f2 2679098 0 2024-09-21T04:15:10.554 INFO:tasks.workunit.client.0.smithi110.stdout:9/131: stat d7/db/d1f 0 2024-09-21T04:15:10.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/95: write d0/d5/ff [3446535,74865] 0 2024-09-21T04:15:10.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/143: creat d4/d23/f2b x:0 0 0 2024-09-21T04:15:10.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/144: fsync d4/f10 0 2024-09-21T04:15:10.556 INFO:tasks.workunit.client.0.smithi110.stdout:0/398: mkdir d4/dd/d10/d29/d75 0 2024-09-21T04:15:10.557 INFO:tasks.workunit.client.0.smithi110.stdout:9/132: creat d7/db/d11/f2e x:0 0 0 2024-09-21T04:15:10.558 INFO:tasks.workunit.client.1.smithi148.stdout:4/145: truncate d4/d12/f15 596678 0 2024-09-21T04:15:10.560 INFO:tasks.workunit.client.1.smithi148.stdout:4/146: mkdir d4/d12/d2c 0 2024-09-21T04:15:10.564 INFO:tasks.workunit.client.1.smithi148.stdout:4/147: unlink d4/d12/d16/d26/l24 0 2024-09-21T04:15:10.565 INFO:tasks.workunit.client.0.smithi110.stdout:8/252: fsync d0/d7/d13/d1c/f50 0 2024-09-21T04:15:10.567 INFO:tasks.workunit.client.0.smithi110.stdout:8/253: getdents d0/d7/d13/d1c/d41 0 2024-09-21T04:15:10.569 INFO:tasks.workunit.client.1.smithi148.stdout:4/148: dread d4/d12/f1b [0,4194304] 0 2024-09-21T04:15:10.569 INFO:tasks.workunit.client.1.smithi148.stdout:4/149: fdatasync d4/d12/f1b 0 2024-09-21T04:15:10.570 INFO:tasks.workunit.client.0.smithi110.stdout:8/254: unlink d0/d7/d13/d4b/f45 0 2024-09-21T04:15:10.571 INFO:tasks.workunit.client.1.smithi148.stdout:1/141: dwrite db/d24/f25 [0,4194304] 0 2024-09-21T04:15:10.572 INFO:tasks.workunit.client.1.smithi148.stdout:4/150: mkdir d4/d12/d2c/d2d 0 2024-09-21T04:15:10.572 INFO:tasks.workunit.client.1.smithi148.stdout:4/151: chown d4/l25 428893220 1 2024-09-21T04:15:10.575 INFO:tasks.workunit.client.0.smithi110.stdout:3/241: dwrite d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:10.575 INFO:tasks.workunit.client.1.smithi148.stdout:4/152: dread d4/d12/f1b [0,4194304] 0 2024-09-21T04:15:10.577 INFO:tasks.workunit.client.1.smithi148.stdout:4/153: rename d4/f17 to d4/d23/f2e 0 2024-09-21T04:15:10.578 INFO:tasks.workunit.client.1.smithi148.stdout:6/96: read d0/d5/f12 [3602987,76475] 0 2024-09-21T04:15:10.578 INFO:tasks.workunit.client.0.smithi110.stdout:3/242: dread d0/d12/d16/d1a/f39 [0,4194304] 0 2024-09-21T04:15:10.579 INFO:tasks.workunit.client.1.smithi148.stdout:6/97: mknod d0/d5/dc/c15 0 2024-09-21T04:15:10.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/98: rmdir d0/d8 39 2024-09-21T04:15:10.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/99: write d0/d5/fb [5130333,107342] 0 2024-09-21T04:15:10.583 INFO:tasks.workunit.client.1.smithi148.stdout:8/111: dwrite dc/fe [0,4194304] 0 2024-09-21T04:15:10.587 INFO:tasks.workunit.client.1.smithi148.stdout:8/112: rename dc/f1a to dc/f21 0 2024-09-21T04:15:10.587 INFO:tasks.workunit.client.1.smithi148.stdout:8/113: symlink dc/l22 0 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/114: symlink dc/l23 0 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/115: dread - dc/f1d zero size 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/116: creat dc/d13/f24 x:0 0 0 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/117: readlink dc/l15 0 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/118: chown dc/l12 1873 1 2024-09-21T04:15:10.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/119: dread - dc/f1d zero size 2024-09-21T04:15:10.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/131: dwrite d3/f1b [0,4194304] 0 2024-09-21T04:15:10.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/132: unlink d3/fd 0 2024-09-21T04:15:10.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/161: dwrite d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:10.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/162: creat d5/d22/d28/f43 x:0 0 0 2024-09-21T04:15:10.602 INFO:tasks.workunit.client.0.smithi110.stdout:0/399: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:10.602 INFO:tasks.workunit.client.0.smithi110.stdout:0/400: chown d4/d1c/f24 25 1 2024-09-21T04:15:10.602 INFO:tasks.workunit.client.0.smithi110.stdout:0/401: write d4/dd/d53/d66/f74 [283354,41636] 0 2024-09-21T04:15:10.602 INFO:tasks.workunit.client.0.smithi110.stdout:9/133: dwrite d7/d10/f13 [0,4194304] 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.0.smithi110.stdout:0/402: link d4/dd/l17 d4/dd/d53/l76 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.0.smithi110.stdout:7/216: dwrite dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.0.smithi110.stdout:7/217: fsync dd/d1c/f2c 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/100: fsync d0/d5/ff 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/213: dwrite d0/f4 [4194304,4194304] 0 2024-09-21T04:15:10.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/101: write d0/d5/f14 [441948,101520] 0 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:6/102: chown d0/d5/dc/c15 268315 1 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:6/103: stat d0/d8/c9 0 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:8/120: dwrite dc/f17 [0,4194304] 0 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:1/142: dwrite db/d24/f25 [0,4194304] 0 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:1/143: chown db/dd/f13 6011562 1 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:1/144: dread - db/dd/f23 zero size 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:9/163: creat d5/d22/d28/d38/f44 x:0 0 0 2024-09-21T04:15:10.608 INFO:tasks.workunit.client.1.smithi148.stdout:9/164: stat d5/d22/d28/f43 0 2024-09-21T04:15:10.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/134: mknod d7/d10/c2f 0 2024-09-21T04:15:10.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/135: fdatasync d7/d10/f13 0 2024-09-21T04:15:10.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/136: write d7/d26/f2c [487549,48662] 0 2024-09-21T04:15:10.612 INFO:tasks.workunit.client.1.smithi148.stdout:5/133: dwrite d3/d14/d1a/f1c [0,4194304] 0 2024-09-21T04:15:10.613 INFO:tasks.workunit.client.1.smithi148.stdout:8/121: dread f5 [0,4194304] 0 2024-09-21T04:15:10.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/255: getdents d0/d7/d13/d4b 0 2024-09-21T04:15:10.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/256: creat d0/d7/d13/f5a x:0 0 0 2024-09-21T04:15:10.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/257: chown d0/d7/d13/d4b 126606490 1 2024-09-21T04:15:10.621 INFO:tasks.workunit.client.1.smithi148.stdout:6/104: mkdir d0/d5/d16 0 2024-09-21T04:15:10.623 INFO:tasks.workunit.client.0.smithi110.stdout:0/403: getdents d4/d1c/d62/d69 0 2024-09-21T04:15:10.623 INFO:tasks.workunit.client.0.smithi110.stdout:0/404: stat d4/l4d 0 2024-09-21T04:15:10.626 INFO:tasks.workunit.client.0.smithi110.stdout:8/258: creat d0/d7/d13/d4b/f5b x:0 0 0 2024-09-21T04:15:10.626 INFO:tasks.workunit.client.0.smithi110.stdout:8/259: chown d0/d7/d13/f2d 63 1 2024-09-21T04:15:10.627 INFO:tasks.workunit.client.0.smithi110.stdout:8/260: fdatasync d0/d7/f1a 0 2024-09-21T04:15:10.629 INFO:tasks.workunit.client.1.smithi148.stdout:1/145: dwrite db/dd/f13 [4194304,4194304] 0 2024-09-21T04:15:10.630 INFO:tasks.workunit.client.0.smithi110.stdout:9/137: rmdir d7/db/d1f 39 2024-09-21T04:15:10.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/405: dwrite d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:10.644 INFO:tasks.workunit.client.1.smithi148.stdout:5/134: write d3/f7 [6449998,125421] 0 2024-09-21T04:15:10.645 INFO:tasks.workunit.client.0.smithi110.stdout:9/138: dwrite d7/d26/f2d [0,4194304] 0 2024-09-21T04:15:10.645 INFO:tasks.workunit.client.0.smithi110.stdout:9/139: fsync d7/db/d11/f2e 0 2024-09-21T04:15:10.651 INFO:tasks.workunit.client.0.smithi110.stdout:9/140: dread f6 [0,4194304] 0 2024-09-21T04:15:10.652 INFO:tasks.workunit.client.0.smithi110.stdout:9/141: dread d7/f8 [0,4194304] 0 2024-09-21T04:15:10.653 INFO:tasks.workunit.client.0.smithi110.stdout:9/142: dread - d7/d10/d24/f2a zero size 2024-09-21T04:15:10.656 INFO:tasks.workunit.client.0.smithi110.stdout:9/143: dread f3 [0,4194304] 0 2024-09-21T04:15:10.703 INFO:tasks.workunit.client.1.smithi148.stdout:8/122: unlink dc/l15 0 2024-09-21T04:15:10.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/214: link d0/d9/f39 d0/d9/d40/f44 0 2024-09-21T04:15:10.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/215: dread - d0/d9/dc/d13/f21 zero size 2024-09-21T04:15:10.705 INFO:tasks.workunit.client.1.smithi148.stdout:6/105: mknod d0/c17 0 2024-09-21T04:15:10.707 INFO:tasks.workunit.client.0.smithi110.stdout:0/406: creat d4/d1c/d62/d69/f77 x:0 0 0 2024-09-21T04:15:10.707 INFO:tasks.workunit.client.0.smithi110.stdout:0/407: stat d4/d47/d2c/c65 0 2024-09-21T04:15:10.708 INFO:tasks.workunit.client.1.smithi148.stdout:1/146: rmdir db/dd/d21 39 2024-09-21T04:15:10.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/135: creat d3/d24/f2f x:0 0 0 2024-09-21T04:15:10.711 INFO:tasks.workunit.client.0.smithi110.stdout:2/133: sync 2024-09-21T04:15:10.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/408: mknod d4/dd/d10/d29/d75/c78 0 2024-09-21T04:15:10.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/409: dread - d4/d47/d2c/d3a/f6c zero size 2024-09-21T04:15:10.712 INFO:tasks.workunit.client.0.smithi110.stdout:0/410: getdents d4/dd/d10/d29/d5f 0 2024-09-21T04:15:10.712 INFO:tasks.workunit.client.0.smithi110.stdout:0/411: creat d4/dd/d53/f79 x:0 0 0 2024-09-21T04:15:10.712 INFO:tasks.workunit.client.0.smithi110.stdout:9/144: getdents d7/db 0 2024-09-21T04:15:10.714 INFO:tasks.workunit.client.1.smithi148.stdout:6/106: unlink d0/f1 0 2024-09-21T04:15:10.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/123: write f7 [792981,128066] 0 2024-09-21T04:15:10.715 INFO:tasks.workunit.client.1.smithi148.stdout:6/107: truncate d0/d5/f12 4768345 0 2024-09-21T04:15:10.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/124: chown dc/c1b 428114 1 2024-09-21T04:15:10.716 INFO:tasks.workunit.client.0.smithi110.stdout:2/134: link d2/d1a/de/l11 d2/l23 0 2024-09-21T04:15:10.716 INFO:tasks.workunit.client.0.smithi110.stdout:2/135: readlink d2/d1a/d1f/l10 0 2024-09-21T04:15:10.720 INFO:tasks.workunit.client.1.smithi148.stdout:2/216: dwrite d0/f1e [0,4194304] 0 2024-09-21T04:15:10.720 INFO:tasks.workunit.client.1.smithi148.stdout:2/217: creat d0/d9/dc/d13/d25/f45 x:0 0 0 2024-09-21T04:15:10.720 INFO:tasks.workunit.client.1.smithi148.stdout:2/218: creat d0/d9/dc/f46 x:0 0 0 2024-09-21T04:15:10.721 INFO:tasks.workunit.client.0.smithi110.stdout:0/412: symlink d4/dd/l7a 0 2024-09-21T04:15:10.721 INFO:tasks.workunit.client.0.smithi110.stdout:0/413: dread - d4/f57 zero size 2024-09-21T04:15:10.722 INFO:tasks.workunit.client.0.smithi110.stdout:0/414: dread d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:10.722 INFO:tasks.workunit.client.1.smithi148.stdout:2/219: dread d0/d9/dc/d13/f1a [0,4194304] 0 2024-09-21T04:15:10.723 INFO:tasks.workunit.client.1.smithi148.stdout:6/108: mkdir d0/d8/d18 0 2024-09-21T04:15:10.724 INFO:tasks.workunit.client.1.smithi148.stdout:8/125: symlink dc/l25 0 2024-09-21T04:15:10.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/220: write d0/d9/dc/f24 [3735581,30984] 0 2024-09-21T04:15:10.756 INFO:tasks.workunit.client.1.smithi148.stdout:5/136: dwrite d3/f7 [4194304,4194304] 0 2024-09-21T04:15:10.756 INFO:tasks.workunit.client.1.smithi148.stdout:5/137: chown d3/d14/f29 61134106 1 2024-09-21T04:15:10.757 INFO:tasks.workunit.client.1.smithi148.stdout:5/138: symlink d3/l30 0 2024-09-21T04:15:10.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/139: creat d3/d14/d1a/f31 x:0 0 0 2024-09-21T04:15:10.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/140: truncate d3/dc/f19 1401607 0 2024-09-21T04:15:10.760 INFO:tasks.workunit.client.1.smithi148.stdout:5/141: rename d3/d14/d1f/f28 to d3/d14/d1f/d26/f32 0 2024-09-21T04:15:10.760 INFO:tasks.workunit.client.1.smithi148.stdout:1/147: dwrite db/dd/f13 [4194304,4194304] 0 2024-09-21T04:15:10.760 INFO:tasks.workunit.client.0.smithi110.stdout:9/145: dwrite f6 [0,4194304] 0 2024-09-21T04:15:10.760 INFO:tasks.workunit.client.0.smithi110.stdout:9/146: stat d7/db/d1f/f23 0 2024-09-21T04:15:10.761 INFO:tasks.workunit.client.1.smithi148.stdout:8/126: dwrite dc/f18 [0,4194304] 0 2024-09-21T04:15:10.761 INFO:tasks.workunit.client.0.smithi110.stdout:0/415: dwrite d4/f31 [0,4194304] 0 2024-09-21T04:15:10.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/148: link db/dd/c2f db/d24/c36 0 2024-09-21T04:15:10.767 INFO:tasks.workunit.client.0.smithi110.stdout:0/416: mkdir d4/dd/d53/d7b 0 2024-09-21T04:15:10.769 INFO:tasks.workunit.client.0.smithi110.stdout:0/417: symlink d4/dd/d10/l7c 0 2024-09-21T04:15:10.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/418: truncate d4/dd/d10/f70 247254 0 2024-09-21T04:15:10.771 INFO:tasks.workunit.client.0.smithi110.stdout:0/419: rename d4/l4d to d4/dd/l7d 0 2024-09-21T04:15:10.772 INFO:tasks.workunit.client.1.smithi148.stdout:6/109: truncate d0/d5/f12 2286420 0 2024-09-21T04:15:10.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/142: dwrite d3/d14/d1a/f27 [0,4194304] 0 2024-09-21T04:15:10.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/143: dread - d3/d14/d1a/f31 zero size 2024-09-21T04:15:10.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/144: chown d3 9815279 1 2024-09-21T04:15:10.780 INFO:tasks.workunit.client.1.smithi148.stdout:8/127: dwrite f7 [4194304,4194304] 0 2024-09-21T04:15:10.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/110: dread d0/d5/fb [0,4194304] 0 2024-09-21T04:15:10.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/111: creat d0/d5/f19 x:0 0 0 2024-09-21T04:15:10.790 INFO:tasks.workunit.client.1.smithi148.stdout:5/145: dwrite d3/fa [0,4194304] 0 2024-09-21T04:15:10.806 INFO:tasks.workunit.client.0.smithi110.stdout:0/420: creat d4/dd/d60/f7e x:0 0 0 2024-09-21T04:15:10.806 INFO:tasks.workunit.client.0.smithi110.stdout:0/421: fsync d4/dd/f1b 0 2024-09-21T04:15:10.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/112: mkdir d0/d1a 0 2024-09-21T04:15:10.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/113: read - d0/d5/f19 zero size 2024-09-21T04:15:10.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/114: stat d0 0 2024-09-21T04:15:10.811 INFO:tasks.workunit.client.1.smithi148.stdout:5/146: rename d3/d14/d1a/f1c to d3/d14/d1a/f33 0 2024-09-21T04:15:10.811 INFO:tasks.workunit.client.1.smithi148.stdout:5/147: fsync d3/fa 0 2024-09-21T04:15:10.814 INFO:tasks.workunit.client.0.smithi110.stdout:0/422: write d4/d47/f23 [4066030,118333] 0 2024-09-21T04:15:10.815 INFO:tasks.workunit.client.1.smithi148.stdout:8/128: getdents dc 0 2024-09-21T04:15:10.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/129: mknod dc/c26 0 2024-09-21T04:15:10.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/130: chown dc/d13/l20 16962224 1 2024-09-21T04:15:10.818 INFO:tasks.workunit.client.1.smithi148.stdout:8/131: write f8 [600060,24652] 0 2024-09-21T04:15:10.820 INFO:tasks.workunit.client.1.smithi148.stdout:3/115: sync 2024-09-21T04:15:10.821 INFO:tasks.workunit.client.1.smithi148.stdout:5/148: mkdir d3/d34 0 2024-09-21T04:15:10.822 INFO:tasks.workunit.client.1.smithi148.stdout:8/132: symlink dc/l27 0 2024-09-21T04:15:10.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/116: write d3/d5/d6/d12/f1a [2271546,126082] 0 2024-09-21T04:15:10.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/117: write d3/d5/d6/d12/da/f1d [39305,34469] 0 2024-09-21T04:15:10.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/133: mkdir dc/d13/d28 0 2024-09-21T04:15:10.824 INFO:tasks.workunit.client.0.smithi110.stdout:0/423: dread d4/f31 [0,4194304] 0 2024-09-21T04:15:10.824 INFO:tasks.workunit.client.0.smithi110.stdout:0/424: read - d4/f57 zero size 2024-09-21T04:15:10.825 INFO:tasks.workunit.client.0.smithi110.stdout:0/425: mknod d4/d1c/d62/c7f 0 2024-09-21T04:15:10.825 INFO:tasks.workunit.client.1.smithi148.stdout:5/149: dread d3/f8 [0,4194304] 0 2024-09-21T04:15:10.825 INFO:tasks.workunit.client.1.smithi148.stdout:5/150: write d3/d14/d1f/d26/f32 [934535,123763] 0 2024-09-21T04:15:10.829 INFO:tasks.workunit.client.1.smithi148.stdout:3/118: creat d3/d5/f1e x:0 0 0 2024-09-21T04:15:10.830 INFO:tasks.workunit.client.1.smithi148.stdout:5/151: symlink d3/d14/d1a/l35 0 2024-09-21T04:15:10.830 INFO:tasks.workunit.client.1.smithi148.stdout:8/134: dread f7 [0,4194304] 0 2024-09-21T04:15:10.830 INFO:tasks.workunit.client.1.smithi148.stdout:8/135: write dc/d13/f19 [782413,114] 0 2024-09-21T04:15:10.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/115: rename d0/d5 to d0/d8/d1b 0 2024-09-21T04:15:10.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/116: write d0/d8/d1b/fb [5476975,40691] 0 2024-09-21T04:15:10.832 INFO:tasks.workunit.client.1.smithi148.stdout:6/117: stat d0/d8/d1b 0 2024-09-21T04:15:10.832 INFO:tasks.workunit.client.1.smithi148.stdout:6/118: write d0/d8/d1b/f14 [1206733,80792] 0 2024-09-21T04:15:10.832 INFO:tasks.workunit.client.1.smithi148.stdout:6/119: readlink d0/d8/le 0 2024-09-21T04:15:10.835 INFO:tasks.workunit.client.1.smithi148.stdout:3/119: dread d3/d5/d6/d12/f1a [0,4194304] 0 2024-09-21T04:15:10.837 INFO:tasks.workunit.client.1.smithi148.stdout:5/152: rename d3/fb to d3/f36 0 2024-09-21T04:15:10.838 INFO:tasks.workunit.client.0.smithi110.stdout:0/426: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:10.842 INFO:tasks.workunit.client.1.smithi148.stdout:5/153: rename d3/d14/d1f/d20 to d3/d37 0 2024-09-21T04:15:10.844 INFO:tasks.workunit.client.1.smithi148.stdout:5/154: symlink d3/d14/d1f/d26/l38 0 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:5/245: sync 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:6/200: sync 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:4/337: sync 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:1/461: sync 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:1/462: creat d2/d9/d23/d4e/f97 x:0 0 0 2024-09-21T04:15:10.849 INFO:tasks.workunit.client.0.smithi110.stdout:1/463: fdatasync d2/d9/d49/d4f/d53/f8c 0 2024-09-21T04:15:10.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/154: sync 2024-09-21T04:15:10.850 INFO:tasks.workunit.client.1.smithi148.stdout:0/149: sync 2024-09-21T04:15:10.850 INFO:tasks.workunit.client.1.smithi148.stdout:7/176: sync 2024-09-21T04:15:10.850 INFO:tasks.workunit.client.1.smithi148.stdout:0/150: creat d5/d6/d2a/d2b/f36 x:0 0 0 2024-09-21T04:15:10.851 INFO:tasks.workunit.client.0.smithi110.stdout:5/246: rename d1/dc/c26 to d1/d49/c53 0 2024-09-21T04:15:10.851 INFO:tasks.workunit.client.0.smithi110.stdout:6/201: mknod d4/d12/d17/d2e/d38/c41 0 2024-09-21T04:15:10.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/177: mknod d5/dc/c33 0 2024-09-21T04:15:10.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/178: truncate d5/dc/f10 155116 0 2024-09-21T04:15:10.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/179: chown d5/d8/l11 1973321 1 2024-09-21T04:15:10.855 INFO:tasks.workunit.client.0.smithi110.stdout:1/464: symlink d2/d9/d26/d3d/d8f/l98 0 2024-09-21T04:15:10.856 INFO:tasks.workunit.client.1.smithi148.stdout:7/180: truncate d5/f13 3171039 0 2024-09-21T04:15:10.856 INFO:tasks.workunit.client.1.smithi148.stdout:7/181: chown d5/l32 0 1 2024-09-21T04:15:10.856 INFO:tasks.workunit.client.0.smithi110.stdout:5/247: creat d1/d22/d51/f54 x:0 0 0 2024-09-21T04:15:10.857 INFO:tasks.workunit.client.1.smithi148.stdout:6/120: dwrite d0/d8/f10 [0,4194304] 0 2024-09-21T04:15:10.858 INFO:tasks.workunit.client.1.smithi148.stdout:7/182: creat d5/f34 x:0 0 0 2024-09-21T04:15:10.858 INFO:tasks.workunit.client.1.smithi148.stdout:7/183: dread - d5/f34 zero size 2024-09-21T04:15:10.858 INFO:tasks.workunit.client.0.smithi110.stdout:5/248: mknod d1/dc/c55 0 2024-09-21T04:15:10.861 INFO:tasks.workunit.client.1.smithi148.stdout:5/155: dread f0 [0,4194304] 0 2024-09-21T04:15:10.861 INFO:tasks.workunit.client.0.smithi110.stdout:5/249: symlink d1/d22/l56 0 2024-09-21T04:15:10.862 INFO:tasks.workunit.client.1.smithi148.stdout:6/121: creat d0/d1a/f1c x:0 0 0 2024-09-21T04:15:10.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/338: rename d17/d3e/d60 to d17/d1e/d7b 0 2024-09-21T04:15:10.865 INFO:tasks.workunit.client.1.smithi148.stdout:7/184: mkdir d5/d14/d35 0 2024-09-21T04:15:10.865 INFO:tasks.workunit.client.0.smithi110.stdout:6/202: dread d4/d12/d17/d2e/d40/f2f [0,4194304] 0 2024-09-21T04:15:10.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/203: readlink d4/l32 0 2024-09-21T04:15:10.867 INFO:tasks.workunit.client.1.smithi148.stdout:6/122: rename d0/c17 to d0/d8/d1b/dc/c1d 0 2024-09-21T04:15:10.867 INFO:tasks.workunit.client.1.smithi148.stdout:6/123: write d0/d8/d1b/f19 [578283,127113] 0 2024-09-21T04:15:10.868 INFO:tasks.workunit.client.0.smithi110.stdout:4/339: truncate d17/d1e/d21/f2e 1396820 0 2024-09-21T04:15:10.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/204: creat d4/d12/d17/d33/d3e/f42 x:0 0 0 2024-09-21T04:15:10.871 INFO:tasks.workunit.client.0.smithi110.stdout:1/465: rename d2/d9/d49/d4f/f6a to d2/d9/d49/f99 0 2024-09-21T04:15:10.872 INFO:tasks.workunit.client.0.smithi110.stdout:1/466: symlink d2/d9/d26/d3d/l9a 0 2024-09-21T04:15:10.873 INFO:tasks.workunit.client.0.smithi110.stdout:1/467: symlink d2/d9/d26/d3d/d8f/l9b 0 2024-09-21T04:15:10.880 INFO:tasks.workunit.client.0.smithi110.stdout:6/205: dread d4/de/ff [0,4194304] 0 2024-09-21T04:15:10.881 INFO:tasks.workunit.client.0.smithi110.stdout:1/468: dread d2/d9/f1f [0,4194304] 0 2024-09-21T04:15:10.883 INFO:tasks.workunit.client.0.smithi110.stdout:4/340: dread d17/d34/f37 [0,4194304] 0 2024-09-21T04:15:10.883 INFO:tasks.workunit.client.0.smithi110.stdout:1/469: mkdir d2/d9/d23/d3b/d9c 0 2024-09-21T04:15:10.883 INFO:tasks.workunit.client.0.smithi110.stdout:4/341: chown d17/d34/l42 169401998 1 2024-09-21T04:15:10.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/470: creat d2/d9/d26/d3d/f9d x:0 0 0 2024-09-21T04:15:10.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/471: chown d2/d9/d49/d5c/f8a 72885897 1 2024-09-21T04:15:10.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/472: chown d2/d9/d26/c76 11 1 2024-09-21T04:15:10.887 INFO:tasks.workunit.client.0.smithi110.stdout:1/473: unlink d2/d9/d26/d3d/f9d 0 2024-09-21T04:15:10.888 INFO:tasks.workunit.client.0.smithi110.stdout:1/474: creat d2/d9/d23/d3b/d93/f9e x:0 0 0 2024-09-21T04:15:10.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/206: dread d4/fa [4194304,4194304] 0 2024-09-21T04:15:10.889 INFO:tasks.workunit.client.0.smithi110.stdout:6/207: symlink d4/d12/d17/d33/l43 0 2024-09-21T04:15:10.889 INFO:tasks.workunit.client.0.smithi110.stdout:6/208: creat d4/de/f44 x:0 0 0 2024-09-21T04:15:10.889 INFO:tasks.workunit.client.0.smithi110.stdout:6/209: fdatasync d4/fb 0 2024-09-21T04:15:10.892 INFO:tasks.workunit.client.0.smithi110.stdout:6/210: link d4/d12/d17/d33/l39 d4/d12/d17/d2e/d40/l45 0 2024-09-21T04:15:10.893 INFO:tasks.workunit.client.0.smithi110.stdout:6/211: symlink d4/l46 0 2024-09-21T04:15:10.894 INFO:tasks.workunit.client.1.smithi148.stdout:5/156: dwrite d3/d14/d1a/f31 [0,4194304] 0 2024-09-21T04:15:10.894 INFO:tasks.workunit.client.1.smithi148.stdout:5/157: stat d3/d14/d1f/d26/f32 0 2024-09-21T04:15:10.895 INFO:tasks.workunit.client.0.smithi110.stdout:6/212: creat d4/d12/d17/d2e/f47 x:0 0 0 2024-09-21T04:15:10.895 INFO:tasks.workunit.client.0.smithi110.stdout:6/213: read d4/d12/f1d [2110349,72171] 0 2024-09-21T04:15:10.896 INFO:tasks.workunit.client.1.smithi148.stdout:8/136: dwrite dc/f1d [0,4194304] 0 2024-09-21T04:15:10.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/158: write d3/dc/f19 [1048935,44216] 0 2024-09-21T04:15:10.897 INFO:tasks.workunit.client.0.smithi110.stdout:0/427: dwrite d4/d1c/d62/f36 [0,4194304] 0 2024-09-21T04:15:10.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/342: dread d17/d34/f65 [0,4194304] 0 2024-09-21T04:15:10.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/343: chown d17/f50 284706 1 2024-09-21T04:15:10.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/344: fdatasync f4 0 2024-09-21T04:15:10.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/345: chown c9 30 1 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/346: write d17/d34/d3d/d20/f49 [6139939,71346] 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/250: dwrite d1/d39/f45 [0,4194304] 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/251: creat d1/dc/f57 x:0 0 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/252: dread - d1/d39/f1c zero size 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/253: write d1/f3e [369067,70681] 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/347: write d17/f27 [5797534,4809] 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/348: getdents d17/d1e/d6a 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/349: fdatasync d17/d1e/d21/f54 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/350: write f5 [3008419,122422] 0 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/351: chown d17/d34/c46 24825639 1 2024-09-21T04:15:10.898 INFO:tasks.workunit.client.0.smithi110.stdout:4/352: creat d17/f7c x:0 0 0 2024-09-21T04:15:10.899 INFO:tasks.workunit.client.1.smithi148.stdout:3/120: dwrite f1 [0,4194304] 0 2024-09-21T04:15:10.899 INFO:tasks.workunit.client.1.smithi148.stdout:4/155: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:10.902 INFO:tasks.workunit.client.1.smithi148.stdout:7/185: dread d5/f13 [0,4194304] 0 2024-09-21T04:15:10.902 INFO:tasks.workunit.client.1.smithi148.stdout:7/186: chown d5/dc/d24/c2d 73257 1 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/428: symlink d4/dd/d10/d29/d75/l80 0 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/429: readlink d4/dd/d10/d29/d75/l80 0 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/430: chown d4/dd/d10/d29/d75/c78 218 1 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.0.smithi110.stdout:4/353: read d17/d34/f37 [1422031,106224] 0 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.1.smithi148.stdout:8/137: mknod dc/d13/d28/c29 0 2024-09-21T04:15:10.903 INFO:tasks.workunit.client.0.smithi110.stdout:1/475: dwrite d2/d9/d26/f8b [0,4194304] 0 2024-09-21T04:15:10.908 INFO:tasks.workunit.client.1.smithi148.stdout:5/159: mknod d3/d34/c39 0 2024-09-21T04:15:10.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/124: getdents d0/d8/d1b/dc 0 2024-09-21T04:15:10.909 INFO:tasks.workunit.client.1.smithi148.stdout:3/121: read f1 [686373,98923] 0 2024-09-21T04:15:10.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:10 smithi110 ceph-mon[27723]: pgmap v270: 65 pgs: 65 active+clean; 549 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 36 MiB/s rd, 47 MiB/s wr, 642 op/s 2024-09-21T04:15:10.918 INFO:tasks.workunit.client.0.smithi110.stdout:0/431: creat d4/dd/d53/d66/f81 x:0 0 0 2024-09-21T04:15:10.918 INFO:tasks.workunit.client.0.smithi110.stdout:0/432: chown d4/dd/d53/f5a 242942825 1 2024-09-21T04:15:10.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/160: write d3/d14/d1a/f31 [2066749,16580] 0 2024-09-21T04:15:10.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/161: fdatasync d3/d14/f29 0 2024-09-21T04:15:10.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/162: write d3/d24/f2f [508716,84832] 0 2024-09-21T04:15:10.920 INFO:tasks.workunit.client.1.smithi148.stdout:4/156: rename d4/f9 to d4/d23/d2a/f2f 0 2024-09-21T04:15:10.920 INFO:tasks.workunit.client.1.smithi148.stdout:4/157: truncate d4/d12/f1b 1359744 0 2024-09-21T04:15:10.920 INFO:tasks.workunit.client.1.smithi148.stdout:4/158: chown d4/fb 1707712 1 2024-09-21T04:15:10.921 INFO:tasks.workunit.client.1.smithi148.stdout:8/138: mkdir dc/d2a 0 2024-09-21T04:15:10.923 INFO:tasks.workunit.client.1.smithi148.stdout:3/122: mkdir d3/d1f 0 2024-09-21T04:15:10.923 INFO:tasks.workunit.client.0.smithi110.stdout:3/243: sync 2024-09-21T04:15:10.926 INFO:tasks.workunit.client.1.smithi148.stdout:5/163: rmdir d3/d14/d1f 39 2024-09-21T04:15:10.928 INFO:tasks.workunit.client.1.smithi148.stdout:7/187: rename d5/d8/c26 to d5/d14/d35/c36 0 2024-09-21T04:15:10.928 INFO:tasks.workunit.client.1.smithi148.stdout:7/188: stat d5/d14 0 2024-09-21T04:15:10.928 INFO:tasks.workunit.client.0.smithi110.stdout:4/354: dwrite d17/d34/d3d/d20/d43/d51/d52/f59 [0,4194304] 0 2024-09-21T04:15:10.928 INFO:tasks.workunit.client.0.smithi110.stdout:1/476: dwrite d2/d9/d26/f8b [0,4194304] 0 2024-09-21T04:15:10.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/254: mkdir d1/d7/d58 0 2024-09-21T04:15:10.929 INFO:tasks.workunit.client.1.smithi148.stdout:4/159: rmdir d4/d12/d2c 39 2024-09-21T04:15:10.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/255: read d1/d49/d4d/d1e/f3b [502584,63920] 0 2024-09-21T04:15:10.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/256: readlink d1/d49/l52 0 2024-09-21T04:15:10.930 INFO:tasks.workunit.client.0.smithi110.stdout:0/433: mknod d4/dd/d10/d29/d5f/c82 0 2024-09-21T04:15:10.931 INFO:tasks.workunit.client.0.smithi110.stdout:0/434: read d4/f21 [694117,68076] 0 2024-09-21T04:15:10.931 INFO:tasks.workunit.client.0.smithi110.stdout:0/435: readlink d4/d1c/l6d 0 2024-09-21T04:15:10.931 INFO:tasks.workunit.client.0.smithi110.stdout:0/436: fdatasync d4/d47/d2c/d3a/f51 0 2024-09-21T04:15:10.931 INFO:tasks.workunit.client.1.smithi148.stdout:8/139: getdents dc/d13/d28 0 2024-09-21T04:15:10.933 INFO:tasks.workunit.client.0.smithi110.stdout:3/244: symlink d0/d22/l54 0 2024-09-21T04:15:10.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/123: creat d3/d13/f20 x:0 0 0 2024-09-21T04:15:10.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/124: write d3/d5/d6/f17 [1016969,992] 0 2024-09-21T04:15:10.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/125: stat d3/d13/c16 0 2024-09-21T04:15:10.937 INFO:tasks.workunit.client.1.smithi148.stdout:5/164: symlink d3/d14/l3a 0 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/165: sync 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:2/221: sync 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/166: creat d5/d22/f45 x:0 0 0 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/167: fdatasync d5/d1e/f2d 0 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/168: read d5/d1e/f2d [496984,70917] 0 2024-09-21T04:15:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:9/169: write d5/d1e/d37/f3b [4372063,88515] 0 2024-09-21T04:15:10.940 INFO:tasks.workunit.client.0.smithi110.stdout:1/477: rmdir d2/d9/d49/d7a 39 2024-09-21T04:15:10.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:10 smithi148 ceph-mon[34957]: pgmap v270: 65 pgs: 65 active+clean; 549 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 36 MiB/s rd, 47 MiB/s wr, 642 op/s 2024-09-21T04:15:10.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/189: truncate d5/f6 2382884 0 2024-09-21T04:15:10.941 INFO:tasks.workunit.client.0.smithi110.stdout:5/257: truncate d1/d39/f3d 3762401 0 2024-09-21T04:15:10.942 INFO:tasks.workunit.client.0.smithi110.stdout:4/355: dwrite d17/d34/d3d/d57/f5a [0,4194304] 0 2024-09-21T04:15:10.943 INFO:tasks.workunit.client.0.smithi110.stdout:0/437: link d4/dd/d10/d29/d75/l80 d4/d47/d2c/d3a/l83 0 2024-09-21T04:15:10.943 INFO:tasks.workunit.client.0.smithi110.stdout:0/438: write d4/dd/d53/f79 [652152,81477] 0 2024-09-21T04:15:10.946 INFO:tasks.workunit.client.0.smithi110.stdout:3/245: rename d0/d12/d3c/c40 to d0/d22/c55 0 2024-09-21T04:15:10.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/258: write d1/d39/f2d [1125692,8600] 0 2024-09-21T04:15:10.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/259: dread - d1/d22/f4f zero size 2024-09-21T04:15:10.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/260: getdents d1/d49/d4d/d1e/d4a 0 2024-09-21T04:15:10.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/261: write d1/d22/f4f [225535,2577] 0 2024-09-21T04:15:10.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/262: readlink d1/d39/l42 0 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:2/136: sync 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:8/261: sync 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:2/137: readlink d2/d1a/d1f/l10 0 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:8/262: truncate d0/d7/f20 5135316 0 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:8/263: dread - d0/d7/f36 zero size 2024-09-21T04:15:10.949 INFO:tasks.workunit.client.0.smithi110.stdout:2/138: write d2/d1a/de/f21 [1037842,71975] 0 2024-09-21T04:15:10.950 INFO:tasks.workunit.client.1.smithi148.stdout:3/126: mkdir d3/d5/d6/d12/da/d21 0 2024-09-21T04:15:10.950 INFO:tasks.workunit.client.0.smithi110.stdout:7/218: sync 2024-09-21T04:15:10.951 INFO:tasks.workunit.client.1.smithi148.stdout:2/222: fsync d0/d9/f29 0 2024-09-21T04:15:10.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/140: dwrite dc/f14 [0,4194304] 0 2024-09-21T04:15:10.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/127: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:10.952 INFO:tasks.workunit.client.1.smithi148.stdout:4/160: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:10.953 INFO:tasks.workunit.client.0.smithi110.stdout:0/439: mkdir d4/d47/d2c/d3a/d84 0 2024-09-21T04:15:10.954 INFO:tasks.workunit.client.0.smithi110.stdout:0/440: read d4/f3b [3161884,61039] 0 2024-09-21T04:15:10.954 INFO:tasks.workunit.client.0.smithi110.stdout:0/441: read - d4/dd/d10/d29/f5d zero size 2024-09-21T04:15:10.955 INFO:tasks.workunit.client.0.smithi110.stdout:4/356: rename d17/d34/f74 to d17/d3e/d6b/f7d 0 2024-09-21T04:15:10.956 INFO:tasks.workunit.client.1.smithi148.stdout:9/170: mkdir d5/d1e/d37/d46 0 2024-09-21T04:15:10.957 INFO:tasks.workunit.client.0.smithi110.stdout:5/263: symlink d1/d49/d4d/d1e/d4a/l59 0 2024-09-21T04:15:10.958 INFO:tasks.workunit.client.1.smithi148.stdout:7/190: truncate d5/f13 872421 0 2024-09-21T04:15:10.958 INFO:tasks.workunit.client.0.smithi110.stdout:5/264: dread d1/f3e [0,4194304] 0 2024-09-21T04:15:10.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/478: link d2/d9/d23/c39 d2/d9/d23/d3b/d9c/c9f 0 2024-09-21T04:15:10.959 INFO:tasks.workunit.client.1.smithi148.stdout:2/223: creat d0/d9/f47 x:0 0 0 2024-09-21T04:15:10.963 INFO:tasks.workunit.client.1.smithi148.stdout:7/191: write f2 [2019321,24634] 0 2024-09-21T04:15:10.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/128: mknod d3/d5/d6/d12/da/d19/c22 0 2024-09-21T04:15:10.968 INFO:tasks.workunit.client.1.smithi148.stdout:2/224: rename d0/d9/d40 to d0/d2b/d48 0 2024-09-21T04:15:10.969 INFO:tasks.workunit.client.0.smithi110.stdout:5/265: mkdir d1/d49/d4d/d1e/d5a 0 2024-09-21T04:15:10.969 INFO:tasks.workunit.client.0.smithi110.stdout:5/266: fsync d1/d7/f4e 0 2024-09-21T04:15:10.970 INFO:tasks.workunit.client.0.smithi110.stdout:1/479: rmdir d2/d9/d49 39 2024-09-21T04:15:10.970 INFO:tasks.workunit.client.1.smithi148.stdout:8/141: dread dc/f17 [0,4194304] 0 2024-09-21T04:15:10.971 INFO:tasks.workunit.client.1.smithi148.stdout:4/161: getdents d4/d23 0 2024-09-21T04:15:10.973 INFO:tasks.workunit.client.0.smithi110.stdout:0/442: link d4/dd/d53/d66/f74 d4/f85 0 2024-09-21T04:15:10.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/225: symlink d0/d9/dc/d13/d2c/l49 0 2024-09-21T04:15:10.976 INFO:tasks.workunit.client.0.smithi110.stdout:5/267: mknod d1/d49/d4d/d1e/c5b 0 2024-09-21T04:15:10.976 INFO:tasks.workunit.client.0.smithi110.stdout:5/268: rename d1/d7/d18 to d1/d7/d18/d5c 22 2024-09-21T04:15:10.976 INFO:tasks.workunit.client.0.smithi110.stdout:5/269: read - d1/dc/f33 zero size 2024-09-21T04:15:10.978 INFO:tasks.workunit.client.0.smithi110.stdout:0/443: symlink d4/d1c/d62/l86 0 2024-09-21T04:15:10.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/270: symlink d1/d49/d4d/l5d 0 2024-09-21T04:15:10.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/271: write d1/dc/f57 [8040,83820] 0 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.0.smithi110.stdout:5/272: chown d1/d49/d4d/d1e 3280046 1 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/142: creat dc/d13/d28/f2b x:0 0 0 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/143: chown dc/d13/d28/f2b 54531 1 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/144: chown dc/f17 1361887 1 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/145: read - dc/d13/d28/f2b zero size 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/146: fdatasync fb 0 2024-09-21T04:15:10.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/147: dread - dc/d13/f1c zero size 2024-09-21T04:15:10.983 INFO:tasks.workunit.client.1.smithi148.stdout:4/162: mknod d4/d12/d16/c30 0 2024-09-21T04:15:10.985 INFO:tasks.workunit.client.1.smithi148.stdout:2/226: unlink d0/d2f/f35 0 2024-09-21T04:15:10.985 INFO:tasks.workunit.client.1.smithi148.stdout:2/227: creat d0/d2f/f4a x:0 0 0 2024-09-21T04:15:10.985 INFO:tasks.workunit.client.1.smithi148.stdout:2/228: creat d0/d2b/f4b x:0 0 0 2024-09-21T04:15:10.988 INFO:tasks.workunit.client.0.smithi110.stdout:5/273: mknod d1/d7/d18/d3f/c5e 0 2024-09-21T04:15:10.990 INFO:tasks.workunit.client.1.smithi148.stdout:4/163: rmdir d4/d23/d2a 39 2024-09-21T04:15:10.991 INFO:tasks.workunit.client.0.smithi110.stdout:0/444: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:10.992 INFO:tasks.workunit.client.1.smithi148.stdout:8/148: dread dc/f14 [0,4194304] 0 2024-09-21T04:15:10.993 INFO:tasks.workunit.client.0.smithi110.stdout:0/445: symlink d4/d47/l87 0 2024-09-21T04:15:10.994 INFO:tasks.workunit.client.1.smithi148.stdout:2/229: mkdir d0/d9/dc/d13/d4c 0 2024-09-21T04:15:10.995 INFO:tasks.workunit.client.0.smithi110.stdout:0/446: dread d4/d47/d2c/d3a/f43 [0,4194304] 0 2024-09-21T04:15:10.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/129: rename d3/d13 to d3/d5/d6/d12/da/d23 0 2024-09-21T04:15:10.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/130: chown d3/d5/cd 100 1 2024-09-21T04:15:10.996 INFO:tasks.workunit.client.0.smithi110.stdout:0/447: creat d4/d47/d2c/f88 x:0 0 0 2024-09-21T04:15:10.996 INFO:tasks.workunit.client.0.smithi110.stdout:0/448: truncate d4/dd/d53/f79 1703178 0 2024-09-21T04:15:10.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/131: mknod d3/d5/d6/d12/da/d19/c24 0 2024-09-21T04:15:10.999 INFO:tasks.workunit.client.0.smithi110.stdout:0/449: dread d4/dd/d53/f5a [0,4194304] 0 2024-09-21T04:15:10.999 INFO:tasks.workunit.client.0.smithi110.stdout:0/450: fsync d4/dd/d53/f79 0 2024-09-21T04:15:10.999 INFO:tasks.workunit.client.0.smithi110.stdout:0/451: write d4/d1c/f3c [589837,34623] 0 2024-09-21T04:15:10.999 INFO:tasks.workunit.client.1.smithi148.stdout:4/164: rename d4/d12/c27 to d4/d12/d2c/c31 0 2024-09-21T04:15:11.000 INFO:tasks.workunit.client.0.smithi110.stdout:0/452: symlink d4/dd/l89 0 2024-09-21T04:15:11.000 INFO:tasks.workunit.client.0.smithi110.stdout:0/453: fdatasync d4/f85 0 2024-09-21T04:15:11.000 INFO:tasks.workunit.client.0.smithi110.stdout:0/454: fsync d4/dd/d10/f50 0 2024-09-21T04:15:11.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/149: rename dc/f1d to dc/d2a/f2c 0 2024-09-21T04:15:11.001 INFO:tasks.workunit.client.1.smithi148.stdout:8/150: readlink dc/d13/l20 0 2024-09-21T04:15:11.002 INFO:tasks.workunit.client.0.smithi110.stdout:0/455: truncate d4/d1c/f25 4193445 0 2024-09-21T04:15:11.004 INFO:tasks.workunit.client.0.smithi110.stdout:0/456: mkdir d4/dd/d10/d8a 0 2024-09-21T04:15:11.004 INFO:tasks.workunit.client.0.smithi110.stdout:0/457: write d4/f31 [4302566,68232] 0 2024-09-21T04:15:11.004 INFO:tasks.workunit.client.1.smithi148.stdout:2/230: rename d0/c20 to d0/d9/dc/d13/d25/c4d 0 2024-09-21T04:15:11.006 INFO:tasks.workunit.client.1.smithi148.stdout:4/165: rename d4/d12/f15 to d4/d12/d2c/f32 0 2024-09-21T04:15:11.007 INFO:tasks.workunit.client.1.smithi148.stdout:4/166: creat d4/d12/d16/d29/f33 x:0 0 0 2024-09-21T04:15:11.007 INFO:tasks.workunit.client.1.smithi148.stdout:4/167: fsync d4/d12/d16/f19 0 2024-09-21T04:15:11.009 INFO:tasks.workunit.client.1.smithi148.stdout:4/168: creat d4/d12/d16/f34 x:0 0 0 2024-09-21T04:15:11.010 INFO:tasks.workunit.client.1.smithi148.stdout:4/169: creat d4/d12/d16/d26/f35 x:0 0 0 2024-09-21T04:15:11.012 INFO:tasks.workunit.client.0.smithi110.stdout:2/139: dwrite d2/d3/fd [0,4194304] 0 2024-09-21T04:15:11.012 INFO:tasks.workunit.client.1.smithi148.stdout:4/170: read d4/f11 [364190,5570] 0 2024-09-21T04:15:11.014 INFO:tasks.workunit.client.0.smithi110.stdout:2/140: dread d2/d1a/de/f21 [0,4194304] 0 2024-09-21T04:15:11.016 INFO:tasks.workunit.client.0.smithi110.stdout:0/458: write d4/dd/d10/f2a [1109457,113692] 0 2024-09-21T04:15:11.017 INFO:tasks.workunit.client.0.smithi110.stdout:2/141: rename d2/d3/f12 to d2/d1a/de/d18/f24 0 2024-09-21T04:15:11.021 INFO:tasks.workunit.client.0.smithi110.stdout:0/459: rmdir d4/dd/d60 39 2024-09-21T04:15:11.021 INFO:tasks.workunit.client.0.smithi110.stdout:0/460: dread - d4/f5c zero size 2024-09-21T04:15:11.021 INFO:tasks.workunit.client.0.smithi110.stdout:0/461: dread - d4/d47/d2c/d3a/f6c zero size 2024-09-21T04:15:11.023 INFO:tasks.workunit.client.0.smithi110.stdout:0/462: mkdir d4/d8b 0 2024-09-21T04:15:11.024 INFO:tasks.workunit.client.1.smithi148.stdout:7/192: dwrite d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:11.024 INFO:tasks.workunit.client.1.smithi148.stdout:9/171: dwrite d5/d22/d28/f41 [0,4194304] 0 2024-09-21T04:15:11.024 INFO:tasks.workunit.client.0.smithi110.stdout:1/480: dwrite d2/d9/d23/d4e/f97 [0,4194304] 0 2024-09-21T04:15:11.025 INFO:tasks.workunit.client.0.smithi110.stdout:7/219: dwrite dd/d1c/d39/f3b [0,4194304] 0 2024-09-21T04:15:11.025 INFO:tasks.workunit.client.0.smithi110.stdout:7/220: dread - dd/d14/f36 zero size 2024-09-21T04:15:11.029 INFO:tasks.workunit.client.1.smithi148.stdout:7/193: symlink d5/d8/l37 0 2024-09-21T04:15:11.029 INFO:tasks.workunit.client.1.smithi148.stdout:7/194: chown d5/c27 11593378 1 2024-09-21T04:15:11.030 INFO:tasks.workunit.client.1.smithi148.stdout:3/132: dwrite d3/d5/d6/d12/da/d23/f20 [0,4194304] 0 2024-09-21T04:15:11.032 INFO:tasks.workunit.client.1.smithi148.stdout:8/151: dwrite f2 [0,4194304] 0 2024-09-21T04:15:11.032 INFO:tasks.workunit.client.1.smithi148.stdout:4/171: dwrite d4/f8 [0,4194304] 0 2024-09-21T04:15:11.035 INFO:tasks.workunit.client.1.smithi148.stdout:2/231: dwrite d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:11.035 INFO:tasks.workunit.client.1.smithi148.stdout:2/232: chown d0/d9/f29 0 1 2024-09-21T04:15:11.036 INFO:tasks.workunit.client.1.smithi148.stdout:7/195: symlink d5/dc/d24/l38 0 2024-09-21T04:15:11.041 INFO:tasks.workunit.client.0.smithi110.stdout:0/463: dwrite d4/dd/d53/f5a [0,4194304] 0 2024-09-21T04:15:11.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/133: mknod d3/c25 0 2024-09-21T04:15:11.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/134: write d3/d5/d6/d12/da/f1d [796081,100765] 0 2024-09-21T04:15:11.044 INFO:tasks.workunit.client.0.smithi110.stdout:1/481: rmdir d2/d9/d23/d4e 39 2024-09-21T04:15:11.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/135: dread d3/d5/d6/f17 [0,4194304] 0 2024-09-21T04:15:11.048 INFO:tasks.workunit.client.1.smithi148.stdout:8/152: getdents dc 0 2024-09-21T04:15:11.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/172: creat d4/d23/d2a/f36 x:0 0 0 2024-09-21T04:15:11.051 INFO:tasks.workunit.client.0.smithi110.stdout:1/482: rename d2/l4b to d2/d9/d26/d3d/la0 0 2024-09-21T04:15:11.052 INFO:tasks.workunit.client.1.smithi148.stdout:9/172: rmdir d5/d22/d28/d38 39 2024-09-21T04:15:11.053 INFO:tasks.workunit.client.0.smithi110.stdout:1/483: mknod d2/d9/d23/d3b/d93/ca1 0 2024-09-21T04:15:11.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/233: truncate d0/d9/f17 2610857 0 2024-09-21T04:15:11.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/234: write d0/d2f/f4a [872142,124020] 0 2024-09-21T04:15:11.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/235: stat d0/d9/d16/c22 0 2024-09-21T04:15:11.056 INFO:tasks.workunit.client.1.smithi148.stdout:1/149: sync 2024-09-21T04:15:11.058 INFO:tasks.workunit.client.0.smithi110.stdout:1/484: rename d2/f4 to d2/d9/d23/fa2 0 2024-09-21T04:15:11.058 INFO:tasks.workunit.client.0.smithi110.stdout:1/485: write d2/f94 [867533,42412] 0 2024-09-21T04:15:11.058 INFO:tasks.workunit.client.0.smithi110.stdout:1/486: creat d2/d9/d26/d3d/d8f/fa3 x:0 0 0 2024-09-21T04:15:11.060 INFO:tasks.workunit.client.1.smithi148.stdout:3/136: getdents d3/d5/d6/d12 0 2024-09-21T04:15:11.060 INFO:tasks.workunit.client.1.smithi148.stdout:3/137: fsync d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:11.060 INFO:tasks.workunit.client.1.smithi148.stdout:3/138: chown d3/cc 5791 1 2024-09-21T04:15:11.062 INFO:tasks.workunit.client.1.smithi148.stdout:8/153: link dc/l23 dc/l2d 0 2024-09-21T04:15:11.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/125: sync 2024-09-21T04:15:11.063 INFO:tasks.workunit.client.1.smithi148.stdout:0/151: sync 2024-09-21T04:15:11.063 INFO:tasks.workunit.client.1.smithi148.stdout:4/173: read d4/f10 [556597,6535] 0 2024-09-21T04:15:11.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/464: dwrite d4/d1c/d62/f36 [0,4194304] 0 2024-09-21T04:15:11.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/465: write d4/dd/d60/f7e [636394,45895] 0 2024-09-21T04:15:11.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/466: read - d4/f5c zero size 2024-09-21T04:15:11.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/467: chown d4/dd/d53/f79 497434 1 2024-09-21T04:15:11.065 INFO:tasks.workunit.client.1.smithi148.stdout:2/236: chown d0/d9/l36 51431 1 2024-09-21T04:15:11.067 INFO:tasks.workunit.client.1.smithi148.stdout:7/196: rmdir d5/d8 39 2024-09-21T04:15:11.068 INFO:tasks.workunit.client.1.smithi148.stdout:4/174: dread d4/d12/d2c/f32 [0,4194304] 0 2024-09-21T04:15:11.069 INFO:tasks.workunit.client.0.smithi110.stdout:0/468: mknod d4/dd/d10/d8a/c8c 0 2024-09-21T04:15:11.069 INFO:tasks.workunit.client.0.smithi110.stdout:0/469: getdents d4/d8b 0 2024-09-21T04:15:11.069 INFO:tasks.workunit.client.0.smithi110.stdout:0/470: creat d4/d1c/f8d x:0 0 0 2024-09-21T04:15:11.070 INFO:tasks.workunit.client.0.smithi110.stdout:9/147: sync 2024-09-21T04:15:11.071 INFO:tasks.workunit.client.1.smithi148.stdout:3/139: creat d3/d5/d6/d12/da/d23/f26 x:0 0 0 2024-09-21T04:15:11.072 INFO:tasks.workunit.client.0.smithi110.stdout:0/471: dread d4/dd/f32 [0,4194304] 0 2024-09-21T04:15:11.072 INFO:tasks.workunit.client.0.smithi110.stdout:1/487: dread d2/d9/d23/f38 [0,4194304] 0 2024-09-21T04:15:11.072 INFO:tasks.workunit.client.1.smithi148.stdout:8/154: creat dc/d2a/f2e x:0 0 0 2024-09-21T04:15:11.075 INFO:tasks.workunit.client.1.smithi148.stdout:6/126: rename d0/d8/d1b/c13 to d0/d8/d1b/d16/c1e 0 2024-09-21T04:15:11.076 INFO:tasks.workunit.client.0.smithi110.stdout:9/148: rmdir d7 39 2024-09-21T04:15:11.077 INFO:tasks.workunit.client.1.smithi148.stdout:2/237: mkdir d0/d9/dc/d13/d25/d4e 0 2024-09-21T04:15:11.077 INFO:tasks.workunit.client.1.smithi148.stdout:2/238: stat d0/d9/d16/c3d 0 2024-09-21T04:15:11.077 INFO:tasks.workunit.client.1.smithi148.stdout:2/239: truncate d0/d9/dc/d13/f3b 539656 0 2024-09-21T04:15:11.079 INFO:tasks.workunit.client.0.smithi110.stdout:0/472: creat d4/dd/f8e x:0 0 0 2024-09-21T04:15:11.079 INFO:tasks.workunit.client.1.smithi148.stdout:7/197: symlink d5/dc/d24/l39 0 2024-09-21T04:15:11.080 INFO:tasks.workunit.client.1.smithi148.stdout:9/173: dwrite d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:11.081 INFO:tasks.workunit.client.1.smithi148.stdout:9/174: chown d5/d39/l3a 976818 1 2024-09-21T04:15:11.081 INFO:tasks.workunit.client.1.smithi148.stdout:9/175: fsync f1 0 2024-09-21T04:15:11.081 INFO:tasks.workunit.client.1.smithi148.stdout:1/150: dwrite db/f1b [0,4194304] 0 2024-09-21T04:15:11.081 INFO:tasks.workunit.client.0.smithi110.stdout:1/488: creat d2/d9/d23/d3b/d9c/fa4 x:0 0 0 2024-09-21T04:15:11.081 INFO:tasks.workunit.client.0.smithi110.stdout:1/489: creat d2/d9/d49/d5c/d91/fa5 x:0 0 0 2024-09-21T04:15:11.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/175: dread d4/fb [0,4194304] 0 2024-09-21T04:15:11.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/176: chown d4/d23/f2e 1170359 1 2024-09-21T04:15:11.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/177: truncate d4/f11 980722 0 2024-09-21T04:15:11.084 INFO:tasks.workunit.client.0.smithi110.stdout:9/149: symlink d7/d26/l30 0 2024-09-21T04:15:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/155: read dc/f17 [7190857,75847] 0 2024-09-21T04:15:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/156: creat dc/d13/d28/f2f x:0 0 0 2024-09-21T04:15:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/157: write f6 [4934026,71561] 0 2024-09-21T04:15:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:9/176: dread f0 [0,4194304] 0 2024-09-21T04:15:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:3/140: rename d3/l9 to d3/d5/d6/d12/d15/l27 0 2024-09-21T04:15:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:3/141: getdents d3/d5/d6/d12 0 2024-09-21T04:15:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:3/142: readlink d3/d5/d6/d12/da/l1b 0 2024-09-21T04:15:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:4/178: read d4/d12/d2c/f32 [358824,34989] 0 2024-09-21T04:15:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:4/179: chown d4/fe 988375148 1 2024-09-21T04:15:11.088 INFO:tasks.workunit.client.1.smithi148.stdout:4/180: write d4/d12/d16/f19 [560312,73084] 0 2024-09-21T04:15:11.088 INFO:tasks.workunit.client.1.smithi148.stdout:4/181: write d4/f11 [1511397,70240] 0 2024-09-21T04:15:11.088 INFO:tasks.workunit.client.1.smithi148.stdout:4/182: truncate d4/d23/f2b 357750 0 2024-09-21T04:15:11.089 INFO:tasks.workunit.client.1.smithi148.stdout:4/183: dread - d4/d23/d2a/f36 zero size 2024-09-21T04:15:11.089 INFO:tasks.workunit.client.1.smithi148.stdout:0/152: dwrite d5/d6/d7/d11/d19/f26 [0,4194304] 0 2024-09-21T04:15:11.089 INFO:tasks.workunit.client.1.smithi148.stdout:0/153: creat d5/d6/d7/d22/f37 x:0 0 0 2024-09-21T04:15:11.089 INFO:tasks.workunit.client.1.smithi148.stdout:0/154: stat d5/d6/d7/c1c 0 2024-09-21T04:15:11.090 INFO:tasks.workunit.client.1.smithi148.stdout:6/127: creat d0/d8/d1b/d16/f1f x:0 0 0 2024-09-21T04:15:11.090 INFO:tasks.workunit.client.1.smithi148.stdout:6/128: chown d0/d8 511775 1 2024-09-21T04:15:11.090 INFO:tasks.workunit.client.1.smithi148.stdout:6/129: chown d0/d8/d1b/d16 1441941134 1 2024-09-21T04:15:11.090 INFO:tasks.workunit.client.1.smithi148.stdout:6/130: write d0/d8/d1b/f14 [1536055,72666] 0 2024-09-21T04:15:11.091 INFO:tasks.workunit.client.0.smithi110.stdout:9/150: mkdir d7/db/d14/d31 0 2024-09-21T04:15:11.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/165: sync 2024-09-21T04:15:11.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/166: chown d3/d14/d1a/f33 43 1 2024-09-21T04:15:11.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/240: dread d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:11.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/241: read d0/f1d [1986236,11933] 0 2024-09-21T04:15:11.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/151: unlink d7/f8 0 2024-09-21T04:15:11.095 INFO:tasks.workunit.client.1.smithi148.stdout:1/151: creat db/f37 x:0 0 0 2024-09-21T04:15:11.096 INFO:tasks.workunit.client.1.smithi148.stdout:1/152: fdatasync db/f15 0 2024-09-21T04:15:11.096 INFO:tasks.workunit.client.0.smithi110.stdout:9/152: creat d7/d10/d24/f32 x:0 0 0 2024-09-21T04:15:11.096 INFO:tasks.workunit.client.0.smithi110.stdout:9/153: write d7/d10/d24/f2a [549787,64447] 0 2024-09-21T04:15:11.097 INFO:tasks.workunit.client.1.smithi148.stdout:8/158: symlink dc/l30 0 2024-09-21T04:15:11.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/154: mkdir d7/db/d11/d33 0 2024-09-21T04:15:11.098 INFO:tasks.workunit.client.1.smithi148.stdout:9/177: creat d5/d22/d28/d38/f47 x:0 0 0 2024-09-21T04:15:11.100 INFO:tasks.workunit.client.1.smithi148.stdout:0/155: write d5/d6/d7/d22/f24 [610664,43660] 0 2024-09-21T04:15:11.101 INFO:tasks.workunit.client.1.smithi148.stdout:7/198: rename d5/f13 to d5/dc/d28/f3a 0 2024-09-21T04:15:11.101 INFO:tasks.workunit.client.1.smithi148.stdout:3/143: mkdir d3/d5/d6/d12/da/d23/d28 0 2024-09-21T04:15:11.104 INFO:tasks.workunit.client.1.smithi148.stdout:4/184: link d4/d23/d2a/f2f d4/d23/f37 0 2024-09-21T04:15:11.104 INFO:tasks.workunit.client.1.smithi148.stdout:4/185: stat d4/c5 0 2024-09-21T04:15:11.107 INFO:tasks.workunit.client.1.smithi148.stdout:5/167: symlink d3/d24/l3b 0 2024-09-21T04:15:11.108 INFO:tasks.workunit.client.1.smithi148.stdout:1/153: truncate db/dd/f12 1264430 0 2024-09-21T04:15:11.108 INFO:tasks.workunit.client.1.smithi148.stdout:1/154: truncate db/d24/f2b 492815 0 2024-09-21T04:15:11.110 INFO:tasks.workunit.client.1.smithi148.stdout:9/178: symlink d5/d22/d28/d38/l48 0 2024-09-21T04:15:11.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/156: dread d5/d6/d7/d11/f17 [4194304,4194304] 0 2024-09-21T04:15:11.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/242: rename d0/f1f to d0/d9/d38/f4f 0 2024-09-21T04:15:11.115 INFO:tasks.workunit.client.1.smithi148.stdout:3/144: symlink d3/d5/d6/d12/da/d19/l29 0 2024-09-21T04:15:11.116 INFO:tasks.workunit.client.0.smithi110.stdout:9/155: fdatasync d7/d10/d24/f2a 0 2024-09-21T04:15:11.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/473: dwrite d4/d1c/d62/d69/f77 [0,4194304] 0 2024-09-21T04:15:11.117 INFO:tasks.workunit.client.1.smithi148.stdout:1/155: mknod db/dd/c38 0 2024-09-21T04:15:11.120 INFO:tasks.workunit.client.0.smithi110.stdout:0/474: dread d4/dd/d10/f50 [0,4194304] 0 2024-09-21T04:15:11.122 INFO:tasks.workunit.client.1.smithi148.stdout:9/179: write d5/fd [1280773,63229] 0 2024-09-21T04:15:11.122 INFO:tasks.workunit.client.1.smithi148.stdout:6/131: dwrite d0/d8/d1b/f14 [0,4194304] 0 2024-09-21T04:15:11.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/199: rename d5/dc/d24/c2d to d5/d14/d25/c3b 0 2024-09-21T04:15:11.125 INFO:tasks.workunit.client.1.smithi148.stdout:3/145: symlink d3/d5/d6/d12/da/d19/l2a 0 2024-09-21T04:15:11.126 INFO:tasks.workunit.client.0.smithi110.stdout:9/156: dread d7/d10/f13 [0,4194304] 0 2024-09-21T04:15:11.126 INFO:tasks.workunit.client.0.smithi110.stdout:9/157: chown l4 13 1 2024-09-21T04:15:11.127 INFO:tasks.workunit.client.0.smithi110.stdout:0/475: symlink d4/dd/d10/l8f 0 2024-09-21T04:15:11.127 INFO:tasks.workunit.client.0.smithi110.stdout:0/476: dread - d4/f5c zero size 2024-09-21T04:15:11.127 INFO:tasks.workunit.client.0.smithi110.stdout:1/490: dwrite d2/d9/f6c [0,4194304] 0 2024-09-21T04:15:11.131 INFO:tasks.workunit.client.1.smithi148.stdout:5/168: dwrite d3/f1b [0,4194304] 0 2024-09-21T04:15:11.131 INFO:tasks.workunit.client.0.smithi110.stdout:0/477: dread d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:11.132 INFO:tasks.workunit.client.1.smithi148.stdout:4/186: getdents d4/d12/d16 0 2024-09-21T04:15:11.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/180: link d5/d1e/d37/f3b d5/d1e/f49 0 2024-09-21T04:15:11.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/181: write d5/d1e/f2d [36532,6144] 0 2024-09-21T04:15:11.132 INFO:tasks.workunit.client.1.smithi148.stdout:6/132: creat d0/f20 x:0 0 0 2024-09-21T04:15:11.132 INFO:tasks.workunit.client.1.smithi148.stdout:6/133: creat d0/f21 x:0 0 0 2024-09-21T04:15:11.137 INFO:tasks.workunit.client.1.smithi148.stdout:3/146: mkdir d3/d5/d6/d2b 0 2024-09-21T04:15:11.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/182: mknod d5/d22/d28/d2f/c4a 0 2024-09-21T04:15:11.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/183: creat d5/d1f/f4b x:0 0 0 2024-09-21T04:15:11.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/184: readlink d5/d39/l3d 0 2024-09-21T04:15:11.141 INFO:tasks.workunit.client.0.smithi110.stdout:0/478: write d4/d47/d2c/d3a/f51 [2246018,51937] 0 2024-09-21T04:15:11.151 INFO:tasks.workunit.client.1.smithi148.stdout:9/185: mkdir d5/d22/d4c 0 2024-09-21T04:15:11.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/156: fsync db/dd/f12 0 2024-09-21T04:15:11.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/157: dread - db/f27 zero size 2024-09-21T04:15:11.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/158: chown l9 26161 1 2024-09-21T04:15:11.154 INFO:tasks.workunit.client.0.smithi110.stdout:0/479: rename d4/dd/d10/f41 to d4/d1c/d62/f90 0 2024-09-21T04:15:11.157 INFO:tasks.workunit.client.1.smithi148.stdout:9/186: creat d5/d1e/d37/f4d x:0 0 0 2024-09-21T04:15:11.157 INFO:tasks.workunit.client.1.smithi148.stdout:9/187: readlink d5/d22/d28/d38/l48 0 2024-09-21T04:15:11.157 INFO:tasks.workunit.client.1.smithi148.stdout:9/188: chown d5/f42 12 1 2024-09-21T04:15:11.157 INFO:tasks.workunit.client.1.smithi148.stdout:1/159: rename c2 to db/d24/c39 0 2024-09-21T04:15:11.158 INFO:tasks.workunit.client.1.smithi148.stdout:9/189: mkdir d5/d2b/d4e 0 2024-09-21T04:15:11.160 INFO:tasks.workunit.client.1.smithi148.stdout:1/160: rename db/l1d to db/d24/l3a 0 2024-09-21T04:15:11.161 INFO:tasks.workunit.client.1.smithi148.stdout:8/159: truncate f6 3124290 0 2024-09-21T04:15:11.161 INFO:tasks.workunit.client.1.smithi148.stdout:8/160: truncate f8 1128400 0 2024-09-21T04:15:11.162 INFO:tasks.workunit.client.1.smithi148.stdout:9/190: link d5/d1f/f24 d5/d2b/f4f 0 2024-09-21T04:15:11.164 INFO:tasks.workunit.client.1.smithi148.stdout:1/161: mkdir db/dd/d21/d3b 0 2024-09-21T04:15:11.165 INFO:tasks.workunit.client.1.smithi148.stdout:2/243: dwrite d0/d9/dc/d13/d25/f2a [0,4194304] 0 2024-09-21T04:15:11.165 INFO:tasks.workunit.client.1.smithi148.stdout:2/244: chown d0/d9/d16 72 1 2024-09-21T04:15:11.166 INFO:tasks.workunit.client.1.smithi148.stdout:8/161: creat dc/f31 x:0 0 0 2024-09-21T04:15:11.166 INFO:tasks.workunit.client.1.smithi148.stdout:8/162: readlink dc/l23 0 2024-09-21T04:15:11.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/162: rmdir db/d24/d2e 39 2024-09-21T04:15:11.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/163: chown db/dd/f23 46905 1 2024-09-21T04:15:11.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/157: dwrite f4 [8388608,4194304] 0 2024-09-21T04:15:11.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/169: dwrite d3/d14/d1f/f23 [0,4194304] 0 2024-09-21T04:15:11.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/170: stat d3/d14/f2d 0 2024-09-21T04:15:11.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/200: dwrite f2 [0,4194304] 0 2024-09-21T04:15:11.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/245: rmdir d0/d9/d38 39 2024-09-21T04:15:11.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/246: fdatasync d0/d2f/f4a 0 2024-09-21T04:15:11.173 INFO:tasks.workunit.client.1.smithi148.stdout:8/163: mknod dc/c32 0 2024-09-21T04:15:11.174 INFO:tasks.workunit.client.0.smithi110.stdout:9/158: dwrite d7/d10/d24/f32 [0,4194304] 0 2024-09-21T04:15:11.175 INFO:tasks.workunit.client.0.smithi110.stdout:9/159: fsync d7/d10/d24/f32 0 2024-09-21T04:15:11.179 INFO:tasks.workunit.client.1.smithi148.stdout:3/147: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:11.179 INFO:tasks.workunit.client.1.smithi148.stdout:3/148: rename d3/d5/d6 to d3/d5/d6/d2b/d2c 22 2024-09-21T04:15:11.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/171: truncate d3/f36 3920169 0 2024-09-21T04:15:11.182 INFO:tasks.workunit.client.0.smithi110.stdout:9/160: symlink d7/d10/l34 0 2024-09-21T04:15:11.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/201: creat d5/d14/f3c x:0 0 0 2024-09-21T04:15:11.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/247: mkdir d0/d9/dc/d13/d50 0 2024-09-21T04:15:11.187 INFO:tasks.workunit.client.0.smithi110.stdout:9/161: dread d7/d10/f13 [0,4194304] 0 2024-09-21T04:15:11.187 INFO:tasks.workunit.client.1.smithi148.stdout:8/164: dread fb [0,4194304] 0 2024-09-21T04:15:11.189 INFO:tasks.workunit.client.0.smithi110.stdout:0/480: dwrite d4/f5c [0,4194304] 0 2024-09-21T04:15:11.190 INFO:tasks.workunit.client.0.smithi110.stdout:1/491: dwrite d2/d9/d26/d3d/f58 [0,4194304] 0 2024-09-21T04:15:11.191 INFO:tasks.workunit.client.0.smithi110.stdout:9/162: mkdir d7/db/d35 0 2024-09-21T04:15:11.193 INFO:tasks.workunit.client.1.smithi148.stdout:1/164: dwrite db/d24/f2b [0,4194304] 0 2024-09-21T04:15:11.193 INFO:tasks.workunit.client.1.smithi148.stdout:1/165: truncate db/d24/f26 171999 0 2024-09-21T04:15:11.193 INFO:tasks.workunit.client.1.smithi148.stdout:6/134: dwrite d0/f21 [0,4194304] 0 2024-09-21T04:15:11.199 INFO:tasks.workunit.client.1.smithi148.stdout:9/191: fsync d5/d2b/f4f 0 2024-09-21T04:15:11.199 INFO:tasks.workunit.client.1.smithi148.stdout:9/192: readlink d5/l17 0 2024-09-21T04:15:11.202 INFO:tasks.workunit.client.0.smithi110.stdout:0/481: mkdir d4/d1c/d62/d91 0 2024-09-21T04:15:11.202 INFO:tasks.workunit.client.0.smithi110.stdout:0/482: chown d4/d1c/d62/l68 5 1 2024-09-21T04:15:11.204 INFO:tasks.workunit.client.0.smithi110.stdout:1/492: symlink d2/d9/d26/d3d/la6 0 2024-09-21T04:15:11.205 INFO:tasks.workunit.client.1.smithi148.stdout:0/158: getdents d5/d6/d7/d11 0 2024-09-21T04:15:11.205 INFO:tasks.workunit.client.1.smithi148.stdout:0/159: fdatasync d5/d6/d7/d11/d19/f1b 0 2024-09-21T04:15:11.205 INFO:tasks.workunit.client.1.smithi148.stdout:0/160: write d5/d6/f16 [23135,56668] 0 2024-09-21T04:15:11.205 INFO:tasks.workunit.client.1.smithi148.stdout:6/135: read d0/d8/d1b/f14 [3569622,96172] 0 2024-09-21T04:15:11.206 INFO:tasks.workunit.client.1.smithi148.stdout:6/136: fsync d0/d8/d1b/ff 0 2024-09-21T04:15:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/202: truncate d5/dc/d28/f29 3378714 0 2024-09-21T04:15:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/203: stat d5/d8 0 2024-09-21T04:15:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/204: fdatasync d5/d8/f22 0 2024-09-21T04:15:11.207 INFO:tasks.workunit.client.0.smithi110.stdout:0/483: truncate d4/f56 75162 0 2024-09-21T04:15:11.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/149: dwrite d3/d5/d6/d12/da/d23/f26 [0,4194304] 0 2024-09-21T04:15:11.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/150: write d3/d5/d6/d12/da/f11 [320782,31387] 0 2024-09-21T04:15:11.211 INFO:tasks.workunit.client.0.smithi110.stdout:9/163: symlink d7/l36 0 2024-09-21T04:15:11.214 INFO:tasks.workunit.client.0.smithi110.stdout:0/484: mkdir d4/dd/d10/d8a/d92 0 2024-09-21T04:15:11.215 INFO:tasks.workunit.client.1.smithi148.stdout:1/166: dwrite db/f15 [0,4194304] 0 2024-09-21T04:15:11.215 INFO:tasks.workunit.client.1.smithi148.stdout:2/248: dwrite d0/d9/dc/d13/d2c/f3a [0,4194304] 0 2024-09-21T04:15:11.215 INFO:tasks.workunit.client.1.smithi148.stdout:0/161: dread d5/d6/d7/d11/d19/f26 [0,4194304] 0 2024-09-21T04:15:11.215 INFO:tasks.workunit.client.1.smithi148.stdout:0/162: chown d5/d6/d7 708 1 2024-09-21T04:15:11.215 INFO:tasks.workunit.client.1.smithi148.stdout:0/163: truncate d5/d6/f1f 2964157 0 2024-09-21T04:15:11.216 INFO:tasks.workunit.client.0.smithi110.stdout:1/493: symlink d2/d9/d23/d4e/la7 0 2024-09-21T04:15:11.217 INFO:tasks.workunit.client.1.smithi148.stdout:0/164: dread d5/d6/d7/ff [0,4194304] 0 2024-09-21T04:15:11.217 INFO:tasks.workunit.client.1.smithi148.stdout:0/165: read d5/d6/d7/d11/f1d [4894739,71516] 0 2024-09-21T04:15:11.218 INFO:tasks.workunit.client.0.smithi110.stdout:9/164: unlink d7/d10/d24/c28 0 2024-09-21T04:15:11.219 INFO:tasks.workunit.client.1.smithi148.stdout:9/193: mkdir d5/d1e/d37/d50 0 2024-09-21T04:15:11.219 INFO:tasks.workunit.client.1.smithi148.stdout:9/194: creat d5/d22/d28/f51 x:0 0 0 2024-09-21T04:15:11.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/485: mknod d4/d8b/c93 0 2024-09-21T04:15:11.222 INFO:tasks.workunit.client.0.smithi110.stdout:1/494: mkdir d2/d9/d49/d7a/da8 0 2024-09-21T04:15:11.223 INFO:tasks.workunit.client.0.smithi110.stdout:1/495: write d2/d9/d26/d3d/d8f/f90 [534021,6510] 0 2024-09-21T04:15:11.223 INFO:tasks.workunit.client.1.smithi148.stdout:8/165: getdents dc/d13/d28 0 2024-09-21T04:15:11.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/165: creat d7/d10/f37 x:0 0 0 2024-09-21T04:15:11.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/166: chown d7/db/d14 44635419 1 2024-09-21T04:15:11.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/167: write d7/db/d11/f2e [513523,13132] 0 2024-09-21T04:15:11.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/168: fsync d7/db/f20 0 2024-09-21T04:15:11.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/169: dread d7/db/f20 [0,4194304] 0 2024-09-21T04:15:11.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/170: creat d7/d10/d24/f38 x:0 0 0 2024-09-21T04:15:11.226 INFO:tasks.workunit.client.0.smithi110.stdout:0/486: symlink d4/dd/d53/d66/l94 0 2024-09-21T04:15:11.227 INFO:tasks.workunit.client.1.smithi148.stdout:7/205: dwrite d5/d8/f1e [4194304,4194304] 0 2024-09-21T04:15:11.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/167: write db/d24/f25 [1680129,8720] 0 2024-09-21T04:15:11.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/168: write db/f15 [5205032,52688] 0 2024-09-21T04:15:11.228 INFO:tasks.workunit.client.0.smithi110.stdout:1/496: rename d2/lb to d2/d9/d49/d4f/d87/la9 0 2024-09-21T04:15:11.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/151: symlink d3/d5/d6/d2b/l2d 0 2024-09-21T04:15:11.231 INFO:tasks.workunit.client.0.smithi110.stdout:9/171: link d7/d10/d24/l2b d7/db/d1f/d25/l39 0 2024-09-21T04:15:11.232 INFO:tasks.workunit.client.1.smithi148.stdout:1/169: write f6 [2575347,25429] 0 2024-09-21T04:15:11.232 INFO:tasks.workunit.client.1.smithi148.stdout:1/170: chown db/d24 2964851 1 2024-09-21T04:15:11.233 INFO:tasks.workunit.client.0.smithi110.stdout:0/487: rmdir d4/dd/d10/d8a 39 2024-09-21T04:15:11.233 INFO:tasks.workunit.client.1.smithi148.stdout:3/152: dread d3/d5/d6/f17 [0,4194304] 0 2024-09-21T04:15:11.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/249: symlink d0/d2b/l51 0 2024-09-21T04:15:11.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/250: chown d0/d2b/d48/f44 107203 1 2024-09-21T04:15:11.235 INFO:tasks.workunit.client.0.smithi110.stdout:1/497: symlink d2/d9/d49/d5c/laa 0 2024-09-21T04:15:11.235 INFO:tasks.workunit.client.0.smithi110.stdout:1/498: read - d2/d9/d49/d4f/d53/f8c zero size 2024-09-21T04:15:11.238 INFO:tasks.workunit.client.1.smithi148.stdout:9/195: rename d5/d22/d28/d38/f44 to d5/d22/d28/d38/f52 0 2024-09-21T04:15:11.238 INFO:tasks.workunit.client.1.smithi148.stdout:9/196: dread - d5/d1f/f3e zero size 2024-09-21T04:15:11.246 INFO:tasks.workunit.client.1.smithi148.stdout:6/137: rename d0/d8/d1b to d0/d8/d22 0 2024-09-21T04:15:11.246 INFO:tasks.workunit.client.1.smithi148.stdout:6/138: write d0/d1a/f1c [473288,3062] 0 2024-09-21T04:15:11.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/166: truncate dc/d13/f19 98604 0 2024-09-21T04:15:11.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/167: truncate dc/d13/f24 125633 0 2024-09-21T04:15:11.247 INFO:tasks.workunit.client.1.smithi148.stdout:8/168: write dc/d13/f24 [103357,58153] 0 2024-09-21T04:15:11.247 INFO:tasks.workunit.client.1.smithi148.stdout:8/169: fsync f2 0 2024-09-21T04:15:11.247 INFO:tasks.workunit.client.1.smithi148.stdout:8/170: chown f7 1 1 2024-09-21T04:15:11.247 INFO:tasks.workunit.client.1.smithi148.stdout:8/171: dread - dc/d13/f1c zero size 2024-09-21T04:15:11.247 INFO:tasks.workunit.client.0.smithi110.stdout:9/172: dread d7/db/d1f/f23 [0,4194304] 0 2024-09-21T04:15:11.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/206: truncate d5/f6 247403 0 2024-09-21T04:15:11.249 INFO:tasks.workunit.client.1.smithi148.stdout:2/251: dwrite d0/d9/f39 [0,4194304] 0 2024-09-21T04:15:11.249 INFO:tasks.workunit.client.1.smithi148.stdout:2/252: readlink d0/d9/dc/d13/d2c/l3c 0 2024-09-21T04:15:11.250 INFO:tasks.workunit.client.1.smithi148.stdout:2/253: write d0/d9/dc/f46 [961986,92143] 0 2024-09-21T04:15:11.250 INFO:tasks.workunit.client.0.smithi110.stdout:9/173: unlink f3 0 2024-09-21T04:15:11.251 INFO:tasks.workunit.client.1.smithi148.stdout:8/172: write dc/f14 [3026039,52573] 0 2024-09-21T04:15:11.251 INFO:tasks.workunit.client.1.smithi148.stdout:1/171: truncate db/ff 3923583 0 2024-09-21T04:15:11.253 INFO:tasks.workunit.client.0.smithi110.stdout:1/499: read d2/d9/d26/f34 [211184,75629] 0 2024-09-21T04:15:11.253 INFO:tasks.workunit.client.0.smithi110.stdout:1/500: chown d2/d9/d23/d3b/d9c/fa4 24 1 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.0.smithi110.stdout:0/488: dread d4/d1c/f25 [0,4194304] 0 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.0.smithi110.stdout:0/489: write d4/d1c/f8d [432917,64268] 0 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/153: symlink d3/l2e 0 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/154: chown d3/d5 0 1 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/155: write f1 [1101989,96175] 0 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/156: rename d3/d5 to d3/d5/d6/d12/da/d23/d2f 22 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/157: fsync d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:11.254 INFO:tasks.workunit.client.1.smithi148.stdout:0/166: rmdir d5/d6/d7/d22 39 2024-09-21T04:15:11.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/501: stat d2/d9/d26/c42 0 2024-09-21T04:15:11.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/502: creat d2/d9/d23/d3b/d93/fab x:0 0 0 2024-09-21T04:15:11.256 INFO:tasks.workunit.client.1.smithi148.stdout:3/158: fsync f1 0 2024-09-21T04:15:11.257 INFO:tasks.workunit.client.1.smithi148.stdout:9/197: creat d5/d2b/f53 x:0 0 0 2024-09-21T04:15:11.259 INFO:tasks.workunit.client.0.smithi110.stdout:0/490: symlink d4/dd/d10/d29/l95 0 2024-09-21T04:15:11.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/207: link d5/d8/f1e d5/f3d 0 2024-09-21T04:15:11.266 INFO:tasks.workunit.client.0.smithi110.stdout:0/491: mkdir d4/dd/d53/d96 0 2024-09-21T04:15:11.266 INFO:tasks.workunit.client.0.smithi110.stdout:0/492: fdatasync d4/d47/f40 0 2024-09-21T04:15:11.269 INFO:tasks.workunit.client.1.smithi148.stdout:2/254: rmdir d0/d2b/d48 39 2024-09-21T04:15:11.271 INFO:tasks.workunit.client.0.smithi110.stdout:0/493: truncate d4/d47/f23 1370183 0 2024-09-21T04:15:11.272 INFO:tasks.workunit.client.1.smithi148.stdout:8/173: mknod dc/d2a/c33 0 2024-09-21T04:15:11.272 INFO:tasks.workunit.client.1.smithi148.stdout:8/174: creat dc/d13/f34 x:0 0 0 2024-09-21T04:15:11.272 INFO:tasks.workunit.client.1.smithi148.stdout:8/175: creat dc/d13/f35 x:0 0 0 2024-09-21T04:15:11.273 INFO:tasks.workunit.client.1.smithi148.stdout:8/176: fsync dc/d13/f1c 0 2024-09-21T04:15:11.273 INFO:tasks.workunit.client.1.smithi148.stdout:8/177: fdatasync dc/d13/f35 0 2024-09-21T04:15:11.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/172: unlink db/f1f 0 2024-09-21T04:15:11.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/173: fdatasync db/d24/f33 0 2024-09-21T04:15:11.275 INFO:tasks.workunit.client.1.smithi148.stdout:6/139: dwrite d0/f21 [4194304,4194304] 0 2024-09-21T04:15:11.278 INFO:tasks.workunit.client.0.smithi110.stdout:0/494: symlink d4/dd/d53/l97 0 2024-09-21T04:15:11.278 INFO:tasks.workunit.client.1.smithi148.stdout:0/167: dwrite d5/d6/f16 [0,4194304] 0 2024-09-21T04:15:11.279 INFO:tasks.workunit.client.1.smithi148.stdout:3/159: mknod d3/d5/c30 0 2024-09-21T04:15:11.280 INFO:tasks.workunit.client.0.smithi110.stdout:1/503: dwrite d2/d9/d23/d4e/f7f [0,4194304] 0 2024-09-21T04:15:11.282 INFO:tasks.workunit.client.1.smithi148.stdout:2/255: link d0/d9/d16/c22 d0/d9/dc/c52 0 2024-09-21T04:15:11.286 INFO:tasks.workunit.client.1.smithi148.stdout:0/168: dread d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:11.286 INFO:tasks.workunit.client.1.smithi148.stdout:0/169: chown d5/d6/f16 9765327 1 2024-09-21T04:15:11.287 INFO:tasks.workunit.client.1.smithi148.stdout:8/178: symlink dc/d13/d28/l36 0 2024-09-21T04:15:11.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/170: dread d5/d6/f1f [0,4194304] 0 2024-09-21T04:15:11.294 INFO:tasks.workunit.client.1.smithi148.stdout:1/174: creat db/d24/d2e/f3c x:0 0 0 2024-09-21T04:15:11.296 INFO:tasks.workunit.client.0.smithi110.stdout:0/495: dwrite d4/dd/d10/f50 [0,4194304] 0 2024-09-21T04:15:11.296 INFO:tasks.workunit.client.0.smithi110.stdout:0/496: truncate d4/dd/d53/f5a 4481192 0 2024-09-21T04:15:11.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/208: dwrite d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:11.300 INFO:tasks.workunit.client.0.smithi110.stdout:1/504: mknod d2/cac 0 2024-09-21T04:15:11.302 INFO:tasks.workunit.client.0.smithi110.stdout:0/497: mkdir d4/dd/d10/d29/d98 0 2024-09-21T04:15:11.302 INFO:tasks.workunit.client.0.smithi110.stdout:0/498: creat d4/dd/d10/f99 x:0 0 0 2024-09-21T04:15:11.303 INFO:tasks.workunit.client.1.smithi148.stdout:6/140: rename d0/d8/d22 to d0/d1a/d23 0 2024-09-21T04:15:11.311 INFO:tasks.workunit.client.0.smithi110.stdout:1/505: stat d2/d9/d49/l65 0 2024-09-21T04:15:11.315 INFO:tasks.workunit.client.0.smithi110.stdout:0/499: dwrite d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:11.315 INFO:tasks.workunit.client.0.smithi110.stdout:0/500: dread - d4/dd/d10/d29/f5d zero size 2024-09-21T04:15:11.315 INFO:tasks.workunit.client.0.smithi110.stdout:0/501: chown d4/d47/d2c/d3a/f6c 244969950 1 2024-09-21T04:15:11.316 INFO:tasks.workunit.client.1.smithi148.stdout:3/160: creat d3/d5/d6/d12/da/d21/f31 x:0 0 0 2024-09-21T04:15:11.316 INFO:tasks.workunit.client.1.smithi148.stdout:3/161: fsync d3/d5/d6/d12/da/d23/f26 0 2024-09-21T04:15:11.317 INFO:tasks.workunit.client.1.smithi148.stdout:9/198: rmdir d5/d1e/d37 39 2024-09-21T04:15:11.319 INFO:tasks.workunit.client.0.smithi110.stdout:1/506: write d2/d9/d49/d5c/f8a [2680675,29752] 0 2024-09-21T04:15:11.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/179: symlink dc/d13/l37 0 2024-09-21T04:15:11.322 INFO:tasks.workunit.client.0.smithi110.stdout:0/502: rmdir d4/dd/d10/d29/d75 39 2024-09-21T04:15:11.323 INFO:tasks.workunit.client.0.smithi110.stdout:0/503: fsync d4/d47/f40 0 2024-09-21T04:15:11.323 INFO:tasks.workunit.client.0.smithi110.stdout:0/504: rename d4/d47/d2c/d3a to d4/d47/d2c/d3a/d9a 22 2024-09-21T04:15:11.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/507: dread d2/d9/d23/f38 [0,4194304] 0 2024-09-21T04:15:11.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/508: write d2/d9/f1f [1524178,5342] 0 2024-09-21T04:15:11.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/509: write d2/f45 [1044639,46056] 0 2024-09-21T04:15:11.327 INFO:tasks.workunit.client.0.smithi110.stdout:6/214: sync 2024-09-21T04:15:11.327 INFO:tasks.workunit.client.0.smithi110.stdout:3/246: sync 2024-09-21T04:15:11.329 INFO:tasks.workunit.client.0.smithi110.stdout:0/505: rmdir d4/dd/d10/d29/d98 0 2024-09-21T04:15:11.334 INFO:tasks.workunit.client.0.smithi110.stdout:1/510: creat d2/d9/d23/d4e/fad x:0 0 0 2024-09-21T04:15:11.334 INFO:tasks.workunit.client.0.smithi110.stdout:1/511: fdatasync d2/d9/d23/f82 0 2024-09-21T04:15:11.335 INFO:tasks.workunit.client.0.smithi110.stdout:0/506: dread d4/dd/d10/d29/d45/f5b [0,4194304] 0 2024-09-21T04:15:11.335 INFO:tasks.workunit.client.0.smithi110.stdout:0/507: truncate d4/d1c/d62/f90 147299 0 2024-09-21T04:15:11.335 INFO:tasks.workunit.client.0.smithi110.stdout:0/508: chown d4/dd/l7d 75 1 2024-09-21T04:15:11.335 INFO:tasks.workunit.client.0.smithi110.stdout:3/247: dread d0/f7 [0,4194304] 0 2024-09-21T04:15:11.335 INFO:tasks.workunit.client.0.smithi110.stdout:3/248: write d0/f1f [1697822,85549] 0 2024-09-21T04:15:11.337 INFO:tasks.workunit.client.0.smithi110.stdout:8/264: sync 2024-09-21T04:15:11.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/512: truncate d2/d9/d49/d4f/f67 696669 0 2024-09-21T04:15:11.339 INFO:tasks.workunit.client.1.smithi148.stdout:2/256: rename d0/d9/fe to d0/d2b/f53 0 2024-09-21T04:15:11.339 INFO:tasks.workunit.client.1.smithi148.stdout:7/209: creat d5/dc/f3e x:0 0 0 2024-09-21T04:15:11.340 INFO:tasks.workunit.client.1.smithi148.stdout:7/210: dread - d5/dc/f3e zero size 2024-09-21T04:15:11.340 INFO:tasks.workunit.client.1.smithi148.stdout:1/175: getdents db/d24 0 2024-09-21T04:15:11.341 INFO:tasks.workunit.client.1.smithi148.stdout:3/162: rename d3/d5/d6/d12/da/d23/c16 to d3/d5/d6/d12/da/c32 0 2024-09-21T04:15:11.342 INFO:tasks.workunit.client.1.smithi148.stdout:3/163: fdatasync d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:11.342 INFO:tasks.workunit.client.1.smithi148.stdout:3/164: stat d3/d5/d6/d12/da/f11 0 2024-09-21T04:15:11.342 INFO:tasks.workunit.client.1.smithi148.stdout:3/165: chown d3/d5/d6/d2b 7673771 1 2024-09-21T04:15:11.347 INFO:tasks.workunit.client.1.smithi148.stdout:9/199: dwrite d5/f36 [0,4194304] 0 2024-09-21T04:15:11.348 INFO:tasks.workunit.client.1.smithi148.stdout:8/180: write dc/f18 [2422636,81142] 0 2024-09-21T04:15:11.348 INFO:tasks.workunit.client.0.smithi110.stdout:0/509: dread d4/dd/d10/f2a [0,4194304] 0 2024-09-21T04:15:11.348 INFO:tasks.workunit.client.0.smithi110.stdout:3/249: getdents d0/d12/d2a 0 2024-09-21T04:15:11.352 INFO:tasks.workunit.client.0.smithi110.stdout:3/250: rename d0/d12/f32 to d0/f56 0 2024-09-21T04:15:11.354 INFO:tasks.workunit.client.0.smithi110.stdout:6/215: dwrite d4/de/f44 [0,4194304] 0 2024-09-21T04:15:11.355 INFO:tasks.workunit.client.0.smithi110.stdout:3/251: rename d0/d12/d16/d1a/l49 to d0/l57 0 2024-09-21T04:15:11.355 INFO:tasks.workunit.client.0.smithi110.stdout:3/252: fdatasync d0/d12/d2a/f3e 0 2024-09-21T04:15:11.355 INFO:tasks.workunit.client.0.smithi110.stdout:3/253: creat d0/d12/f58 x:0 0 0 2024-09-21T04:15:11.355 INFO:tasks.workunit.client.0.smithi110.stdout:3/254: write d0/d12/d3c/f4f [304617,23738] 0 2024-09-21T04:15:11.359 INFO:tasks.workunit.client.0.smithi110.stdout:0/510: dread d4/d1c/d62/d69/f77 [0,4194304] 0 2024-09-21T04:15:11.359 INFO:tasks.workunit.client.0.smithi110.stdout:0/511: readlink d4/dd/l89 0 2024-09-21T04:15:11.360 INFO:tasks.workunit.client.0.smithi110.stdout:0/512: fsync d4/f3b 0 2024-09-21T04:15:11.361 INFO:tasks.workunit.client.0.smithi110.stdout:6/216: stat d4/l32 0 2024-09-21T04:15:11.361 INFO:tasks.workunit.client.0.smithi110.stdout:6/217: chown d4/d12/d16/c20 61253 1 2024-09-21T04:15:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/211: unlink d5/dc/d24/f31 0 2024-09-21T04:15:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:1/176: rmdir db/d24/d2a 39 2024-09-21T04:15:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:1/177: chown db/d24/d2a 2073116437 1 2024-09-21T04:15:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/212: chown d5/d14/d35 7908710 1 2024-09-21T04:15:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/213: fdatasync d5/f34 0 2024-09-21T04:15:11.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/255: symlink d0/d22/l59 0 2024-09-21T04:15:11.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/256: chown d0/d12/d16/c3f 180620680 1 2024-09-21T04:15:11.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/218: unlink d4/d12/d17/d2e/d40/c3a 0 2024-09-21T04:15:11.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/219: chown d4/d12/d17/d2e/d40/l45 1206 1 2024-09-21T04:15:11.375 INFO:tasks.workunit.client.1.smithi148.stdout:4/187: sync 2024-09-21T04:15:11.375 INFO:tasks.workunit.client.0.smithi110.stdout:6/220: dread d4/f6 [0,4194304] 0 2024-09-21T04:15:11.375 INFO:tasks.workunit.client.0.smithi110.stdout:6/221: truncate d4/fb 4363953 0 2024-09-21T04:15:11.377 INFO:tasks.workunit.client.1.smithi148.stdout:1/178: write db/dd/f13 [2325796,35923] 0 2024-09-21T04:15:11.378 INFO:tasks.workunit.client.0.smithi110.stdout:6/222: stat d4/d12/d16/c31 0 2024-09-21T04:15:11.383 INFO:tasks.workunit.client.0.smithi110.stdout:6/223: dread d4/f13 [0,4194304] 0 2024-09-21T04:15:11.385 INFO:tasks.workunit.client.0.smithi110.stdout:6/224: mkdir d4/d12/d16/d48 0 2024-09-21T04:15:11.385 INFO:tasks.workunit.client.0.smithi110.stdout:6/225: stat d4/l32 0 2024-09-21T04:15:11.386 INFO:tasks.workunit.client.0.smithi110.stdout:6/226: truncate d4/d12/d17/f2a 942775 0 2024-09-21T04:15:11.387 INFO:tasks.workunit.client.0.smithi110.stdout:3/257: dread d0/f6 [0,4194304] 0 2024-09-21T04:15:11.387 INFO:tasks.workunit.client.0.smithi110.stdout:3/258: truncate d0/fe 1502465 0 2024-09-21T04:15:11.395 INFO:tasks.workunit.client.1.smithi148.stdout:7/214: link d5/d8/c20 d5/d14/c3f 0 2024-09-21T04:15:11.395 INFO:tasks.workunit.client.1.smithi148.stdout:5/172: sync 2024-09-21T04:15:11.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/257: dread d0/d9/f17 [0,4194304] 0 2024-09-21T04:15:11.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/258: chown d0/d2b/f4b 46556647 1 2024-09-21T04:15:11.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/188: mknod d4/d12/d16/d26/c38 0 2024-09-21T04:15:11.397 INFO:tasks.workunit.client.0.smithi110.stdout:8/265: dwrite d0/d7/d13/f2d [0,4194304] 0 2024-09-21T04:15:11.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/179: symlink db/d24/d2e/l3d 0 2024-09-21T04:15:11.399 INFO:tasks.workunit.client.1.smithi148.stdout:8/181: dwrite dc/f14 [0,4194304] 0 2024-09-21T04:15:11.399 INFO:tasks.workunit.client.1.smithi148.stdout:8/182: rename dc/d13 to dc/d13/d38 22 2024-09-21T04:15:11.400 INFO:tasks.workunit.client.0.smithi110.stdout:1/513: dwrite d2/d9/d23/d3b/d93/fab [0,4194304] 0 2024-09-21T04:15:11.400 INFO:tasks.workunit.client.1.smithi148.stdout:7/215: mkdir d5/d14/d25/d40 0 2024-09-21T04:15:11.401 INFO:tasks.workunit.client.1.smithi148.stdout:9/200: dwrite d5/d2b/f53 [0,4194304] 0 2024-09-21T04:15:11.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/259: mkdir d0/d9/dc/d13/d54 0 2024-09-21T04:15:11.403 INFO:tasks.workunit.client.1.smithi148.stdout:5/173: dread d3/f8 [0,4194304] 0 2024-09-21T04:15:11.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/189: dread d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:11.406 INFO:tasks.workunit.client.0.smithi110.stdout:0/513: dwrite d4/dd/d53/d66/f74 [0,4194304] 0 2024-09-21T04:15:11.413 INFO:tasks.workunit.client.0.smithi110.stdout:6/227: dwrite d4/f13 [0,4194304] 0 2024-09-21T04:15:11.414 INFO:tasks.workunit.client.1.smithi148.stdout:8/183: mkdir dc/d13/d39 0 2024-09-21T04:15:11.414 INFO:tasks.workunit.client.1.smithi148.stdout:7/216: truncate d5/d8/f22 866477 0 2024-09-21T04:15:11.422 INFO:tasks.workunit.client.0.smithi110.stdout:3/259: dwrite d0/f56 [0,4194304] 0 2024-09-21T04:15:11.424 INFO:tasks.workunit.client.1.smithi148.stdout:2/260: mkdir d0/d9/d16/d55 0 2024-09-21T04:15:11.424 INFO:tasks.workunit.client.1.smithi148.stdout:2/261: chown d0/d2b 4115 1 2024-09-21T04:15:11.427 INFO:tasks.workunit.client.1.smithi148.stdout:1/180: mknod db/c3e 0 2024-09-21T04:15:11.429 INFO:tasks.workunit.client.1.smithi148.stdout:4/190: rename d4/d12/d16/c30 to d4/d12/d16/c39 0 2024-09-21T04:15:11.430 INFO:tasks.workunit.client.1.smithi148.stdout:4/191: chown d4/d12 114 1 2024-09-21T04:15:11.430 INFO:tasks.workunit.client.1.smithi148.stdout:4/192: write d4/d23/f2b [725069,64724] 0 2024-09-21T04:15:11.430 INFO:tasks.workunit.client.1.smithi148.stdout:7/217: creat d5/d14/d25/d40/f41 x:0 0 0 2024-09-21T04:15:11.431 INFO:tasks.workunit.client.0.smithi110.stdout:6/228: write d4/de/ff [1659613,107681] 0 2024-09-21T04:15:11.433 INFO:tasks.workunit.client.1.smithi148.stdout:9/201: rename d5/l1d to d5/d2b/d32/l54 0 2024-09-21T04:15:11.436 INFO:tasks.workunit.client.1.smithi148.stdout:2/262: rmdir d0/d9/dc/d13/d54 0 2024-09-21T04:15:11.437 INFO:tasks.workunit.client.0.smithi110.stdout:1/514: link d2/d9/c22 d2/cae 0 2024-09-21T04:15:11.437 INFO:tasks.workunit.client.0.smithi110.stdout:1/515: stat d2/d9/d49/d5c 0 2024-09-21T04:15:11.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/184: dwrite dc/d13/d28/f2b [0,4194304] 0 2024-09-21T04:15:11.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/185: readlink dc/l22 0 2024-09-21T04:15:11.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/174: getdents d3/d14 0 2024-09-21T04:15:11.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/175: write d3/f8 [2629128,111900] 0 2024-09-21T04:15:11.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/176: stat d3/fa 0 2024-09-21T04:15:11.441 INFO:tasks.workunit.client.0.smithi110.stdout:8/266: dwrite d0/d7/d13/d4b/f35 [4194304,4194304] 0 2024-09-21T04:15:11.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/181: symlink db/dd/l3f 0 2024-09-21T04:15:11.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/182: read - db/dd/f23 zero size 2024-09-21T04:15:11.445 INFO:tasks.workunit.client.1.smithi148.stdout:4/193: dread d4/d23/f2b [0,4194304] 0 2024-09-21T04:15:11.448 INFO:tasks.workunit.client.0.smithi110.stdout:0/514: mkdir d4/d1c/d62/d69/d9b 0 2024-09-21T04:15:11.455 INFO:tasks.workunit.client.1.smithi148.stdout:2/263: link d0/d9/dc/f24 d0/d9/d16/f56 0 2024-09-21T04:15:11.455 INFO:tasks.workunit.client.1.smithi148.stdout:2/264: write d0/f1e [4537968,32093] 0 2024-09-21T04:15:11.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/260: mkdir d0/d12/d3c/d5a 0 2024-09-21T04:15:11.461 INFO:tasks.workunit.client.1.smithi148.stdout:2/265: dread d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:11.462 INFO:tasks.workunit.client.0.smithi110.stdout:4/357: sync 2024-09-21T04:15:11.462 INFO:tasks.workunit.client.0.smithi110.stdout:7/221: sync 2024-09-21T04:15:11.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/142: sync 2024-09-21T04:15:11.462 INFO:tasks.workunit.client.0.smithi110.stdout:5/274: sync 2024-09-21T04:15:11.463 INFO:tasks.workunit.client.0.smithi110.stdout:6/229: symlink d4/d12/d16/l49 0 2024-09-21T04:15:11.464 INFO:tasks.workunit.client.1.smithi148.stdout:8/186: symlink dc/d2a/l3a 0 2024-09-21T04:15:11.465 INFO:tasks.workunit.client.0.smithi110.stdout:1/516: rmdir d2/d9/d49/d7a 39 2024-09-21T04:15:11.468 INFO:tasks.workunit.client.1.smithi148.stdout:7/218: dread d5/d8/f22 [0,4194304] 0 2024-09-21T04:15:11.476 INFO:tasks.workunit.client.0.smithi110.stdout:9/174: sync 2024-09-21T04:15:11.477 INFO:tasks.workunit.client.1.smithi148.stdout:5/177: dwrite d3/dc/f15 [0,4194304] 0 2024-09-21T04:15:11.479 INFO:tasks.workunit.client.0.smithi110.stdout:4/358: rename d17/d1e/d21/c66 to d17/d34/c7e 0 2024-09-21T04:15:11.479 INFO:tasks.workunit.client.0.smithi110.stdout:4/359: chown d17/d34/d3d/d20/f2c 11228 1 2024-09-21T04:15:11.480 INFO:tasks.workunit.client.0.smithi110.stdout:3/261: write d0/d12/d16/d1a/f35 [4004241,68411] 0 2024-09-21T04:15:11.481 INFO:tasks.workunit.client.0.smithi110.stdout:5/275: truncate d1/d39/f50 5017634 0 2024-09-21T04:15:11.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/187: dread f2 [0,4194304] 0 2024-09-21T04:15:11.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/188: fsync dc/f31 0 2024-09-21T04:15:11.483 INFO:tasks.workunit.client.1.smithi148.stdout:2/266: link d0/f1d d0/d2b/d48/f57 0 2024-09-21T04:15:11.483 INFO:tasks.workunit.client.1.smithi148.stdout:2/267: write d0/d9/dc/d13/f21 [189096,49850] 0 2024-09-21T04:15:11.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/219: unlink f2 0 2024-09-21T04:15:11.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/220: fsync d5/d8/f22 0 2024-09-21T04:15:11.489 INFO:tasks.workunit.client.0.smithi110.stdout:1/517: unlink d2/d9/d49/d5c/f8a 0 2024-09-21T04:15:11.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/222: dwrite dd/f2e [0,4194304] 0 2024-09-21T04:15:11.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/223: creat dd/d14/f41 x:0 0 0 2024-09-21T04:15:11.491 INFO:tasks.workunit.client.1.smithi148.stdout:1/183: rename db/c1a to db/d24/c40 0 2024-09-21T04:15:11.493 INFO:tasks.workunit.client.0.smithi110.stdout:9/175: creat d7/db/d35/f3a x:0 0 0 2024-09-21T04:15:11.495 INFO:tasks.workunit.client.1.smithi148.stdout:4/194: rmdir d4/d12/d16/d29 39 2024-09-21T04:15:11.497 INFO:tasks.workunit.client.1.smithi148.stdout:2/268: write d0/d9/dc/f24 [3690589,110920] 0 2024-09-21T04:15:11.498 INFO:tasks.workunit.client.0.smithi110.stdout:4/360: unlink d17/d34/d3d/d20/f2c 0 2024-09-21T04:15:11.499 INFO:tasks.workunit.client.0.smithi110.stdout:6/230: dwrite d4/d12/d17/d2e/d40/f2f [4194304,4194304] 0 2024-09-21T04:15:11.501 INFO:tasks.workunit.client.0.smithi110.stdout:2/143: rename d2/d3 to d2/d1a/de/d25 0 2024-09-21T04:15:11.501 INFO:tasks.workunit.client.0.smithi110.stdout:2/144: chown d2/d1a/de/d25/c14 18717305 1 2024-09-21T04:15:11.503 INFO:tasks.workunit.client.1.smithi148.stdout:2/269: read d0/d9/f17 [75282,58164] 0 2024-09-21T04:15:11.503 INFO:tasks.workunit.client.0.smithi110.stdout:4/361: dread d17/f27 [4194304,4194304] 0 2024-09-21T04:15:11.504 INFO:tasks.workunit.client.0.smithi110.stdout:3/262: symlink d0/d12/d16/d1a/l5b 0 2024-09-21T04:15:11.506 INFO:tasks.workunit.client.0.smithi110.stdout:5/276: creat d1/d49/d4d/d1e/f5f x:0 0 0 2024-09-21T04:15:11.507 INFO:tasks.workunit.client.1.smithi148.stdout:5/178: mkdir d3/d34/d3c 0 2024-09-21T04:15:11.507 INFO:tasks.workunit.client.0.smithi110.stdout:8/267: dwrite d0/d7/d13/d4b/f4e [0,4194304] 0 2024-09-21T04:15:11.509 INFO:tasks.workunit.client.0.smithi110.stdout:7/224: getdents dd/d1c/d39 0 2024-09-21T04:15:11.510 INFO:tasks.workunit.client.1.smithi148.stdout:7/221: creat d5/d14/d25/f42 x:0 0 0 2024-09-21T04:15:11.510 INFO:tasks.workunit.client.0.smithi110.stdout:9/176: symlink d7/db/d14/d31/l3b 0 2024-09-21T04:15:11.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/195: mkdir d4/d3a 0 2024-09-21T04:15:11.513 INFO:tasks.workunit.client.0.smithi110.stdout:6/231: link d4/d12/d17/d2e/d40/l3b d4/d12/d17/d33/l4a 0 2024-09-21T04:15:11.516 INFO:tasks.workunit.client.0.smithi110.stdout:2/145: mkdir d2/d1a/de/d26 0 2024-09-21T04:15:11.516 INFO:tasks.workunit.client.0.smithi110.stdout:2/146: read - d2/d1a/de/f22 zero size 2024-09-21T04:15:11.518 INFO:tasks.workunit.client.0.smithi110.stdout:3/263: mkdir d0/d12/d16/d1a/d5c 0 2024-09-21T04:15:11.518 INFO:tasks.workunit.client.0.smithi110.stdout:3/264: creat d0/d12/f5d x:0 0 0 2024-09-21T04:15:11.518 INFO:tasks.workunit.client.0.smithi110.stdout:3/265: creat d0/d12/d3c/f5e x:0 0 0 2024-09-21T04:15:11.519 INFO:tasks.workunit.client.0.smithi110.stdout:5/277: creat d1/d7/d58/f60 x:0 0 0 2024-09-21T04:15:11.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/268: creat d0/d7/f5c x:0 0 0 2024-09-21T04:15:11.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/269: stat d0/d7/d13/d1c/f1d 0 2024-09-21T04:15:11.523 INFO:tasks.workunit.client.1.smithi148.stdout:4/196: dread d4/d12/f1b [0,4194304] 0 2024-09-21T04:15:11.523 INFO:tasks.workunit.client.1.smithi148.stdout:2/270: dwrite d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:11.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/222: dread d5/dc/d28/f29 [0,4194304] 0 2024-09-21T04:15:11.525 INFO:tasks.workunit.client.0.smithi110.stdout:7/225: mknod dd/d14/c42 0 2024-09-21T04:15:11.526 INFO:tasks.workunit.client.1.smithi148.stdout:0/171: sync 2024-09-21T04:15:11.530 INFO:tasks.workunit.client.1.smithi148.stdout:2/271: unlink d0/d9/d16/c3d 0 2024-09-21T04:15:11.530 INFO:tasks.workunit.client.1.smithi148.stdout:5/179: getdents d3/d34 0 2024-09-21T04:15:11.531 INFO:tasks.workunit.client.0.smithi110.stdout:6/232: mkdir d4/d12/d16/d48/d4b 0 2024-09-21T04:15:11.532 INFO:tasks.workunit.client.1.smithi148.stdout:7/223: readlink d5/dc/d24/l38 0 2024-09-21T04:15:11.533 INFO:tasks.workunit.client.0.smithi110.stdout:4/362: getdents d17/d34/d3d/d57 0 2024-09-21T04:15:11.533 INFO:tasks.workunit.client.0.smithi110.stdout:4/363: read d17/d34/d3d/d20/f49 [5153358,64667] 0 2024-09-21T04:15:11.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/172: rename d5/d6/d7/d18/f2d to d5/d6/d7/d11/f38 0 2024-09-21T04:15:11.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/173: dread - d5/d6/d2a/d2b/f36 zero size 2024-09-21T04:15:11.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/174: chown d5/d6/d7/d11/f17 666020575 1 2024-09-21T04:15:11.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/175: creat d5/d6/d2a/d2b/f39 x:0 0 0 2024-09-21T04:15:11.537 INFO:tasks.workunit.client.1.smithi148.stdout:4/197: dwrite d4/d12/d2c/f32 [0,4194304] 0 2024-09-21T04:15:11.537 INFO:tasks.workunit.client.1.smithi148.stdout:5/180: mkdir d3/d14/d1f/d26/d2e/d3d 0 2024-09-21T04:15:11.538 INFO:tasks.workunit.client.1.smithi148.stdout:5/181: dread - d3/d14/f29 zero size 2024-09-21T04:15:11.540 INFO:tasks.workunit.client.1.smithi148.stdout:7/224: mknod d5/d14/d25/c43 0 2024-09-21T04:15:11.540 INFO:tasks.workunit.client.1.smithi148.stdout:7/225: stat d5/dc/d24/c30 0 2024-09-21T04:15:11.540 INFO:tasks.workunit.client.1.smithi148.stdout:7/226: chown d5/c2a 55 1 2024-09-21T04:15:11.540 INFO:tasks.workunit.client.0.smithi110.stdout:3/266: link d0/d12/d3c/f48 d0/d22/d29/f5f 0 2024-09-21T04:15:11.540 INFO:tasks.workunit.client.0.smithi110.stdout:3/267: fdatasync d0/d12/d16/d1a/f35 0 2024-09-21T04:15:11.542 INFO:tasks.workunit.client.0.smithi110.stdout:3/268: dread d0/fe [0,4194304] 0 2024-09-21T04:15:11.543 INFO:tasks.workunit.client.0.smithi110.stdout:5/278: unlink d1/d7/d18/d3f/l40 0 2024-09-21T04:15:11.543 INFO:tasks.workunit.client.0.smithi110.stdout:5/279: truncate d1/dc/f57 216565 0 2024-09-21T04:15:11.543 INFO:tasks.workunit.client.0.smithi110.stdout:5/280: dread - d1/d7/f2b zero size 2024-09-21T04:15:11.544 INFO:tasks.workunit.client.1.smithi148.stdout:0/176: unlink d5/d6/d7/f2f 0 2024-09-21T04:15:11.545 INFO:tasks.workunit.client.0.smithi110.stdout:8/270: mkdir d0/d30/d34/d3e/d5d 0 2024-09-21T04:15:11.552 INFO:tasks.workunit.client.0.smithi110.stdout:9/177: rename d7/db to d7/d10/d24/d3c 0 2024-09-21T04:15:11.552 INFO:tasks.workunit.client.0.smithi110.stdout:2/147: rename d2/d1a/d1f to d2/d1a/d1f/d27 22 2024-09-21T04:15:11.552 INFO:tasks.workunit.client.0.smithi110.stdout:2/148: stat d2/d1a/f20 0 2024-09-21T04:15:11.553 INFO:tasks.workunit.client.0.smithi110.stdout:9/178: write d7/d10/d24/f2a [99676,14076] 0 2024-09-21T04:15:11.553 INFO:tasks.workunit.client.0.smithi110.stdout:9/179: write d7/d10/d24/f2a [45601,42728] 0 2024-09-21T04:15:11.553 INFO:tasks.workunit.client.0.smithi110.stdout:6/233: mknod d4/d3c/c4c 0 2024-09-21T04:15:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:4/198: link d4/d12/d16/d26/c38 d4/c3b 0 2024-09-21T04:15:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/177: write d5/d6/d7/d11/f1d [535536,42610] 0 2024-09-21T04:15:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/182: mknod d3/d37/c3e 0 2024-09-21T04:15:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/178: write d5/d6/d7/d18/f32 [847597,82822] 0 2024-09-21T04:15:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/179: creat d5/d6/d7/d11/d19/f3a x:0 0 0 2024-09-21T04:15:11.555 INFO:tasks.workunit.client.0.smithi110.stdout:3/269: creat d0/d12/d16/d1a/d5c/f60 x:0 0 0 2024-09-21T04:15:11.556 INFO:tasks.workunit.client.1.smithi148.stdout:0/180: dread d5/d6/f1f [0,4194304] 0 2024-09-21T04:15:11.557 INFO:tasks.workunit.client.0.smithi110.stdout:2/149: truncate d2/d1a/f20 3538145 0 2024-09-21T04:15:11.557 INFO:tasks.workunit.client.0.smithi110.stdout:2/150: chown d2/f1c 78745 1 2024-09-21T04:15:11.557 INFO:tasks.workunit.client.0.smithi110.stdout:2/151: chown d2/d1a/f20 2 1 2024-09-21T04:15:11.560 INFO:tasks.workunit.client.0.smithi110.stdout:7/226: rename dd/d14/c2d to dd/d1c/c43 0 2024-09-21T04:15:11.561 INFO:tasks.workunit.client.1.smithi148.stdout:5/183: dread d3/f9 [0,4194304] 0 2024-09-21T04:15:11.562 INFO:tasks.workunit.client.1.smithi148.stdout:7/227: truncate d5/dc/f19 3923875 0 2024-09-21T04:15:11.563 INFO:tasks.workunit.client.0.smithi110.stdout:9/180: rmdir d7/d10/d24/d3c/d35 39 2024-09-21T04:15:11.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/181: symlink d5/d6/d2a/l3b 0 2024-09-21T04:15:11.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/182: chown d5/d6/d7/f31 540 1 2024-09-21T04:15:11.566 INFO:tasks.workunit.client.0.smithi110.stdout:8/271: dwrite d0/d7/d13/f37 [0,4194304] 0 2024-09-21T04:15:11.566 INFO:tasks.workunit.client.1.smithi148.stdout:4/199: dread d4/f8 [0,4194304] 0 2024-09-21T04:15:11.566 INFO:tasks.workunit.client.1.smithi148.stdout:7/228: mknod d5/d8/c44 0 2024-09-21T04:15:11.567 INFO:tasks.workunit.client.0.smithi110.stdout:3/270: unlink d0/d12/d16/f34 0 2024-09-21T04:15:11.570 INFO:tasks.workunit.client.0.smithi110.stdout:2/152: mknod d2/d1a/d1f/c28 0 2024-09-21T04:15:11.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/229: write d5/dc/f10 [214436,31279] 0 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/281: rename d1/d39/f1c to d1/d49/d4d/d1e/f61 0 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/282: write d1/d49/d4d/d1e/f2f [1470686,75132] 0 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/283: write d1/f4c [653955,7762] 0 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/284: rename d1/d49 to d1/d49/d4d/d1e/d4a/d62 22 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/285: chown d1/d7/d58 237202 1 2024-09-21T04:15:11.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/286: creat d1/d39/f63 x:0 0 0 2024-09-21T04:15:11.576 INFO:tasks.workunit.client.0.smithi110.stdout:5/287: chown d1/dc/f1a 23495543 1 2024-09-21T04:15:11.576 INFO:tasks.workunit.client.0.smithi110.stdout:9/181: symlink d7/d10/d24/d3c/d1f/l3d 0 2024-09-21T04:15:11.579 INFO:tasks.workunit.client.0.smithi110.stdout:4/364: rmdir d17/d1e/d6a 0 2024-09-21T04:15:11.579 INFO:tasks.workunit.client.0.smithi110.stdout:4/365: chown d17/d34/c46 47 1 2024-09-21T04:15:11.583 INFO:tasks.workunit.client.0.smithi110.stdout:9/182: read d7/d10/f13 [3829913,55546] 0 2024-09-21T04:15:11.584 INFO:tasks.workunit.client.1.smithi148.stdout:7/230: mknod d5/d14/c45 0 2024-09-21T04:15:11.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/141: sync 2024-09-21T04:15:11.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/166: sync 2024-09-21T04:15:11.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/142: dread - d0/f20 zero size 2024-09-21T04:15:11.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/143: chown d0/d1a/d23/ca 176012 1 2024-09-21T04:15:11.585 INFO:tasks.workunit.client.0.smithi110.stdout:3/271: unlink d0/c14 0 2024-09-21T04:15:11.585 INFO:tasks.workunit.client.0.smithi110.stdout:3/272: write d0/d12/d2a/f53 [1016315,59995] 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/227: dwrite dd/d1c/f2c [0,4194304] 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/228: fdatasync dd/d14/f36 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/229: fsync dd/d1c/f38 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/230: readlink dd/d14/d17/l1f 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/167: truncate f1 1713672 0 2024-09-21T04:15:11.588 INFO:tasks.workunit.client.1.smithi148.stdout:0/183: rmdir d5/d6/d7/d11/d19 39 2024-09-21T04:15:11.590 INFO:tasks.workunit.client.1.smithi148.stdout:6/144: truncate d0/d1a/d23/fb 5252828 0 2024-09-21T04:15:11.591 INFO:tasks.workunit.client.1.smithi148.stdout:4/200: dwrite d4/d23/d2a/f36 [0,4194304] 0 2024-09-21T04:15:11.591 INFO:tasks.workunit.client.1.smithi148.stdout:4/201: stat d4/d23 0 2024-09-21T04:15:11.591 INFO:tasks.workunit.client.1.smithi148.stdout:4/202: dread - d4/d12/d16/d29/f33 zero size 2024-09-21T04:15:11.591 INFO:tasks.workunit.client.1.smithi148.stdout:4/203: write d4/d23/f2b [837834,38521] 0 2024-09-21T04:15:11.591 INFO:tasks.workunit.client.1.smithi148.stdout:5/184: dwrite d3/f9 [4194304,4194304] 0 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/288: mkdir d1/d49/d4d/d1e/d5a/d64 0 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/289: stat d1/dc/f1d 0 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/290: creat d1/d22/d51/f65 x:0 0 0 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/291: read - d1/d39/f63 zero size 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/292: chown d1/d49/d4d 1133921 1 2024-09-21T04:15:11.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/293: chown d1/d49/d4d/l5d 2453534 1 2024-09-21T04:15:11.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/231: rename d5/d8/c17 to d5/dc/d28/c46 0 2024-09-21T04:15:11.595 INFO:tasks.workunit.client.0.smithi110.stdout:4/366: creat d17/d34/d3d/d20/d43/d51/d7a/f7f x:0 0 0 2024-09-21T04:15:11.595 INFO:tasks.workunit.client.0.smithi110.stdout:4/367: write d17/d1e/f79 [408887,385] 0 2024-09-21T04:15:11.595 INFO:tasks.workunit.client.0.smithi110.stdout:4/368: write d17/d1e/f48 [4241343,107649] 0 2024-09-21T04:15:11.597 INFO:tasks.workunit.client.1.smithi148.stdout:3/168: mkdir d3/d33 0 2024-09-21T04:15:11.597 INFO:tasks.workunit.client.0.smithi110.stdout:9/183: mkdir d7/d10/d24/d3c/d14/d3e 0 2024-09-21T04:15:11.597 INFO:tasks.workunit.client.0.smithi110.stdout:9/184: fdatasync d7/d26/f2c 0 2024-09-21T04:15:11.599 INFO:tasks.workunit.client.0.smithi110.stdout:8/272: mknod d0/d7/d13/d4b/c5e 0 2024-09-21T04:15:11.599 INFO:tasks.workunit.client.0.smithi110.stdout:8/273: chown d0/d7/f5c 1609821 1 2024-09-21T04:15:11.599 INFO:tasks.workunit.client.0.smithi110.stdout:4/369: write d17/d1e/d21/f5b [1878995,85105] 0 2024-09-21T04:15:11.602 INFO:tasks.workunit.client.0.smithi110.stdout:3/273: creat d0/d12/d2a/f61 x:0 0 0 2024-09-21T04:15:11.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/204: chown d4/d12/l21 66972693 1 2024-09-21T04:15:11.604 INFO:tasks.workunit.client.0.smithi110.stdout:2/153: getdents d2/d1a/de 0 2024-09-21T04:15:11.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/185: creat d3/dc/f3f x:0 0 0 2024-09-21T04:15:11.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/186: truncate d3/dc/f19 1428799 0 2024-09-21T04:15:11.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/187: write d3/f8 [1263154,36951] 0 2024-09-21T04:15:11.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/188: chown d3/dc 31541 1 2024-09-21T04:15:11.611 INFO:tasks.workunit.client.0.smithi110.stdout:7/231: mknod dd/c44 0 2024-09-21T04:15:11.613 INFO:tasks.workunit.client.1.smithi148.stdout:0/184: dwrite d5/d6/d7/d22/f24 [4194304,4194304] 0 2024-09-21T04:15:11.613 INFO:tasks.workunit.client.1.smithi148.stdout:0/185: readlink d5/d6/d2a/l3b 0 2024-09-21T04:15:11.614 INFO:tasks.workunit.client.1.smithi148.stdout:7/232: mknod d5/d14/d25/d40/c47 0 2024-09-21T04:15:11.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/205: dread d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:11.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/206: readlink d4/d12/d16/l1d 0 2024-09-21T04:15:11.615 INFO:tasks.workunit.client.1.smithi148.stdout:0/186: dread d5/d6/d7/d11/d19/f27 [0,4194304] 0 2024-09-21T04:15:11.617 INFO:tasks.workunit.client.1.smithi148.stdout:3/169: symlink d3/d5/d6/d12/da/d23/l34 0 2024-09-21T04:15:11.617 INFO:tasks.workunit.client.1.smithi148.stdout:3/170: stat d3/d5/d6/d12/da/d23/d28 0 2024-09-21T04:15:11.617 INFO:tasks.workunit.client.1.smithi148.stdout:3/171: write d3/d5/d6/f17 [1345814,48204] 0 2024-09-21T04:15:11.618 INFO:tasks.workunit.client.1.smithi148.stdout:3/172: creat d3/d5/f35 x:0 0 0 2024-09-21T04:15:11.618 INFO:tasks.workunit.client.1.smithi148.stdout:3/173: chown d3/d5 106 1 2024-09-21T04:15:11.619 INFO:tasks.workunit.client.0.smithi110.stdout:5/294: truncate d1/d7/f13 126115 0 2024-09-21T04:15:11.621 INFO:tasks.workunit.client.1.smithi148.stdout:5/189: symlink d3/d34/d3c/l40 0 2024-09-21T04:15:11.625 INFO:tasks.workunit.client.1.smithi148.stdout:7/233: link d5/d8/f22 d5/dc/d24/f48 0 2024-09-21T04:15:11.626 INFO:tasks.workunit.client.0.smithi110.stdout:3/274: unlink d0/f1d 0 2024-09-21T04:15:11.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/207: mknod d4/c3c 0 2024-09-21T04:15:11.628 INFO:tasks.workunit.client.0.smithi110.stdout:2/154: unlink d2/f1c 0 2024-09-21T04:15:11.629 INFO:tasks.workunit.client.0.smithi110.stdout:4/370: dwrite d17/d1e/d7b/f6d [0,4194304] 0 2024-09-21T04:15:11.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/274: dwrite d0/d7/d13/d1c/f1d [0,4194304] 0 2024-09-21T04:15:11.630 INFO:tasks.workunit.client.0.smithi110.stdout:7/232: symlink dd/d1c/l45 0 2024-09-21T04:15:11.630 INFO:tasks.workunit.client.1.smithi148.stdout:0/187: creat d5/d6/f3c x:0 0 0 2024-09-21T04:15:11.630 INFO:tasks.workunit.client.1.smithi148.stdout:0/188: chown d5/d6/d2a/d2b/f2c 3530644 1 2024-09-21T04:15:11.630 INFO:tasks.workunit.client.1.smithi148.stdout:0/189: rename d5 to d5/d3d 22 2024-09-21T04:15:11.631 INFO:tasks.workunit.client.1.smithi148.stdout:5/190: write d3/f1b [685142,15706] 0 2024-09-21T04:15:11.631 INFO:tasks.workunit.client.0.smithi110.stdout:5/295: mknod d1/dc/c66 0 2024-09-21T04:15:11.632 INFO:tasks.workunit.client.0.smithi110.stdout:5/296: fsync d1/dc/f1a 0 2024-09-21T04:15:11.632 INFO:tasks.workunit.client.0.smithi110.stdout:5/297: write d1/d22/d51/f65 [130011,109089] 0 2024-09-21T04:15:11.632 INFO:tasks.workunit.client.0.smithi110.stdout:9/185: getdents d7/d26 0 2024-09-21T04:15:11.632 INFO:tasks.workunit.client.0.smithi110.stdout:9/186: creat d7/d10/f3f x:0 0 0 2024-09-21T04:15:11.633 INFO:tasks.workunit.client.1.smithi148.stdout:3/174: rmdir d3/d33 0 2024-09-21T04:15:11.634 INFO:tasks.workunit.client.0.smithi110.stdout:3/275: truncate d0/d12/d3c/f4f 251706 0 2024-09-21T04:15:11.634 INFO:tasks.workunit.client.0.smithi110.stdout:3/276: chown d0/d12/d3c 33338 1 2024-09-21T04:15:11.634 INFO:tasks.workunit.client.0.smithi110.stdout:3/277: creat d0/d12/d16/f62 x:0 0 0 2024-09-21T04:15:11.635 INFO:tasks.workunit.client.1.smithi148.stdout:4/208: truncate d4/d23/f2e 2062989 0 2024-09-21T04:15:11.635 INFO:tasks.workunit.client.1.smithi148.stdout:4/209: write d4/d12/d2c/f32 [4941367,93713] 0 2024-09-21T04:15:11.636 INFO:tasks.workunit.client.0.smithi110.stdout:2/155: rmdir d2/d1a/de/d26 0 2024-09-21T04:15:11.639 INFO:tasks.workunit.client.1.smithi148.stdout:0/190: rename d5/d6/d7/d11/d19 to d5/d6/d3e 0 2024-09-21T04:15:11.639 INFO:tasks.workunit.client.1.smithi148.stdout:0/191: chown d5/d6/d7/cd 8103962 1 2024-09-21T04:15:11.643 INFO:tasks.workunit.client.0.smithi110.stdout:7/233: dread dd/d1c/f38 [0,4194304] 0 2024-09-21T04:15:11.643 INFO:tasks.workunit.client.0.smithi110.stdout:7/234: chown dd/d14/f36 61134660 1 2024-09-21T04:15:11.643 INFO:tasks.workunit.client.0.smithi110.stdout:7/235: fdatasync dd/d1c/d39/f3c 0 2024-09-21T04:15:11.644 INFO:tasks.workunit.client.1.smithi148.stdout:3/175: write d3/d5/d6/d12/da/d23/f26 [3895927,41489] 0 2024-09-21T04:15:11.644 INFO:tasks.workunit.client.1.smithi148.stdout:3/176: truncate d3/d5/f1e 951325 0 2024-09-21T04:15:11.644 INFO:tasks.workunit.client.1.smithi148.stdout:3/177: rename d3 to d3/d5/d6/d12/da/d23/d28/d36 22 2024-09-21T04:15:11.645 INFO:tasks.workunit.client.0.smithi110.stdout:7/236: dread dd/f1a [0,4194304] 0 2024-09-21T04:15:11.645 INFO:tasks.workunit.client.0.smithi110.stdout:7/237: creat dd/d14/f46 x:0 0 0 2024-09-21T04:15:11.645 INFO:tasks.workunit.client.0.smithi110.stdout:4/371: rename d17/d34/d3d/d20/f29 to d17/d34/d3d/d20/f80 0 2024-09-21T04:15:11.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/191: truncate d3/d14/d1a/f33 336133 0 2024-09-21T04:15:11.648 INFO:tasks.workunit.client.0.smithi110.stdout:8/275: truncate d0/fb 2759811 0 2024-09-21T04:15:11.653 INFO:tasks.workunit.client.0.smithi110.stdout:5/298: creat d1/d49/d4d/d1e/d5a/f67 x:0 0 0 2024-09-21T04:15:11.657 INFO:tasks.workunit.client.0.smithi110.stdout:9/187: mknod d7/d26/c40 0 2024-09-21T04:15:11.663 INFO:tasks.workunit.client.0.smithi110.stdout:3/278: mkdir d0/d12/d3c/d63 0 2024-09-21T04:15:11.665 INFO:tasks.workunit.client.1.smithi148.stdout:0/192: mkdir d5/d6/d2a/d3f 0 2024-09-21T04:15:11.665 INFO:tasks.workunit.client.1.smithi148.stdout:0/193: chown d5/d6/d2a 2 1 2024-09-21T04:15:11.665 INFO:tasks.workunit.client.0.smithi110.stdout:4/372: dwrite d17/f58 [0,4194304] 0 2024-09-21T04:15:11.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/210: dwrite d4/d23/d2a/f2f [0,4194304] 0 2024-09-21T04:15:11.666 INFO:tasks.workunit.client.0.smithi110.stdout:2/156: rename d2/d1a/de/d25 to d2/d1a/d1f/d29 0 2024-09-21T04:15:11.671 INFO:tasks.workunit.client.1.smithi148.stdout:3/178: truncate d3/d5/d6/d12/da/f11 599873 0 2024-09-21T04:15:11.672 INFO:tasks.workunit.client.0.smithi110.stdout:4/373: dread d17/d34/d3d/d20/f80 [0,4194304] 0 2024-09-21T04:15:11.672 INFO:tasks.workunit.client.0.smithi110.stdout:4/374: stat d17/d34/d3d/d20/d43/d51/d52/f59 0 2024-09-21T04:15:11.672 INFO:tasks.workunit.client.1.smithi148.stdout:3/179: dread d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:11.673 INFO:tasks.workunit.client.0.smithi110.stdout:7/238: symlink dd/d1c/l47 0 2024-09-21T04:15:11.673 INFO:tasks.workunit.client.0.smithi110.stdout:7/239: truncate dd/d14/f46 227310 0 2024-09-21T04:15:11.674 INFO:tasks.workunit.client.1.smithi148.stdout:5/192: mkdir d3/d14/d41 0 2024-09-21T04:15:11.677 INFO:tasks.workunit.client.0.smithi110.stdout:9/188: dwrite d7/d10/d24/d3c/f20 [0,4194304] 0 2024-09-21T04:15:11.683 INFO:tasks.workunit.client.0.smithi110.stdout:9/189: write d7/d10/d24/d3c/d1f/f23 [419538,72320] 0 2024-09-21T04:15:11.686 INFO:tasks.workunit.client.0.smithi110.stdout:7/240: read dd/d14/d17/f2b [1440425,38961] 0 2024-09-21T04:15:11.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/211: truncate d4/fb 3166905 0 2024-09-21T04:15:11.689 INFO:tasks.workunit.client.0.smithi110.stdout:8/276: mknod d0/d30/d34/d3e/c5f 0 2024-09-21T04:15:11.692 INFO:tasks.workunit.client.1.smithi148.stdout:3/180: rename d3/d5/d6/d12/da/d19/c24 to d3/d1f/c37 0 2024-09-21T04:15:11.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/202: sync 2024-09-21T04:15:11.693 INFO:tasks.workunit.client.0.smithi110.stdout:5/299: truncate d1/d39/f45 2849491 0 2024-09-21T04:15:11.694 INFO:tasks.workunit.client.0.smithi110.stdout:5/300: fdatasync d1/d39/f50 0 2024-09-21T04:15:11.694 INFO:tasks.workunit.client.0.smithi110.stdout:9/190: read d7/d10/d24/d3c/f20 [2902127,86119] 0 2024-09-21T04:15:11.694 INFO:tasks.workunit.client.0.smithi110.stdout:9/191: chown d7/d10/d24/f2a 12 1 2024-09-21T04:15:11.695 INFO:tasks.workunit.client.1.smithi148.stdout:5/193: creat d3/d14/d1f/f42 x:0 0 0 2024-09-21T04:15:11.696 INFO:tasks.workunit.client.1.smithi148.stdout:5/194: chown d3/d14/d1f/d26 399 1 2024-09-21T04:15:11.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/279: creat d0/d12/d3c/d5a/f64 x:0 0 0 2024-09-21T04:15:11.697 INFO:tasks.workunit.client.0.smithi110.stdout:3/280: stat d0/f56 0 2024-09-21T04:15:11.698 INFO:tasks.workunit.client.0.smithi110.stdout:7/241: write dd/d1c/d39/f3c [1248063,49294] 0 2024-09-21T04:15:11.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/157: rename d2/d1a/de to d2/d1a/d1f/d2a 0 2024-09-21T04:15:11.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/194: rmdir d5/d6/d2a/d2b 39 2024-09-21T04:15:11.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/195: chown f4 47381689 1 2024-09-21T04:15:11.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/212: creat d4/d12/d16/d29/f3d x:0 0 0 2024-09-21T04:15:11.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/213: dread - d4/fe zero size 2024-09-21T04:15:11.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/214: creat d4/d12/d16/d26/f3e x:0 0 0 2024-09-21T04:15:11.711 INFO:tasks.workunit.client.1.smithi148.stdout:8/189: sync 2024-09-21T04:15:11.711 INFO:tasks.workunit.client.1.smithi148.stdout:1/184: sync 2024-09-21T04:15:11.713 INFO:tasks.workunit.client.1.smithi148.stdout:0/196: symlink d5/d6/d7/l40 0 2024-09-21T04:15:11.713 INFO:tasks.workunit.client.1.smithi148.stdout:0/197: read - d5/d6/d7/d11/f29 zero size 2024-09-21T04:15:11.714 INFO:tasks.workunit.client.0.smithi110.stdout:5/301: mknod d1/d22/d51/c68 0 2024-09-21T04:15:11.716 INFO:tasks.workunit.client.0.smithi110.stdout:9/192: creat d7/d10/d24/d3c/d1f/f41 x:0 0 0 2024-09-21T04:15:11.719 INFO:tasks.workunit.client.1.smithi148.stdout:4/215: creat d4/d23/f3f x:0 0 0 2024-09-21T04:15:11.719 INFO:tasks.workunit.client.1.smithi148.stdout:4/216: creat d4/d23/f40 x:0 0 0 2024-09-21T04:15:11.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/242: mkdir dd/d48 0 2024-09-21T04:15:11.725 INFO:tasks.workunit.client.0.smithi110.stdout:4/375: rename d17/d3e/c62 to d17/c81 0 2024-09-21T04:15:11.729 INFO:tasks.workunit.client.0.smithi110.stdout:9/193: mknod d7/d10/d24/c42 0 2024-09-21T04:15:11.729 INFO:tasks.workunit.client.0.smithi110.stdout:9/194: read - d7/d10/f12 zero size 2024-09-21T04:15:11.730 INFO:tasks.workunit.client.1.smithi148.stdout:3/181: rmdir d3/d5 39 2024-09-21T04:15:11.731 INFO:tasks.workunit.client.1.smithi148.stdout:3/182: rename d3 to d3/d5/d6/d12/da/d23/d38 22 2024-09-21T04:15:11.731 INFO:tasks.workunit.client.1.smithi148.stdout:3/183: chown d3/d5/d6/d12/da/d23/f20 18547185 1 2024-09-21T04:15:11.731 INFO:tasks.workunit.client.1.smithi148.stdout:3/184: readlink l2 0 2024-09-21T04:15:11.731 INFO:tasks.workunit.client.0.smithi110.stdout:8/277: dread d0/d7/d13/d4b/f1f [0,4194304] 0 2024-09-21T04:15:11.734 INFO:tasks.workunit.client.1.smithi148.stdout:2/272: sync 2024-09-21T04:15:11.734 INFO:tasks.workunit.client.1.smithi148.stdout:6/145: sync 2024-09-21T04:15:11.734 INFO:tasks.workunit.client.1.smithi148.stdout:7/234: sync 2024-09-21T04:15:11.735 INFO:tasks.workunit.client.0.smithi110.stdout:3/281: rename d0/d12/d3c/d63 to d0/d12/d3c/d5a/d65 0 2024-09-21T04:15:11.735 INFO:tasks.workunit.client.0.smithi110.stdout:3/282: creat d0/d12/d16/d1a/d5c/f66 x:0 0 0 2024-09-21T04:15:11.735 INFO:tasks.workunit.client.0.smithi110.stdout:9/195: mkdir d7/d10/d24/d3c/d1f/d43 0 2024-09-21T04:15:11.735 INFO:tasks.workunit.client.0.smithi110.stdout:9/196: chown d7/d10/d24/d3c/d11 1229884 1 2024-09-21T04:15:11.736 INFO:tasks.workunit.client.1.smithi148.stdout:0/198: creat d5/d6/d2a/d3f/f41 x:0 0 0 2024-09-21T04:15:11.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/199: chown d5/d6/d7/d18/c30 75 1 2024-09-21T04:15:11.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/200: write d5/d6/d2a/d3f/f41 [295483,87339] 0 2024-09-21T04:15:11.737 INFO:tasks.workunit.client.0.smithi110.stdout:8/278: mkdir d0/d30/d34/d3e/d60 0 2024-09-21T04:15:11.737 INFO:tasks.workunit.client.0.smithi110.stdout:8/279: chown d0/d7/f5c 44 1 2024-09-21T04:15:11.741 INFO:tasks.workunit.client.1.smithi148.stdout:2/273: creat d0/d2b/d48/f58 x:0 0 0 2024-09-21T04:15:11.743 INFO:tasks.workunit.client.1.smithi148.stdout:6/146: creat d0/d8/f24 x:0 0 0 2024-09-21T04:15:11.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/203: dwrite f0 [0,4194304] 0 2024-09-21T04:15:11.745 INFO:tasks.workunit.client.1.smithi148.stdout:5/195: dwrite d3/f9 [8388608,4194304] 0 2024-09-21T04:15:11.745 INFO:tasks.workunit.client.0.smithi110.stdout:3/283: rename d0/d12/f52 to d0/d51/f67 0 2024-09-21T04:15:11.745 INFO:tasks.workunit.client.0.smithi110.stdout:3/284: creat d0/d12/d16/d1a/f68 x:0 0 0 2024-09-21T04:15:11.746 INFO:tasks.workunit.client.1.smithi148.stdout:5/196: write f0 [5225822,69577] 0 2024-09-21T04:15:11.746 INFO:tasks.workunit.client.1.smithi148.stdout:3/185: read d3/d5/d6/d12/da/d23/f20 [913629,79821] 0 2024-09-21T04:15:11.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/185: symlink db/l41 0 2024-09-21T04:15:11.746 INFO:tasks.workunit.client.0.smithi110.stdout:3/285: write d0/f10 [4395656,45151] 0 2024-09-21T04:15:11.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/280: truncate d0/d7/d13/d4b/f27 264716 0 2024-09-21T04:15:11.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/281: truncate d0/d30/f58 901454 0 2024-09-21T04:15:11.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/282: dread - d0/d7/f5c zero size 2024-09-21T04:15:11.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/283: fsync d0/d7/d13/d4b/f5b 0 2024-09-21T04:15:11.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/284: creat d0/d7/d13/d1c/f61 x:0 0 0 2024-09-21T04:15:11.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/190: dwrite dc/f21 [0,4194304] 0 2024-09-21T04:15:11.749 INFO:tasks.workunit.client.0.smithi110.stdout:2/158: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:0/201: symlink d5/d6/l42 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:2/274: dread d0/f23 [0,4194304] 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:7/235: write d5/d8/f1a [2585691,88844] 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:2/275: fsync d0/d9/f17 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:2/276: stat d0/d2b/f32 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:6/147: creat d0/d8/d18/f25 x:0 0 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:6/148: creat d0/f26 x:0 0 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/186: dread db/dd/f12 [0,4194304] 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:4/217: dwrite d4/d23/f40 [0,4194304] 0 2024-09-21T04:15:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:4/218: read - d4/d12/d16/d29/f3d zero size 2024-09-21T04:15:11.751 INFO:tasks.workunit.client.0.smithi110.stdout:7/243: fdatasync dd/d1c/d39/f3c 0 2024-09-21T04:15:11.751 INFO:tasks.workunit.client.0.smithi110.stdout:4/376: dwrite d17/d34/d3d/d20/d43/d51/d7a/f7f [0,4194304] 0 2024-09-21T04:15:11.753 INFO:tasks.workunit.client.0.smithi110.stdout:9/197: dwrite d7/d10/d24/f2a [0,4194304] 0 2024-09-21T04:15:11.754 INFO:tasks.workunit.client.1.smithi148.stdout:9/204: truncate d5/fe 1158233 0 2024-09-21T04:15:11.755 INFO:tasks.workunit.client.1.smithi148.stdout:0/202: write d5/d6/d2a/d2b/f35 [1169462,116521] 0 2024-09-21T04:15:11.755 INFO:tasks.workunit.client.1.smithi148.stdout:5/197: fsync f0 0 2024-09-21T04:15:11.755 INFO:tasks.workunit.client.1.smithi148.stdout:4/219: dread d4/f11 [0,4194304] 0 2024-09-21T04:15:11.755 INFO:tasks.workunit.client.1.smithi148.stdout:5/198: fsync d3/d14/f2d 0 2024-09-21T04:15:11.756 INFO:tasks.workunit.client.1.smithi148.stdout:5/199: write d3/dc/f3f [790046,51053] 0 2024-09-21T04:15:11.756 INFO:tasks.workunit.client.1.smithi148.stdout:5/200: creat d3/d24/f43 x:0 0 0 2024-09-21T04:15:11.757 INFO:tasks.workunit.client.1.smithi148.stdout:3/186: rename d3/d5/lb to d3/d5/d6/l39 0 2024-09-21T04:15:11.758 INFO:tasks.workunit.client.0.smithi110.stdout:8/285: chown d0/d7/d13/d4b/f40 254397971 1 2024-09-21T04:15:11.761 INFO:tasks.workunit.client.0.smithi110.stdout:3/286: dread d0/f7 [0,4194304] 0 2024-09-21T04:15:11.761 INFO:tasks.workunit.client.0.smithi110.stdout:3/287: dread - d0/d12/d16/d1a/f68 zero size 2024-09-21T04:15:11.761 INFO:tasks.workunit.client.0.smithi110.stdout:2/159: creat d2/d1a/d1f/d2a/d18/f2b x:0 0 0 2024-09-21T04:15:11.761 INFO:tasks.workunit.client.0.smithi110.stdout:2/160: write d2/d1a/d1f/d2a/f1b [545434,59310] 0 2024-09-21T04:15:11.764 INFO:tasks.workunit.client.0.smithi110.stdout:7/244: mknod dd/d14/c49 0 2024-09-21T04:15:11.766 INFO:tasks.workunit.client.1.smithi148.stdout:8/191: creat dc/d13/d39/f3b x:0 0 0 2024-09-21T04:15:11.767 INFO:tasks.workunit.client.1.smithi148.stdout:9/205: dread d5/f36 [0,4194304] 0 2024-09-21T04:15:11.770 INFO:tasks.workunit.client.1.smithi148.stdout:7/236: creat d5/dc/d28/f49 x:0 0 0 2024-09-21T04:15:11.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/237: fdatasync d5/d8/f1b 0 2024-09-21T04:15:11.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/238: write d5/d14/d25/f42 [268734,10872] 0 2024-09-21T04:15:11.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/239: chown d5/d8/f22 2258863 1 2024-09-21T04:15:11.772 INFO:tasks.workunit.client.0.smithi110.stdout:4/377: rename d17/d1e/d21/f31 to d17/d34/d3d/f82 0 2024-09-21T04:15:11.772 INFO:tasks.workunit.client.0.smithi110.stdout:4/378: stat d17/f1a 0 2024-09-21T04:15:11.773 INFO:tasks.workunit.client.1.smithi148.stdout:2/277: getdents d0/d9/dc/d13/d25/d2d 0 2024-09-21T04:15:11.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/278: read d0/d2f/f4a [241541,1625] 0 2024-09-21T04:15:11.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/279: read d0/d9/f17 [497492,54279] 0 2024-09-21T04:15:11.775 INFO:tasks.workunit.client.1.smithi148.stdout:4/220: dwrite d4/f11 [0,4194304] 0 2024-09-21T04:15:11.775 INFO:tasks.workunit.client.1.smithi148.stdout:4/221: write d4/d23/f3f [2439,32164] 0 2024-09-21T04:15:11.777 INFO:tasks.workunit.client.0.smithi110.stdout:2/161: dread d2/f8 [0,4194304] 0 2024-09-21T04:15:11.780 INFO:tasks.workunit.client.1.smithi148.stdout:6/149: mkdir d0/d1a/d23/d27 0 2024-09-21T04:15:11.781 INFO:tasks.workunit.client.0.smithi110.stdout:7/245: dread dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:11.781 INFO:tasks.workunit.client.0.smithi110.stdout:7/246: dread - dd/d14/f36 zero size 2024-09-21T04:15:11.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/288: write d0/d12/d2a/f31 [2880018,117463] 0 2024-09-21T04:15:11.783 INFO:tasks.workunit.client.0.smithi110.stdout:8/286: dwrite d0/d7/d13/f37 [0,4194304] 0 2024-09-21T04:15:11.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/187: unlink db/dd/c29 0 2024-09-21T04:15:11.790 INFO:tasks.workunit.client.0.smithi110.stdout:7/247: truncate dd/f2e 7403455 0 2024-09-21T04:15:11.794 INFO:tasks.workunit.client.0.smithi110.stdout:3/289: mknod d0/d12/d16/d1a/c69 0 2024-09-21T04:15:11.796 INFO:tasks.workunit.client.1.smithi148.stdout:5/201: rename d3/d14/d1f/d26/f32 to d3/d14/f44 0 2024-09-21T04:15:11.796 INFO:tasks.workunit.client.0.smithi110.stdout:8/287: creat d0/d30/d34/d3e/d5d/f62 x:0 0 0 2024-09-21T04:15:11.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/187: mknod d3/d5/d6/d12/da/d19/c3a 0 2024-09-21T04:15:11.798 INFO:tasks.workunit.client.0.smithi110.stdout:7/248: mkdir dd/d1c/d39/d4a 0 2024-09-21T04:15:11.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/192: mkdir dc/d13/d3c 0 2024-09-21T04:15:11.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/193: dread - dc/d13/f34 zero size 2024-09-21T04:15:11.801 INFO:tasks.workunit.client.0.smithi110.stdout:3/290: rename d0/f28 to d0/d12/d16/f6a 0 2024-09-21T04:15:11.801 INFO:tasks.workunit.client.0.smithi110.stdout:3/291: read d0/d12/d3c/f45 [2279126,24713] 0 2024-09-21T04:15:11.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/292: symlink d0/d12/d16/l6b 0 2024-09-21T04:15:11.805 INFO:tasks.workunit.client.0.smithi110.stdout:8/288: dread d0/d7/d13/d4b/f40 [0,4194304] 0 2024-09-21T04:15:11.806 INFO:tasks.workunit.client.1.smithi148.stdout:4/222: creat d4/d12/f41 x:0 0 0 2024-09-21T04:15:11.806 INFO:tasks.workunit.client.1.smithi148.stdout:4/223: fsync d4/d12/d16/d26/f35 0 2024-09-21T04:15:11.806 INFO:tasks.workunit.client.0.smithi110.stdout:3/293: getdents d0/d51 0 2024-09-21T04:15:11.806 INFO:tasks.workunit.client.0.smithi110.stdout:3/294: chown d0/d12/d2a/f3e 517 1 2024-09-21T04:15:11.807 INFO:tasks.workunit.client.1.smithi148.stdout:6/150: symlink d0/d8/l28 0 2024-09-21T04:15:11.808 INFO:tasks.workunit.client.1.smithi148.stdout:6/151: creat d0/d8/f29 x:0 0 0 2024-09-21T04:15:11.808 INFO:tasks.workunit.client.1.smithi148.stdout:6/152: truncate d0/d1a/d23/ff 4830922 0 2024-09-21T04:15:11.808 INFO:tasks.workunit.client.1.smithi148.stdout:6/153: dread - d0/f20 zero size 2024-09-21T04:15:11.808 INFO:tasks.workunit.client.0.smithi110.stdout:8/289: rename d0/d30/l38 to d0/d30/d54/l63 0 2024-09-21T04:15:11.810 INFO:tasks.workunit.client.1.smithi148.stdout:1/188: creat db/d24/d2a/f42 x:0 0 0 2024-09-21T04:15:11.810 INFO:tasks.workunit.client.1.smithi148.stdout:1/189: chown db/dd/f23 297271132 1 2024-09-21T04:15:11.811 INFO:tasks.workunit.client.0.smithi110.stdout:3/295: link d0/d12/d3c/f45 d0/d12/f6c 0 2024-09-21T04:15:11.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/296: dread d0/d12/f1b [0,4194304] 0 2024-09-21T04:15:11.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/297: chown d0/d12/d2a/f53 281592 1 2024-09-21T04:15:11.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/298: stat d0/d12/f6c 0 2024-09-21T04:15:11.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/299: readlink d0/ld 0 2024-09-21T04:15:11.813 INFO:tasks.workunit.client.1.smithi148.stdout:0/203: getdents d5/d6/d2a/d3f 0 2024-09-21T04:15:11.814 INFO:tasks.workunit.client.1.smithi148.stdout:0/204: fsync d5/d6/d7/d11/f29 0 2024-09-21T04:15:11.814 INFO:tasks.workunit.client.0.smithi110.stdout:8/290: getdents d0/d7/d23 0 2024-09-21T04:15:11.816 INFO:tasks.workunit.client.1.smithi148.stdout:9/206: rename d5/d22/d28/f43 to d5/d1e/d37/d50/f55 0 2024-09-21T04:15:11.816 INFO:tasks.workunit.client.1.smithi148.stdout:9/207: write d5/d22/d28/f30 [4012355,32000] 0 2024-09-21T04:15:11.816 INFO:tasks.workunit.client.1.smithi148.stdout:9/208: readlink d5/d22/d28/d38/l48 0 2024-09-21T04:15:11.817 INFO:tasks.workunit.client.0.smithi110.stdout:3/300: symlink d0/d51/l6d 0 2024-09-21T04:15:11.818 INFO:tasks.workunit.client.1.smithi148.stdout:5/202: mknod d3/d14/d41/c45 0 2024-09-21T04:15:11.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/188: mknod d3/d5/d6/d12/da/c3b 0 2024-09-21T04:15:11.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/189: readlink d3/d5/d6/d12/da/d19/l29 0 2024-09-21T04:15:11.821 INFO:tasks.workunit.client.1.smithi148.stdout:2/280: mkdir d0/d9/d59 0 2024-09-21T04:15:11.821 INFO:tasks.workunit.client.1.smithi148.stdout:2/281: write d0/d9/f17 [903329,88222] 0 2024-09-21T04:15:11.822 INFO:tasks.workunit.client.1.smithi148.stdout:2/282: dread - d0/d9/dc/d13/d2c/f33 zero size 2024-09-21T04:15:11.823 INFO:tasks.workunit.client.0.smithi110.stdout:3/301: rmdir d0/d12/d2a 39 2024-09-21T04:15:11.826 INFO:tasks.workunit.client.1.smithi148.stdout:6/154: dwrite d0/f20 [0,4194304] 0 2024-09-21T04:15:11.826 INFO:tasks.workunit.client.1.smithi148.stdout:6/155: fdatasync d0/d1a/d23/ff 0 2024-09-21T04:15:11.827 INFO:tasks.workunit.client.1.smithi148.stdout:4/224: symlink d4/d12/d16/d26/l42 0 2024-09-21T04:15:11.829 INFO:tasks.workunit.client.0.smithi110.stdout:7/249: dwrite dd/d14/d17/f2b [4194304,4194304] 0 2024-09-21T04:15:11.829 INFO:tasks.workunit.client.0.smithi110.stdout:8/291: write d0/d7/d13/f37 [644604,97134] 0 2024-09-21T04:15:11.830 INFO:tasks.workunit.client.0.smithi110.stdout:3/302: link d0/d12/d3c/c3d d0/c6e 0 2024-09-21T04:15:11.831 INFO:tasks.workunit.client.1.smithi148.stdout:1/190: creat db/dd/d21/f43 x:0 0 0 2024-09-21T04:15:11.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/205: truncate d5/d6/d7/d11/f1d 6329291 0 2024-09-21T04:15:11.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/206: readlink d5/d6/d7/l40 0 2024-09-21T04:15:11.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/207: dread - d5/d6/d7/d22/f37 zero size 2024-09-21T04:15:11.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/208: write d5/d6/d7/d22/f24 [4028989,120687] 0 2024-09-21T04:15:11.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/156: dread d0/f21 [0,4194304] 0 2024-09-21T04:15:11.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/157: fdatasync d0/d1a/d23/ff 0 2024-09-21T04:15:11.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/158: stat d0/d1a/d23/f14 0 2024-09-21T04:15:11.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/159: chown d0/d1a/f1c 1799 1 2024-09-21T04:15:11.836 INFO:tasks.workunit.client.1.smithi148.stdout:6/160: chown d0/d8/f24 2364008 1 2024-09-21T04:15:11.836 INFO:tasks.workunit.client.1.smithi148.stdout:6/161: getdents d0/d1a/d23/d27 0 2024-09-21T04:15:11.836 INFO:tasks.workunit.client.0.smithi110.stdout:7/250: truncate dd/d1c/f38 1376951 0 2024-09-21T04:15:11.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/292: mknod d0/d30/d34/d3e/d60/c64 0 2024-09-21T04:15:11.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/293: fdatasync d0/fb 0 2024-09-21T04:15:11.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/240: rename d5/d14/d35 to d5/dc/d28/d4a 0 2024-09-21T04:15:11.838 INFO:tasks.workunit.client.1.smithi148.stdout:7/241: chown d5/d8/f1b 40087672 1 2024-09-21T04:15:11.839 INFO:tasks.workunit.client.0.smithi110.stdout:3/303: mkdir d0/d12/d16/d1a/d6f 0 2024-09-21T04:15:11.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/209: getdents d5/d22/d28/d2f 0 2024-09-21T04:15:11.842 INFO:tasks.workunit.client.1.smithi148.stdout:5/203: truncate d3/d14/d1f/f23 1625052 0 2024-09-21T04:15:11.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/209: dread d5/d6/d2a/d2b/f35 [0,4194304] 0 2024-09-21T04:15:11.845 INFO:tasks.workunit.client.1.smithi148.stdout:3/190: mknod d3/d5/d6/d12/d15/c3c 0 2024-09-21T04:15:11.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/191: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:11.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/192: stat d3/d5/d6/l14 0 2024-09-21T04:15:11.848 INFO:tasks.workunit.client.0.smithi110.stdout:8/294: dread d0/fb [0,4194304] 0 2024-09-21T04:15:11.850 INFO:tasks.workunit.client.0.smithi110.stdout:7/251: read dd/d1c/f2c [962585,6352] 0 2024-09-21T04:15:11.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/304: mkdir d0/d12/d3c/d5a/d70 0 2024-09-21T04:15:11.858 INFO:tasks.workunit.client.0.smithi110.stdout:8/295: unlink d0/d7/d23/c2f 0 2024-09-21T04:15:11.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/252: creat dd/d48/f4b x:0 0 0 2024-09-21T04:15:11.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/253: fdatasync dd/d1c/f2c 0 2024-09-21T04:15:11.862 INFO:tasks.workunit.client.0.smithi110.stdout:3/305: mknod d0/d12/d16/d1a/c71 0 2024-09-21T04:15:11.864 INFO:tasks.workunit.client.0.smithi110.stdout:8/296: symlink d0/d7/d23/l65 0 2024-09-21T04:15:11.867 INFO:tasks.workunit.client.0.smithi110.stdout:7/254: link dd/d14/c21 dd/c4c 0 2024-09-21T04:15:11.868 INFO:tasks.workunit.client.0.smithi110.stdout:7/255: write dd/f3e [794591,42213] 0 2024-09-21T04:15:11.869 INFO:tasks.workunit.client.0.smithi110.stdout:8/297: rename d0/d30/f3b to d0/d30/d54/f66 0 2024-09-21T04:15:11.870 INFO:tasks.workunit.client.1.smithi148.stdout:9/210: dwrite d5/fc [0,4194304] 0 2024-09-21T04:15:11.870 INFO:tasks.workunit.client.1.smithi148.stdout:9/211: chown d5/d39 2825263 1 2024-09-21T04:15:11.870 INFO:tasks.workunit.client.1.smithi148.stdout:0/210: dwrite d5/d6/d7/ff [0,4194304] 0 2024-09-21T04:15:11.871 INFO:tasks.workunit.client.0.smithi110.stdout:7/256: mkdir dd/d4d 0 2024-09-21T04:15:11.873 INFO:tasks.workunit.client.0.smithi110.stdout:8/298: creat d0/d7/d13/d4b/f67 x:0 0 0 2024-09-21T04:15:11.873 INFO:tasks.workunit.client.0.smithi110.stdout:8/299: creat d0/d7/d13/d46/f68 x:0 0 0 2024-09-21T04:15:11.874 INFO:tasks.workunit.client.0.smithi110.stdout:8/300: write d0/d7/d13/d4b/f1f [4884308,105330] 0 2024-09-21T04:15:11.875 INFO:tasks.workunit.client.1.smithi148.stdout:4/225: unlink d4/d23/f3f 0 2024-09-21T04:15:11.875 INFO:tasks.workunit.client.1.smithi148.stdout:4/226: readlink d4/d12/d16/l1d 0 2024-09-21T04:15:11.875 INFO:tasks.workunit.client.1.smithi148.stdout:4/227: write d4/d12/d16/f19 [349510,19320] 0 2024-09-21T04:15:11.876 INFO:tasks.workunit.client.0.smithi110.stdout:7/257: link dd/d14/l2f dd/d1c/d39/d4a/l4e 0 2024-09-21T04:15:11.877 INFO:tasks.workunit.client.1.smithi148.stdout:1/191: symlink db/d24/d2e/l44 0 2024-09-21T04:15:11.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/162: mkdir d0/d1a/d23/d27/d2a 0 2024-09-21T04:15:11.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/163: dread - d0/d1a/d23/d16/f1f zero size 2024-09-21T04:15:11.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/164: stat d0/d8/f29 0 2024-09-21T04:15:11.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/165: read - d0/d8/f24 zero size 2024-09-21T04:15:11.881 INFO:tasks.workunit.client.1.smithi148.stdout:8/194: rename dc/cd to dc/d2a/c3d 0 2024-09-21T04:15:11.882 INFO:tasks.workunit.client.0.smithi110.stdout:8/301: creat d0/d7/d23/f69 x:0 0 0 2024-09-21T04:15:11.884 INFO:tasks.workunit.client.1.smithi148.stdout:7/242: unlink d5/ca 0 2024-09-21T04:15:11.884 INFO:tasks.workunit.client.1.smithi148.stdout:7/243: creat d5/d14/f4b x:0 0 0 2024-09-21T04:15:11.884 INFO:tasks.workunit.client.1.smithi148.stdout:1/192: dread db/ff [0,4194304] 0 2024-09-21T04:15:11.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/193: chown db/dd/d21/l31 3 1 2024-09-21T04:15:11.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/194: dread db/d24/f26 [0,4194304] 0 2024-09-21T04:15:11.885 INFO:tasks.workunit.client.1.smithi148.stdout:5/204: mknod d3/d14/d1f/d26/c46 0 2024-09-21T04:15:11.887 INFO:tasks.workunit.client.0.smithi110.stdout:7/258: link dd/d1c/d39/f3b dd/d1c/d39/f4f 0 2024-09-21T04:15:11.887 INFO:tasks.workunit.client.0.smithi110.stdout:7/259: chown dd/c3d 1099 1 2024-09-21T04:15:11.890 INFO:tasks.workunit.client.0.smithi110.stdout:8/302: symlink d0/d7/d23/d52/l6a 0 2024-09-21T04:15:11.892 INFO:tasks.workunit.client.0.smithi110.stdout:1/518: sync 2024-09-21T04:15:11.892 INFO:tasks.workunit.client.0.smithi110.stdout:6/234: sync 2024-09-21T04:15:11.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/515: sync 2024-09-21T04:15:11.892 INFO:tasks.workunit.client.0.smithi110.stdout:6/235: truncate d4/d12/f34 1332147 0 2024-09-21T04:15:11.893 INFO:tasks.workunit.client.0.smithi110.stdout:6/236: read - d4/d12/d17/d2e/f47 zero size 2024-09-21T04:15:11.893 INFO:tasks.workunit.client.0.smithi110.stdout:6/237: read d4/d12/f1d [3740711,97185] 0 2024-09-21T04:15:11.895 INFO:tasks.workunit.client.1.smithi148.stdout:4/228: symlink d4/l43 0 2024-09-21T04:15:11.897 INFO:tasks.workunit.client.1.smithi148.stdout:6/166: symlink d0/d1a/d23/l2b 0 2024-09-21T04:15:11.898 INFO:tasks.workunit.client.1.smithi148.stdout:6/167: read - d0/d8/d18/f25 zero size 2024-09-21T04:15:11.898 INFO:tasks.workunit.client.1.smithi148.stdout:6/168: truncate d0/d1a/d23/d16/f1f 589197 0 2024-09-21T04:15:11.901 INFO:tasks.workunit.client.1.smithi148.stdout:8/195: mkdir dc/d3e 0 2024-09-21T04:15:11.901 INFO:tasks.workunit.client.1.smithi148.stdout:8/196: fsync dc/f14 0 2024-09-21T04:15:11.901 INFO:tasks.workunit.client.1.smithi148.stdout:8/197: fdatasync dc/f31 0 2024-09-21T04:15:11.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/516: symlink d4/d1c/d62/d91/l9c 0 2024-09-21T04:15:11.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/517: dread - d4/d47/d2c/f88 zero size 2024-09-21T04:15:11.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/518: chown d4/f56 243567 1 2024-09-21T04:15:11.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/519: write d4/dd/f1b [4428684,5410] 0 2024-09-21T04:15:11.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/520: write d4/f11 [3295468,365] 0 2024-09-21T04:15:11.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/521: readlink d4/dd/l3e 0 2024-09-21T04:15:11.903 INFO:tasks.workunit.client.1.smithi148.stdout:3/193: rename d3/d5/d6/d12/f1a to d3/d5/d6/d12/da/f3d 0 2024-09-21T04:15:11.903 INFO:tasks.workunit.client.1.smithi148.stdout:3/194: chown d3/l2e 1889328 1 2024-09-21T04:15:11.903 INFO:tasks.workunit.client.1.smithi148.stdout:3/195: readlink d3/d5/d6/d12/da/d19/l2a 0 2024-09-21T04:15:11.904 INFO:tasks.workunit.client.0.smithi110.stdout:0/522: dread d4/d47/d2c/d3a/f43 [0,4194304] 0 2024-09-21T04:15:11.904 INFO:tasks.workunit.client.0.smithi110.stdout:6/238: truncate d4/de/f1f 3185321 0 2024-09-21T04:15:11.906 INFO:tasks.workunit.client.0.smithi110.stdout:1/519: rmdir d2/d9/d49/d5c/d91 39 2024-09-21T04:15:11.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/169: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:15:11.908 INFO:tasks.workunit.client.0.smithi110.stdout:7/260: dwrite dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:11.909 INFO:tasks.workunit.client.0.smithi110.stdout:8/303: dwrite d0/d7/d13/d4b/f4e [0,4194304] 0 2024-09-21T04:15:11.911 INFO:tasks.workunit.client.0.smithi110.stdout:0/523: write f2 [4102469,21969] 0 2024-09-21T04:15:11.914 INFO:tasks.workunit.client.0.smithi110.stdout:6/239: creat d4/d12/d16/f4d x:0 0 0 2024-09-21T04:15:11.916 INFO:tasks.workunit.client.1.smithi148.stdout:5/205: creat d3/d34/f47 x:0 0 0 2024-09-21T04:15:11.919 INFO:tasks.workunit.client.0.smithi110.stdout:8/304: dread d0/d7/d13/d4b/f4e [0,4194304] 0 2024-09-21T04:15:11.919 INFO:tasks.workunit.client.0.smithi110.stdout:7/261: truncate dd/d14/f20 1078203 0 2024-09-21T04:15:11.919 INFO:tasks.workunit.client.0.smithi110.stdout:7/262: chown dd/f31 29270 1 2024-09-21T04:15:11.920 INFO:tasks.workunit.client.1.smithi148.stdout:4/229: truncate d4/d23/f40 2813608 0 2024-09-21T04:15:11.923 INFO:tasks.workunit.client.1.smithi148.stdout:8/198: mknod dc/d2a/c3f 0 2024-09-21T04:15:11.923 INFO:tasks.workunit.client.1.smithi148.stdout:8/199: creat dc/d2a/f40 x:0 0 0 2024-09-21T04:15:11.923 INFO:tasks.workunit.client.1.smithi148.stdout:8/200: stat f6 0 2024-09-21T04:15:11.924 INFO:tasks.workunit.client.0.smithi110.stdout:0/524: link d4/d47/d2c/d3a/f51 d4/dd/d10/d29/d75/f9d 0 2024-09-21T04:15:11.927 INFO:tasks.workunit.client.1.smithi148.stdout:9/212: rename d5/d1f/c35 to d5/d2b/d4e/c56 0 2024-09-21T04:15:11.929 INFO:tasks.workunit.client.0.smithi110.stdout:1/520: dwrite d2/d9/d23/d3b/d5e/d86/f89 [0,4194304] 0 2024-09-21T04:15:11.930 INFO:tasks.workunit.client.0.smithi110.stdout:6/240: rename d4/d12/d16/f4d to d4/f4e 0 2024-09-21T04:15:11.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/170: creat d0/d1a/d23/d16/f2c x:0 0 0 2024-09-21T04:15:11.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/171: chown d0/d1a/d23/f14 3011 1 2024-09-21T04:15:11.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/172: write d0/d1a/d23/f12 [378757,25225] 0 2024-09-21T04:15:11.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/244: rmdir d5/dc/d24 39 2024-09-21T04:15:11.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/245: creat d5/d14/d25/f4c x:0 0 0 2024-09-21T04:15:11.937 INFO:tasks.workunit.client.0.smithi110.stdout:7/263: mknod dd/d1c/d39/c50 0 2024-09-21T04:15:11.940 INFO:tasks.workunit.client.0.smithi110.stdout:8/305: dwrite d0/fb [0,4194304] 0 2024-09-21T04:15:11.941 INFO:tasks.workunit.client.1.smithi148.stdout:1/195: rmdir db/dd 39 2024-09-21T04:15:11.941 INFO:tasks.workunit.client.1.smithi148.stdout:1/196: readlink db/d24/d2e/l3d 0 2024-09-21T04:15:11.942 INFO:tasks.workunit.client.1.smithi148.stdout:1/197: creat db/d24/d2a/f45 x:0 0 0 2024-09-21T04:15:11.942 INFO:tasks.workunit.client.1.smithi148.stdout:1/198: chown db/dd/c32 350 1 2024-09-21T04:15:11.942 INFO:tasks.workunit.client.1.smithi148.stdout:1/199: fsync db/d24/d2a/f42 0 2024-09-21T04:15:11.943 INFO:tasks.workunit.client.1.smithi148.stdout:5/206: creat d3/d34/d3c/f48 x:0 0 0 2024-09-21T04:15:11.945 INFO:tasks.workunit.client.0.smithi110.stdout:1/521: rmdir d2/d9/d23 39 2024-09-21T04:15:11.948 INFO:tasks.workunit.client.0.smithi110.stdout:6/241: creat d4/d12/d17/d2e/d38/f4f x:0 0 0 2024-09-21T04:15:11.950 INFO:tasks.workunit.client.1.smithi148.stdout:4/230: mkdir d4/d12/d2c/d2d/d44 0 2024-09-21T04:15:11.951 INFO:tasks.workunit.client.0.smithi110.stdout:0/525: dwrite d4/dd/d60/f7e [0,4194304] 0 2024-09-21T04:15:11.951 INFO:tasks.workunit.client.0.smithi110.stdout:7/264: dread dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:11.952 INFO:tasks.workunit.client.0.smithi110.stdout:6/242: truncate d4/f13 3693469 0 2024-09-21T04:15:11.952 INFO:tasks.workunit.client.0.smithi110.stdout:6/243: write d4/f3d [77436,95891] 0 2024-09-21T04:15:11.952 INFO:tasks.workunit.client.0.smithi110.stdout:6/244: fsync d4/fc 0 2024-09-21T04:15:11.952 INFO:tasks.workunit.client.0.smithi110.stdout:6/245: creat d4/d12/d16/f50 x:0 0 0 2024-09-21T04:15:11.954 INFO:tasks.workunit.client.0.smithi110.stdout:8/306: dread d0/fb [0,4194304] 0 2024-09-21T04:15:11.956 INFO:tasks.workunit.client.1.smithi148.stdout:6/173: dwrite d0/d1a/d23/d16/f1f [0,4194304] 0 2024-09-21T04:15:11.956 INFO:tasks.workunit.client.1.smithi148.stdout:9/213: rmdir d5/d2b/d4e 39 2024-09-21T04:15:11.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/214: read d5/d1f/f24 [380686,11977] 0 2024-09-21T04:15:11.958 INFO:tasks.workunit.client.0.smithi110.stdout:0/526: unlink d4/f3b 0 2024-09-21T04:15:11.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/522: write d2/d9/d23/d3b/d5e/d86/f89 [2482610,32383] 0 2024-09-21T04:15:11.961 INFO:tasks.workunit.client.0.smithi110.stdout:7/265: link dd/d14/d17/l1f dd/d1c/l51 0 2024-09-21T04:15:11.963 INFO:tasks.workunit.client.1.smithi148.stdout:7/246: symlink d5/dc/d28/d4a/l4d 0 2024-09-21T04:15:11.966 INFO:tasks.workunit.client.0.smithi110.stdout:8/307: mkdir d0/d30/d54/d6b 0 2024-09-21T04:15:11.966 INFO:tasks.workunit.client.0.smithi110.stdout:8/308: readlink d0/l12 0 2024-09-21T04:15:11.968 INFO:tasks.workunit.client.0.smithi110.stdout:7/266: dread dd/d1c/f2c [0,4194304] 0 2024-09-21T04:15:11.970 INFO:tasks.workunit.client.1.smithi148.stdout:5/207: creat d3/d14/d41/f49 x:0 0 0 2024-09-21T04:15:11.970 INFO:tasks.workunit.client.1.smithi148.stdout:5/208: write d3/d24/f2f [1521210,7906] 0 2024-09-21T04:15:11.970 INFO:tasks.workunit.client.1.smithi148.stdout:5/209: truncate d3/d14/d41/f49 657153 0 2024-09-21T04:15:11.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/283: sync 2024-09-21T04:15:11.972 INFO:tasks.workunit.client.0.smithi110.stdout:7/267: dread dd/d1c/f38 [0,4194304] 0 2024-09-21T04:15:11.973 INFO:tasks.workunit.client.1.smithi148.stdout:4/231: rename d4/c3c to d4/d12/d2c/d2d/d44/c45 0 2024-09-21T04:15:11.973 INFO:tasks.workunit.client.1.smithi148.stdout:4/232: truncate d4/d12/d2c/f32 5677936 0 2024-09-21T04:15:11.973 INFO:tasks.workunit.client.1.smithi148.stdout:4/233: write d4/d12/f1b [2085365,8042] 0 2024-09-21T04:15:11.973 INFO:tasks.workunit.client.1.smithi148.stdout:4/234: creat d4/d12/d16/d29/f46 x:0 0 0 2024-09-21T04:15:11.973 INFO:tasks.workunit.client.1.smithi148.stdout:4/235: read - d4/d12/d16/f34 zero size 2024-09-21T04:15:11.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/523: rmdir d2/d9/d49/d4f/d87 39 2024-09-21T04:15:11.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/524: write d2/d9/d49/f5b [3110968,3688] 0 2024-09-21T04:15:11.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/247: read d5/d8/f16 [4893046,40531] 0 2024-09-21T04:15:11.975 INFO:tasks.workunit.client.0.smithi110.stdout:8/309: dread d0/d7/d13/d46/f51 [0,4194304] 0 2024-09-21T04:15:11.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/525: truncate d2/d9/d23/f82 1544330 0 2024-09-21T04:15:11.976 INFO:tasks.workunit.client.0.smithi110.stdout:7/268: write dd/f3e [218505,58424] 0 2024-09-21T04:15:11.976 INFO:tasks.workunit.client.0.smithi110.stdout:7/269: write dd/d14/f41 [1024038,95136] 0 2024-09-21T04:15:11.979 INFO:tasks.workunit.client.1.smithi148.stdout:8/201: rmdir dc 39 2024-09-21T04:15:11.984 INFO:tasks.workunit.client.1.smithi148.stdout:6/174: unlink d0/f26 0 2024-09-21T04:15:11.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/200: mkdir db/d46 0 2024-09-21T04:15:11.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/246: dwrite d4/f3d [0,4194304] 0 2024-09-21T04:15:11.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/247: stat d4/d12/d16/f50 0 2024-09-21T04:15:11.987 INFO:tasks.workunit.client.0.smithi110.stdout:8/310: write d0/d7/d13/d4b/f17 [1827678,113580] 0 2024-09-21T04:15:11.987 INFO:tasks.workunit.client.0.smithi110.stdout:8/311: chown d0/d7/l48 247 1 2024-09-21T04:15:11.987 INFO:tasks.workunit.client.1.smithi148.stdout:5/210: truncate f0 144963 0 2024-09-21T04:15:11.988 INFO:tasks.workunit.client.1.smithi148.stdout:5/211: readlink d3/d14/d1f/d26/l38 0 2024-09-21T04:15:11.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/211: sync 2024-09-21T04:15:11.991 INFO:tasks.workunit.client.1.smithi148.stdout:2/284: creat d0/d9/d59/f5a x:0 0 0 2024-09-21T04:15:11.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/527: dwrite f2 [0,4194304] 0 2024-09-21T04:15:11.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/528: creat d4/dd/d10/f9e x:0 0 0 2024-09-21T04:15:11.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/529: write d4/dd/f1b [5958009,66940] 0 2024-09-21T04:15:11.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/530: creat d4/d47/d2c/d3a/f9f x:0 0 0 2024-09-21T04:15:11.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/531: chown d4/d8b 228 1 2024-09-21T04:15:11.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/215: rename d5/d39/l3d to d5/d22/d4c/l57 0 2024-09-21T04:15:11.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/216: read - d5/d22/d28/f51 zero size 2024-09-21T04:15:11.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/217: fsync d5/d1e/f49 0 2024-09-21T04:15:12.027 INFO:tasks.workunit.client.1.smithi148.stdout:3/196: sync 2024-09-21T04:15:12.027 INFO:tasks.workunit.client.0.smithi110.stdout:8/312: mkdir d0/d7/d23/d6c 0 2024-09-21T04:15:12.028 INFO:tasks.workunit.client.0.smithi110.stdout:8/313: read d0/d7/d13/d4b/f27 [108549,26304] 0 2024-09-21T04:15:12.028 INFO:tasks.workunit.client.0.smithi110.stdout:8/314: read - d0/d7/f5c zero size 2024-09-21T04:15:12.028 INFO:tasks.workunit.client.1.smithi148.stdout:4/236: mknod d4/d12/d16/d29/c47 0 2024-09-21T04:15:12.030 INFO:tasks.workunit.client.1.smithi148.stdout:7/248: symlink d5/d14/d25/l4e 0 2024-09-21T04:15:12.031 INFO:tasks.workunit.client.1.smithi148.stdout:6/175: creat d0/d1a/d23/d27/f2d x:0 0 0 2024-09-21T04:15:12.033 INFO:tasks.workunit.client.1.smithi148.stdout:1/201: creat db/d46/f47 x:0 0 0 2024-09-21T04:15:12.033 INFO:tasks.workunit.client.1.smithi148.stdout:1/202: write db/d24/d2a/f45 [304014,24130] 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/532: mknod d4/d47/d2c/d3a/d84/ca0 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/533: write d4/dd/d10/f9e [362446,91138] 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/534: write d4/d47/f23 [1502658,76094] 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/535: write d4/dd/f8e [844987,124801] 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/536: readlink d4/d47/l6a 0 2024-09-21T04:15:12.034 INFO:tasks.workunit.client.0.smithi110.stdout:0/537: readlink d4/dd/d10/l7c 0 2024-09-21T04:15:12.039 INFO:tasks.workunit.client.1.smithi148.stdout:3/197: creat d3/d1f/f3e x:0 0 0 2024-09-21T04:15:12.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/315: mknod d0/d30/d34/d3e/d5d/c6d 0 2024-09-21T04:15:12.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/316: readlink d0/d7/d13/l2c 0 2024-09-21T04:15:12.041 INFO:tasks.workunit.client.1.smithi148.stdout:7/249: mknod d5/dc/d28/d4a/c4f 0 2024-09-21T04:15:12.041 INFO:tasks.workunit.client.1.smithi148.stdout:7/250: chown d5/l32 158077 1 2024-09-21T04:15:12.041 INFO:tasks.workunit.client.1.smithi148.stdout:2/285: dwrite d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/538: rename d4/dd/d10/f9e to d4/dd/d10/fa1 0 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/539: chown d4/d1c/d62/c49 628589 1 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/540: getdents d4/d1c/d62/d69/d9b 0 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/541: chown d4/dd/d10/d29/f5d 1558454 1 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/542: fsync d4/dd/d10/d29/f5d 0 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/543: creat d4/dd/d53/fa2 x:0 0 0 2024-09-21T04:15:12.042 INFO:tasks.workunit.client.1.smithi148.stdout:6/176: mknod d0/d1a/d23/d27/c2e 0 2024-09-21T04:15:12.043 INFO:tasks.workunit.client.1.smithi148.stdout:5/212: mkdir d3/d4a 0 2024-09-21T04:15:12.043 INFO:tasks.workunit.client.1.smithi148.stdout:5/213: creat d3/d14/d41/f4b x:0 0 0 2024-09-21T04:15:12.043 INFO:tasks.workunit.client.0.smithi110.stdout:1/526: dread d2/d9/d26/d3d/f58 [0,4194304] 0 2024-09-21T04:15:12.043 INFO:tasks.workunit.client.0.smithi110.stdout:1/527: creat d2/d9/d23/d3b/d5e/d86/faf x:0 0 0 2024-09-21T04:15:12.045 INFO:tasks.workunit.client.0.smithi110.stdout:7/270: dwrite dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:12.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/218: dread d5/d1e/f49 [0,4194304] 0 2024-09-21T04:15:12.048 INFO:tasks.workunit.client.1.smithi148.stdout:8/202: dwrite fb [0,4194304] 0 2024-09-21T04:15:12.049 INFO:tasks.workunit.client.0.smithi110.stdout:0/544: symlink d4/dd/d10/d29/la3 0 2024-09-21T04:15:12.050 INFO:tasks.workunit.client.0.smithi110.stdout:7/271: dread dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:12.050 INFO:tasks.workunit.client.0.smithi110.stdout:7/272: chown dd/d14/l2f 270440056 1 2024-09-21T04:15:12.050 INFO:tasks.workunit.client.0.smithi110.stdout:7/273: chown dd/d1c/l51 38 1 2024-09-21T04:15:12.050 INFO:tasks.workunit.client.0.smithi110.stdout:7/274: truncate dd/f3e 1452840 0 2024-09-21T04:15:12.050 INFO:tasks.workunit.client.0.smithi110.stdout:1/528: truncate d2/d9/d26/d3d/f43 917071 0 2024-09-21T04:15:12.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/237: dread d4/f6 [0,4194304] 0 2024-09-21T04:15:12.053 INFO:tasks.workunit.client.1.smithi148.stdout:0/212: rmdir d5/d6/d7 39 2024-09-21T04:15:12.053 INFO:tasks.workunit.client.1.smithi148.stdout:7/251: stat d5/dc/d24/f2b 0 2024-09-21T04:15:12.053 INFO:tasks.workunit.client.1.smithi148.stdout:7/252: fdatasync d5/f3d 0 2024-09-21T04:15:12.058 INFO:tasks.workunit.client.0.smithi110.stdout:7/275: write dd/d1c/f38 [401900,104268] 0 2024-09-21T04:15:12.058 INFO:tasks.workunit.client.0.smithi110.stdout:7/276: readlink dd/d14/d17/l1f 0 2024-09-21T04:15:12.059 INFO:tasks.workunit.client.1.smithi148.stdout:2/286: dread d0/d9/dc/d13/d25/d2d/f2e [0,4194304] 0 2024-09-21T04:15:12.059 INFO:tasks.workunit.client.1.smithi148.stdout:2/287: creat d0/d9/d16/f5b x:0 0 0 2024-09-21T04:15:12.060 INFO:tasks.workunit.client.1.smithi148.stdout:7/253: dread d5/d8/f22 [0,4194304] 0 2024-09-21T04:15:12.060 INFO:tasks.workunit.client.1.smithi148.stdout:1/203: dwrite db/f27 [0,4194304] 0 2024-09-21T04:15:12.064 INFO:tasks.workunit.client.1.smithi148.stdout:5/214: mknod d3/d24/c4c 0 2024-09-21T04:15:12.069 INFO:tasks.workunit.client.1.smithi148.stdout:9/219: creat d5/d2b/d32/f58 x:0 0 0 2024-09-21T04:15:12.071 INFO:tasks.workunit.client.1.smithi148.stdout:0/213: creat d5/d6/d7/f43 x:0 0 0 2024-09-21T04:15:12.071 INFO:tasks.workunit.client.1.smithi148.stdout:0/214: write d5/d6/d2a/d3f/f41 [1179817,8171] 0 2024-09-21T04:15:12.074 INFO:tasks.workunit.client.1.smithi148.stdout:6/177: dwrite d0/f20 [0,4194304] 0 2024-09-21T04:15:12.077 INFO:tasks.workunit.client.1.smithi148.stdout:7/254: write d5/dc/f19 [2550527,108188] 0 2024-09-21T04:15:12.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/529: unlink d2/f8 0 2024-09-21T04:15:12.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/530: readlink d2/d9/d23/d4e/l71 0 2024-09-21T04:15:12.078 INFO:tasks.workunit.client.0.smithi110.stdout:8/317: dwrite d0/d7/f5c [0,4194304] 0 2024-09-21T04:15:12.079 INFO:tasks.workunit.client.0.smithi110.stdout:8/318: fdatasync d0/d7/d13/d4b/f4e 0 2024-09-21T04:15:12.079 INFO:tasks.workunit.client.0.smithi110.stdout:8/319: chown d0/d7/f1b 210 1 2024-09-21T04:15:12.079 INFO:tasks.workunit.client.1.smithi148.stdout:2/288: write d0/f11 [2559231,22098] 0 2024-09-21T04:15:12.079 INFO:tasks.workunit.client.1.smithi148.stdout:4/238: dwrite d4/d12/d16/d26/f3e [0,4194304] 0 2024-09-21T04:15:12.080 INFO:tasks.workunit.client.0.smithi110.stdout:7/277: mkdir dd/d1c/d39/d4a/d52 0 2024-09-21T04:15:12.081 INFO:tasks.workunit.client.1.smithi148.stdout:3/198: dwrite d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:12.084 INFO:tasks.workunit.client.1.smithi148.stdout:1/204: rename db/f37 to db/d46/f48 0 2024-09-21T04:15:12.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/215: dwrite d3/f7 [0,4194304] 0 2024-09-21T04:15:12.085 INFO:tasks.workunit.client.0.smithi110.stdout:8/320: dread d0/d7/fd [4194304,4194304] 0 2024-09-21T04:15:12.085 INFO:tasks.workunit.client.0.smithi110.stdout:8/321: fdatasync d0/d7/d13/d1c/d41/f43 0 2024-09-21T04:15:12.088 INFO:tasks.workunit.client.0.smithi110.stdout:0/545: getdents d4/d47/d2c/d3a 0 2024-09-21T04:15:12.090 INFO:tasks.workunit.client.1.smithi148.stdout:5/216: dread d3/dc/f19 [0,4194304] 0 2024-09-21T04:15:12.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/220: link d5/l17 d5/d1f/l59 0 2024-09-21T04:15:12.091 INFO:tasks.workunit.client.0.smithi110.stdout:7/278: dread dd/d1c/f34 [0,4194304] 0 2024-09-21T04:15:12.091 INFO:tasks.workunit.client.0.smithi110.stdout:7/279: chown dd 10492 1 2024-09-21T04:15:12.091 INFO:tasks.workunit.client.0.smithi110.stdout:7/280: chown dd/d14/d17/f2b 8445800 1 2024-09-21T04:15:12.092 INFO:tasks.workunit.client.0.smithi110.stdout:7/281: write dd/f3e [2263553,101194] 0 2024-09-21T04:15:12.092 INFO:tasks.workunit.client.0.smithi110.stdout:7/282: truncate dd/d14/f36 412823 0 2024-09-21T04:15:12.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/531: symlink d2/d9/d23/d3b/d5e/lb0 0 2024-09-21T04:15:12.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/532: fsync d2/d9/d23/d3b/d5e/d86/faf 0 2024-09-21T04:15:12.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/302: sync 2024-09-21T04:15:12.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/303: truncate d1/dc/f33 716353 0 2024-09-21T04:15:12.099 INFO:tasks.workunit.client.1.smithi148.stdout:2/289: dwrite d0/d2f/f4a [0,4194304] 0 2024-09-21T04:15:12.099 INFO:tasks.workunit.client.1.smithi148.stdout:2/290: creat d0/d2b/d48/f5c x:0 0 0 2024-09-21T04:15:12.100 INFO:tasks.workunit.client.1.smithi148.stdout:0/215: getdents d5/d6/d7/d22 0 2024-09-21T04:15:12.100 INFO:tasks.workunit.client.1.smithi148.stdout:0/216: chown d5/d6/f1f 5450309 1 2024-09-21T04:15:12.102 INFO:tasks.workunit.client.0.smithi110.stdout:7/283: symlink dd/d14/l53 0 2024-09-21T04:15:12.102 INFO:tasks.workunit.client.0.smithi110.stdout:7/284: creat dd/d14/d17/f54 x:0 0 0 2024-09-21T04:15:12.102 INFO:tasks.workunit.client.0.smithi110.stdout:1/533: truncate d2/d9/d26/f34 3668074 0 2024-09-21T04:15:12.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/221: dread d5/d22/d28/f41 [0,4194304] 0 2024-09-21T04:15:12.106 INFO:tasks.workunit.client.0.smithi110.stdout:5/304: dread d1/d49/d4d/f1b [0,4194304] 0 2024-09-21T04:15:12.107 INFO:tasks.workunit.client.1.smithi148.stdout:2/291: dread d0/d9/d38/f4f [0,4194304] 0 2024-09-21T04:15:12.109 INFO:tasks.workunit.client.1.smithi148.stdout:7/255: creat d5/dc/f50 x:0 0 0 2024-09-21T04:15:12.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/285: mkdir dd/d48/d55 0 2024-09-21T04:15:12.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/286: stat dd/f3e 0 2024-09-21T04:15:12.112 INFO:tasks.workunit.client.0.smithi110.stdout:1/534: creat d2/d9/d49/fb1 x:0 0 0 2024-09-21T04:15:12.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/239: symlink d4/d12/d2c/d2d/l48 0 2024-09-21T04:15:12.117 INFO:tasks.workunit.client.0.smithi110.stdout:7/287: write dd/f2e [6577875,107004] 0 2024-09-21T04:15:12.118 INFO:tasks.workunit.client.1.smithi148.stdout:9/222: read d5/d22/d28/f30 [1707080,87168] 0 2024-09-21T04:15:12.118 INFO:tasks.workunit.client.1.smithi148.stdout:9/223: creat d5/d2b/d32/f5a x:0 0 0 2024-09-21T04:15:12.121 INFO:tasks.workunit.client.1.smithi148.stdout:3/199: rename d3/d5/d6/l14 to d3/d5/d6/d12/l3f 0 2024-09-21T04:15:12.121 INFO:tasks.workunit.client.1.smithi148.stdout:3/200: dread - d3/d5/f35 zero size 2024-09-21T04:15:12.125 INFO:tasks.workunit.client.0.smithi110.stdout:8/322: dwrite d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:12.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/217: truncate d5/d6/f1f 485543 0 2024-09-21T04:15:12.129 INFO:tasks.workunit.client.1.smithi148.stdout:6/178: rmdir d0/d8 39 2024-09-21T04:15:12.131 INFO:tasks.workunit.client.1.smithi148.stdout:2/292: getdents d0 0 2024-09-21T04:15:12.132 INFO:tasks.workunit.client.0.smithi110.stdout:8/323: truncate d0/d7/f1b 3979823 0 2024-09-21T04:15:12.132 INFO:tasks.workunit.client.0.smithi110.stdout:8/324: chown d0/d7/d13/d1c/l53 34953 1 2024-09-21T04:15:12.132 INFO:tasks.workunit.client.0.smithi110.stdout:0/546: dwrite d4/d1c/f8d [0,4194304] 0 2024-09-21T04:15:12.133 INFO:tasks.workunit.client.0.smithi110.stdout:1/535: write d2/d9/d23/d3b/d93/fab [3705402,90068] 0 2024-09-21T04:15:12.133 INFO:tasks.workunit.client.0.smithi110.stdout:1/536: chown d2/d9/d49/d4f/d53/l78 221494 1 2024-09-21T04:15:12.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/240: getdents d4/d12/d2c/d2d/d44 0 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/217: dwrite d3/d14/d1a/f31 [4194304,4194304] 0 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/218: write d3/dc/f17 [2309115,63285] 0 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/219: chown d3/d14/d1f/c2c 6 1 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/220: creat d3/d14/d1a/f4d x:0 0 0 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/221: fdatasync d3/f7 0 2024-09-21T04:15:12.139 INFO:tasks.workunit.client.0.smithi110.stdout:5/305: dwrite d1/dc/f1d [0,4194304] 0 2024-09-21T04:15:12.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/224: rmdir d5/d1e/d37 39 2024-09-21T04:15:12.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/218: dread f4 [0,4194304] 0 2024-09-21T04:15:12.144 INFO:tasks.workunit.client.1.smithi148.stdout:1/205: symlink db/l49 0 2024-09-21T04:15:12.144 INFO:tasks.workunit.client.1.smithi148.stdout:1/206: chown db/dd/f23 186546890 1 2024-09-21T04:15:12.144 INFO:tasks.workunit.client.0.smithi110.stdout:0/547: truncate d4/dd/d10/d29/f6b 1841901 0 2024-09-21T04:15:12.145 INFO:tasks.workunit.client.0.smithi110.stdout:0/548: truncate d4/f21 1168286 0 2024-09-21T04:15:12.145 INFO:tasks.workunit.client.0.smithi110.stdout:7/288: dwrite dd/d1c/d39/f4f [0,4194304] 0 2024-09-21T04:15:12.146 INFO:tasks.workunit.client.1.smithi148.stdout:7/256: getdents d5/d14/d25 0 2024-09-21T04:15:12.150 INFO:tasks.workunit.client.1.smithi148.stdout:4/241: rename c0 to d4/d3a/c49 0 2024-09-21T04:15:12.158 INFO:tasks.workunit.client.0.smithi110.stdout:1/537: mknod d2/d9/d49/d4f/cb2 0 2024-09-21T04:15:12.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/222: mknod d3/d14/d1f/d26/d2e/c4e 0 2024-09-21T04:15:12.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/223: creat d3/d14/d1a/f4f x:0 0 0 2024-09-21T04:15:12.160 INFO:tasks.workunit.client.1.smithi148.stdout:5/224: chown d3/l30 6 1 2024-09-21T04:15:12.160 INFO:tasks.workunit.client.1.smithi148.stdout:5/225: creat d3/d14/d1f/f50 x:0 0 0 2024-09-21T04:15:12.160 INFO:tasks.workunit.client.0.smithi110.stdout:8/325: symlink d0/d7/d13/d4b/l6e 0 2024-09-21T04:15:12.161 INFO:tasks.workunit.client.1.smithi148.stdout:6/179: dwrite d0/d1a/d23/f14 [0,4194304] 0 2024-09-21T04:15:12.161 INFO:tasks.workunit.client.1.smithi148.stdout:3/201: dwrite d3/d5/d6/d12/da/f3d [4194304,4194304] 0 2024-09-21T04:15:12.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/326: dread d0/d7/d13/f1e [0,4194304] 0 2024-09-21T04:15:12.162 INFO:tasks.workunit.client.1.smithi148.stdout:2/293: dwrite d0/f1d [0,4194304] 0 2024-09-21T04:15:12.162 INFO:tasks.workunit.client.1.smithi148.stdout:9/225: stat d5/l7 0 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/226: creat d5/d2b/d32/f5b x:0 0 0 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:6/180: read d0/d1a/d23/f19 [515971,22575] 0 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:6/181: read - d0/d8/f24 zero size 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/227: write d5/d22/d28/f30 [1969044,122379] 0 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:0/219: rmdir d5/d6/d3e 39 2024-09-21T04:15:12.163 INFO:tasks.workunit.client.1.smithi148.stdout:8/203: sync 2024-09-21T04:15:12.165 INFO:tasks.workunit.client.1.smithi148.stdout:3/202: dread d3/d5/d6/d12/da/f11 [0,4194304] 0 2024-09-21T04:15:12.166 INFO:tasks.workunit.client.0.smithi110.stdout:5/306: mkdir d1/d49/d69 0 2024-09-21T04:15:12.167 INFO:tasks.workunit.client.1.smithi148.stdout:3/203: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr fail", "who": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: Activating manager daemon smithi148.zjrahi 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: osdmap e47: 6 total, 6 up, 6 in 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "mgr fail", "who": "smithi110.axmqzb"}]': finished 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: mgrmap e21: smithi148.zjrahi(active, starting, since 0.0129035s) 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:15:12.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:15:12.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:12 smithi110 ceph-mon[27723]: Manager daemon smithi148.zjrahi is now available 2024-09-21T04:15:12.170 INFO:tasks.workunit.client.0.smithi110.stdout:0/549: rename d4/d47/d2c/d3a/f6c to d4/dd/d60/fa4 0 2024-09-21T04:15:12.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/242: rename d4/d12/d16/d26/f35 to d4/d23/f4a 0 2024-09-21T04:15:12.172 INFO:tasks.workunit.client.1.smithi148.stdout:8/204: write f9 [1948749,120698] 0 2024-09-21T04:15:12.174 INFO:tasks.workunit.client.0.smithi110.stdout:7/289: mkdir dd/d1c/d39/d56 0 2024-09-21T04:15:12.174 INFO:tasks.workunit.client.0.smithi110.stdout:7/290: chown dd/d14/d17/l29 93031 1 2024-09-21T04:15:12.176 INFO:tasks.workunit.client.0.smithi110.stdout:1/538: symlink d2/d9/d23/d3b/d9c/lb3 0 2024-09-21T04:15:12.176 INFO:tasks.workunit.client.0.smithi110.stdout:1/539: write d2/d9/d23/d3b/d5e/d86/faf [379655,37659] 0 2024-09-21T04:15:12.178 INFO:tasks.workunit.client.0.smithi110.stdout:8/327: unlink d0/d7/d13/d4b/f40 0 2024-09-21T04:15:12.178 INFO:tasks.workunit.client.0.smithi110.stdout:8/328: write d0/d7/d13/d4b/f1f [4031212,81826] 0 2024-09-21T04:15:12.179 INFO:tasks.workunit.client.0.smithi110.stdout:5/307: mknod d1/c6a 0 2024-09-21T04:15:12.180 INFO:tasks.workunit.client.0.smithi110.stdout:5/308: chown d1/d49/d4d/d1e/f2f 11140 1 2024-09-21T04:15:12.180 INFO:tasks.workunit.client.0.smithi110.stdout:5/309: dread d1/d22/f3c [0,4194304] 0 2024-09-21T04:15:12.185 INFO:tasks.workunit.client.0.smithi110.stdout:1/540: write d2/f94 [897203,66561] 0 2024-09-21T04:15:12.185 INFO:tasks.workunit.client.0.smithi110.stdout:1/541: stat d2/d9/d23/d4e/la7 0 2024-09-21T04:15:12.187 INFO:tasks.workunit.client.0.smithi110.stdout:0/550: truncate d4/f1f 2073023 0 2024-09-21T04:15:12.188 INFO:tasks.workunit.client.0.smithi110.stdout:0/551: write d4/d47/d2c/d3a/f43 [336275,7935] 0 2024-09-21T04:15:12.188 INFO:tasks.workunit.client.0.smithi110.stdout:0/552: readlink d4/dd/d10/l8f 0 2024-09-21T04:15:12.189 INFO:tasks.workunit.client.1.smithi148.stdout:2/294: symlink d0/d9/dc/l5d 0 2024-09-21T04:15:12.190 INFO:tasks.workunit.client.0.smithi110.stdout:7/291: rename dd/f18 to dd/f57 0 2024-09-21T04:15:12.191 INFO:tasks.workunit.client.0.smithi110.stdout:8/329: chown d0/d7/d13/d4b/c29 1787333 1 2024-09-21T04:15:12.193 INFO:tasks.workunit.client.0.smithi110.stdout:5/310: creat d1/d7/d18/d3f/f6b x:0 0 0 2024-09-21T04:15:12.195 INFO:tasks.workunit.client.1.smithi148.stdout:9/228: mknod d5/d22/d28/c5c 0 2024-09-21T04:15:12.196 INFO:tasks.workunit.client.1.smithi148.stdout:0/220: rmdir d5/d6/d3e 39 2024-09-21T04:15:12.197 INFO:tasks.workunit.client.0.smithi110.stdout:0/553: mknod d4/dd/d10/d8a/ca5 0 2024-09-21T04:15:12.199 INFO:tasks.workunit.client.1.smithi148.stdout:1/207: getdents db/d24/d2a 0 2024-09-21T04:15:12.199 INFO:tasks.workunit.client.0.smithi110.stdout:0/554: write d4/d47/f40 [59964,68763] 0 2024-09-21T04:15:12.199 INFO:tasks.workunit.client.0.smithi110.stdout:0/555: write d4/dd/d53/fa2 [567799,5270] 0 2024-09-21T04:15:12.200 INFO:tasks.workunit.client.0.smithi110.stdout:7/292: creat dd/d1c/d39/d4a/d52/f58 x:0 0 0 2024-09-21T04:15:12.202 INFO:tasks.workunit.client.1.smithi148.stdout:3/204: symlink d3/d5/d6/d2b/l40 0 2024-09-21T04:15:12.203 INFO:tasks.workunit.client.0.smithi110.stdout:4/379: sync 2024-09-21T04:15:12.204 INFO:tasks.workunit.client.0.smithi110.stdout:8/330: mkdir d0/d7/d6f 0 2024-09-21T04:15:12.204 INFO:tasks.workunit.client.0.smithi110.stdout:9/198: sync 2024-09-21T04:15:12.205 INFO:tasks.workunit.client.0.smithi110.stdout:8/331: write d0/d7/f20 [4661352,45732] 0 2024-09-21T04:15:12.207 INFO:tasks.workunit.client.1.smithi148.stdout:5/226: rename d3/d34/c39 to d3/d14/d1f/c51 0 2024-09-21T04:15:12.207 INFO:tasks.workunit.client.1.smithi148.stdout:5/227: stat d3/dc/f17 0 2024-09-21T04:15:12.207 INFO:tasks.workunit.client.1.smithi148.stdout:5/228: readlink d3/d14/l3a 0 2024-09-21T04:15:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:2/295: dwrite d0/d9/dc/f24 [0,4194304] 0 2024-09-21T04:15:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:2/296: write d0/d2b/d48/f58 [761122,17346] 0 2024-09-21T04:15:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:2/297: chown d0/d9/f29 7209 1 2024-09-21T04:15:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:2/298: stat d0/d9/dc/d13/d25/d2d/c3f 0 2024-09-21T04:15:12.209 INFO:tasks.workunit.client.0.smithi110.stdout:5/311: creat d1/d49/f6c x:0 0 0 2024-09-21T04:15:12.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/243: creat d4/d12/d2c/d2d/d44/f4b x:0 0 0 2024-09-21T04:15:12.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/244: stat d4/fe 0 2024-09-21T04:15:12.212 INFO:tasks.workunit.client.1.smithi148.stdout:4/245: chown d4/d12/d16/l1d 11 1 2024-09-21T04:15:12.216 INFO:tasks.workunit.client.0.smithi110.stdout:7/293: creat dd/d4d/f59 x:0 0 0 2024-09-21T04:15:12.216 INFO:tasks.workunit.client.0.smithi110.stdout:7/294: chown dd/d1c/f38 2 1 2024-09-21T04:15:12.217 INFO:tasks.workunit.client.1.smithi148.stdout:8/205: creat dc/f41 x:0 0 0 2024-09-21T04:15:12.219 INFO:tasks.workunit.client.0.smithi110.stdout:4/380: rename d17/d34/d3d/l45 to d17/d1e/d7b/l83 0 2024-09-21T04:15:12.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/556: dwrite d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:12.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/557: readlink d4/dd/d10/d29/la3 0 2024-09-21T04:15:12.221 INFO:tasks.workunit.client.0.smithi110.stdout:0/558: chown f2 384724 1 2024-09-21T04:15:12.221 INFO:tasks.workunit.client.0.smithi110.stdout:0/559: fsync d4/f5c 0 2024-09-21T04:15:12.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/182: getdents d0/d1a/d23/d27 0 2024-09-21T04:15:12.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/183: write d0/d1a/d23/d16/f2c [465270,7208] 0 2024-09-21T04:15:12.222 INFO:tasks.workunit.client.1.smithi148.stdout:8/206: dread f2 [0,4194304] 0 2024-09-21T04:15:12.223 INFO:tasks.workunit.client.0.smithi110.stdout:9/199: truncate d7/d26/f2c 729185 0 2024-09-21T04:15:12.224 INFO:tasks.workunit.client.0.smithi110.stdout:7/295: dread dd/d1c/f2c [0,4194304] 0 2024-09-21T04:15:12.224 INFO:tasks.workunit.client.0.smithi110.stdout:0/560: dread d4/d1c/d62/f90 [0,4194304] 0 2024-09-21T04:15:12.224 INFO:tasks.workunit.client.0.smithi110.stdout:7/296: chown dd/f3e 10662145 1 2024-09-21T04:15:12.226 INFO:tasks.workunit.client.0.smithi110.stdout:8/332: mknod d0/d7/d23/d52/c70 0 2024-09-21T04:15:12.226 INFO:tasks.workunit.client.0.smithi110.stdout:8/333: chown d0/f11 31 1 2024-09-21T04:15:12.227 INFO:tasks.workunit.client.0.smithi110.stdout:8/334: dread - d0/d30/d54/f59 zero size 2024-09-21T04:15:12.227 INFO:tasks.workunit.client.0.smithi110.stdout:8/335: fdatasync d0/d7/d13/f5a 0 2024-09-21T04:15:12.228 INFO:tasks.workunit.client.1.smithi148.stdout:1/208: mknod db/dd/c4a 0 2024-09-21T04:15:12.228 INFO:tasks.workunit.client.1.smithi148.stdout:1/209: chown db/dd/d21/f43 16222732 1 2024-09-21T04:15:12.229 INFO:tasks.workunit.client.1.smithi148.stdout:0/221: dwrite d5/d6/d7/d11/f29 [0,4194304] 0 2024-09-21T04:15:12.230 INFO:tasks.workunit.client.1.smithi148.stdout:3/205: dwrite d3/d5/d6/f17 [4194304,4194304] 0 2024-09-21T04:15:12.231 INFO:tasks.workunit.client.1.smithi148.stdout:5/229: link d3/d34/d3c/f48 d3/d4a/f52 0 2024-09-21T04:15:12.236 INFO:tasks.workunit.client.1.smithi148.stdout:4/246: dwrite d4/d12/f1b [0,4194304] 0 2024-09-21T04:15:12.236 INFO:tasks.workunit.client.1.smithi148.stdout:4/247: fsync d4/d12/d2c/f32 0 2024-09-21T04:15:12.236 INFO:tasks.workunit.client.1.smithi148.stdout:4/248: write d4/fe [853307,63217] 0 2024-09-21T04:15:12.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/299: rename d0/d9/d38/f43 to d0/d2b/f5e 0 2024-09-21T04:15:12.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/542: truncate d2/d9/f6c 3036167 0 2024-09-21T04:15:12.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/543: dread - d2/d9/d49/d5c/d91/f96 zero size 2024-09-21T04:15:12.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/544: chown d2/d9/d23/d3b/d9c/fa4 0 1 2024-09-21T04:15:12.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/545: fdatasync d2/d9/d49/d4f/f67 0 2024-09-21T04:15:12.242 INFO:tasks.workunit.client.1.smithi148.stdout:6/184: truncate d0/d8/f10 3071135 0 2024-09-21T04:15:12.244 INFO:tasks.workunit.client.1.smithi148.stdout:8/207: mkdir dc/d2a/d42 0 2024-09-21T04:15:12.245 INFO:tasks.workunit.client.1.smithi148.stdout:3/206: dread d3/d5/d6/d12/da/d23/f26 [0,4194304] 0 2024-09-21T04:15:12.245 INFO:tasks.workunit.client.1.smithi148.stdout:3/207: write d3/d5/d6/d12/da/f11 [1100185,65223] 0 2024-09-21T04:15:12.248 INFO:tasks.workunit.client.1.smithi148.stdout:1/210: mknod db/d24/c4b 0 2024-09-21T04:15:12.250 INFO:tasks.workunit.client.1.smithi148.stdout:0/222: dwrite d5/d6/d2a/d2b/f2c [0,4194304] 0 2024-09-21T04:15:12.250 INFO:tasks.workunit.client.1.smithi148.stdout:0/223: chown d5/d6/d7/d18/c30 21063 1 2024-09-21T04:15:12.250 INFO:tasks.workunit.client.1.smithi148.stdout:0/224: readlink d5/d6/l14 0 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/211: dread db/d24/f25 [0,4194304] 0 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/212: read - db/d46/f48 zero size 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/213: chown db/d24/d2e/l3d 752 1 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/214: fdatasync db/d24/d2a/f45 0 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:1/215: stat db/dd 0 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:8/208: write f6 [2698445,22961] 0 2024-09-21T04:15:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:9/229: truncate d5/d22/d28/f30 962583 0 2024-09-21T04:15:12.257 INFO:tasks.workunit.client.0.smithi110.stdout:9/200: link d7/d10/d24/d3c/d1f/l3d d7/d10/d24/d3c/d14/d3e/l44 0 2024-09-21T04:15:12.262 INFO:tasks.workunit.client.0.smithi110.stdout:0/561: unlink d4/d1c/f8d 0 2024-09-21T04:15:12.262 INFO:tasks.workunit.client.1.smithi148.stdout:6/185: write d0/d1a/d23/fb [122234,82386] 0 2024-09-21T04:15:12.263 INFO:tasks.workunit.client.1.smithi148.stdout:4/249: symlink d4/d12/d16/l4c 0 2024-09-21T04:15:12.263 INFO:tasks.workunit.client.1.smithi148.stdout:4/250: write d4/f8 [4969014,23848] 0 2024-09-21T04:15:12.263 INFO:tasks.workunit.client.1.smithi148.stdout:2/300: dwrite d0/d2f/f4a [0,4194304] 0 2024-09-21T04:15:12.263 INFO:tasks.workunit.client.0.smithi110.stdout:7/297: link dd/d14/f2a dd/d1c/d39/f5a 0 2024-09-21T04:15:12.267 INFO:tasks.workunit.client.0.smithi110.stdout:7/298: dread dd/d1c/f34 [0,4194304] 0 2024-09-21T04:15:12.267 INFO:tasks.workunit.client.0.smithi110.stdout:7/299: dread - dd/d4d/f59 zero size 2024-09-21T04:15:12.271 INFO:tasks.workunit.client.0.smithi110.stdout:5/312: rmdir d1/d49/d4d/d1e 39 2024-09-21T04:15:12.271 INFO:tasks.workunit.client.0.smithi110.stdout:5/313: dread - d1/d7/d18/f41 zero size 2024-09-21T04:15:12.271 INFO:tasks.workunit.client.0.smithi110.stdout:5/314: dread - d1/d49/d4d/d1e/f61 zero size 2024-09-21T04:15:12.271 INFO:tasks.workunit.client.0.smithi110.stdout:0/562: dread d4/dd/d10/f13 [0,4194304] 0 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/381: rmdir d17/d34/d3d/d20 39 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:3/306: sync 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:6/248: sync 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:2/162: sync 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:3/307: chown d0/d12/d16/d1a/d5c/f66 2859199 1 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:3/308: fdatasync d0/d12/f5d 0 2024-09-21T04:15:12.282 INFO:tasks.workunit.client.0.smithi110.stdout:2/163: write d2/d1a/d1f/d2a/f21 [1276109,97608] 0 2024-09-21T04:15:12.283 INFO:tasks.workunit.client.1.smithi148.stdout:9/230: dwrite d5/d1f/f24 [0,4194304] 0 2024-09-21T04:15:12.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/546: mknod d2/d9/d23/d3b/d93/cb4 0 2024-09-21T04:15:12.287 INFO:tasks.workunit.client.1.smithi148.stdout:0/225: symlink d5/l44 0 2024-09-21T04:15:12.288 INFO:tasks.workunit.client.0.smithi110.stdout:9/201: link d7/c9 d7/d10/d24/d3c/d1f/d43/c45 0 2024-09-21T04:15:12.289 INFO:tasks.workunit.client.1.smithi148.stdout:8/209: mkdir dc/d43 0 2024-09-21T04:15:12.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/210: readlink dc/d13/l20 0 2024-09-21T04:15:12.290 INFO:tasks.workunit.client.1.smithi148.stdout:8/211: fdatasync dc/f17 0 2024-09-21T04:15:12.290 INFO:tasks.workunit.client.0.smithi110.stdout:8/336: getdents d0/d7/d23 0 2024-09-21T04:15:12.290 INFO:tasks.workunit.client.0.smithi110.stdout:8/337: write d0/d30/d54/f59 [998558,92961] 0 2024-09-21T04:15:12.291 INFO:tasks.workunit.client.1.smithi148.stdout:1/216: dwrite db/f27 [0,4194304] 0 2024-09-21T04:15:12.292 INFO:tasks.workunit.client.0.smithi110.stdout:0/563: truncate d4/dd/d10/d29/d45/f5b 1799441 0 2024-09-21T04:15:12.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/251: getdents d4/d12/d16 0 2024-09-21T04:15:12.293 INFO:tasks.workunit.client.0.smithi110.stdout:4/382: unlink d17/d1e/d7b/f6d 0 2024-09-21T04:15:12.295 INFO:tasks.workunit.client.0.smithi110.stdout:6/249: link f3 d4/d12/d16/d48/d4b/f51 0 2024-09-21T04:15:12.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/301: unlink d0/d9/dc/c52 0 2024-09-21T04:15:12.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/309: creat d0/d22/f72 x:0 0 0 2024-09-21T04:15:12.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/310: fdatasync d0/d12/f1b 0 2024-09-21T04:15:12.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/311: chown d0/d12/f5d 1665297 1 2024-09-21T04:15:12.300 INFO:tasks.workunit.client.1.smithi148.stdout:9/231: dread d5/d2b/f4f [0,4194304] 0 2024-09-21T04:15:12.301 INFO:tasks.workunit.client.0.smithi110.stdout:2/164: creat d2/d1a/d1f/f2c x:0 0 0 2024-09-21T04:15:12.302 INFO:tasks.workunit.client.1.smithi148.stdout:6/186: dwrite d0/d1a/d23/d16/f1f [0,4194304] 0 2024-09-21T04:15:12.308 INFO:tasks.workunit.client.1.smithi148.stdout:0/226: truncate d5/d6/f15 1410885 0 2024-09-21T04:15:12.311 INFO:tasks.workunit.client.0.smithi110.stdout:9/202: mknod d7/d10/d24/d3c/d1f/c46 0 2024-09-21T04:15:12.311 INFO:tasks.workunit.client.0.smithi110.stdout:9/203: chown d7/d10/d24/d3c/d11/f2e 103 1 2024-09-21T04:15:12.311 INFO:tasks.workunit.client.0.smithi110.stdout:7/300: dwrite dd/d1c/d39/f3c [0,4194304] 0 2024-09-21T04:15:12.312 INFO:tasks.workunit.client.0.smithi110.stdout:7/301: creat dd/d14/d17/f5b x:0 0 0 2024-09-21T04:15:12.313 INFO:tasks.workunit.client.0.smithi110.stdout:1/547: dwrite d2/d9/d49/d4f/d53/f8c [0,4194304] 0 2024-09-21T04:15:12.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/208: dwrite d3/d5/d6/d12/da/f11 [0,4194304] 0 2024-09-21T04:15:12.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/209: chown d3/cc 23141076 1 2024-09-21T04:15:12.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/210: write f1 [82686,89149] 0 2024-09-21T04:15:12.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/211: write d3/d5/d6/d12/da/d23/f20 [2305104,86943] 0 2024-09-21T04:15:12.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/212: creat dc/d43/f44 x:0 0 0 2024-09-21T04:15:12.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/213: write fb [4510241,21275] 0 2024-09-21T04:15:12.315 INFO:tasks.workunit.client.0.smithi110.stdout:1/548: dread d2/d9/d23/f29 [0,4194304] 0 2024-09-21T04:15:12.316 INFO:tasks.workunit.client.0.smithi110.stdout:5/315: dwrite d1/f15 [0,4194304] 0 2024-09-21T04:15:12.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/316: chown d1/dc/f1d 1 1 2024-09-21T04:15:12.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/317: readlink d1/d49/d4d/d1e/l37 0 2024-09-21T04:15:12.318 INFO:tasks.workunit.client.1.smithi148.stdout:1/217: mkdir db/d46/d4c 0 2024-09-21T04:15:12.319 INFO:tasks.workunit.client.0.smithi110.stdout:0/564: unlink d4/dd/c30 0 2024-09-21T04:15:12.319 INFO:tasks.workunit.client.0.smithi110.stdout:0/565: chown d4/f57 365859353 1 2024-09-21T04:15:12.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/252: rmdir d4/d3a 39 2024-09-21T04:15:12.321 INFO:tasks.workunit.client.0.smithi110.stdout:0/566: write d4/dd/f1b [5819368,77995] 0 2024-09-21T04:15:12.322 INFO:tasks.workunit.client.0.smithi110.stdout:4/383: mknod d17/d34/d3d/d20/d43/d51/d7a/c84 0 2024-09-21T04:15:12.325 INFO:tasks.workunit.client.1.smithi148.stdout:6/187: symlink d0/l2f 0 2024-09-21T04:15:12.327 INFO:tasks.workunit.client.0.smithi110.stdout:6/250: unlink d4/d12/d17/c25 0 2024-09-21T04:15:12.328 INFO:tasks.workunit.client.1.smithi148.stdout:4/253: dread d4/fe [0,4194304] 0 2024-09-21T04:15:12.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/227: mkdir d5/d6/d2a/d45 0 2024-09-21T04:15:12.330 INFO:tasks.workunit.client.1.smithi148.stdout:9/232: dwrite d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:12.330 INFO:tasks.workunit.client.1.smithi148.stdout:9/233: chown d5/d1e 9300 1 2024-09-21T04:15:12.331 INFO:tasks.workunit.client.1.smithi148.stdout:9/234: dread - d5/d2b/d32/f5a zero size 2024-09-21T04:15:12.331 INFO:tasks.workunit.client.1.smithi148.stdout:9/235: creat d5/d2b/f5d x:0 0 0 2024-09-21T04:15:12.331 INFO:tasks.workunit.client.0.smithi110.stdout:2/165: mkdir d2/d1a/d1f/d29/d19/d2d 0 2024-09-21T04:15:12.337 INFO:tasks.workunit.client.0.smithi110.stdout:8/338: rename d0/d7/d13/d46/c47 to d0/d7/d13/d4b/c71 0 2024-09-21T04:15:12.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/549: truncate d2/d9/d49/d4f/d53/f85 1278769 0 2024-09-21T04:15:12.345 INFO:tasks.workunit.client.1.smithi148.stdout:3/212: creat d3/d5/f41 x:0 0 0 2024-09-21T04:15:12.346 INFO:tasks.workunit.client.1.smithi148.stdout:3/213: write d3/d5/d6/d12/da/d23/f26 [609637,115403] 0 2024-09-21T04:15:12.346 INFO:tasks.workunit.client.1.smithi148.stdout:3/214: dread - d3/d1f/f3e zero size 2024-09-21T04:15:12.348 INFO:tasks.workunit.client.0.smithi110.stdout:0/567: mknod d4/dd/d10/d8a/ca6 0 2024-09-21T04:15:12.349 INFO:tasks.workunit.client.0.smithi110.stdout:0/568: write d4/dd/d10/d29/f5d [450790,110604] 0 2024-09-21T04:15:12.349 INFO:tasks.workunit.client.0.smithi110.stdout:0/569: chown d4/f21 5489183 1 2024-09-21T04:15:12.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/214: symlink dc/d3e/l45 0 2024-09-21T04:15:12.351 INFO:tasks.workunit.client.0.smithi110.stdout:6/251: unlink d4/f4e 0 2024-09-21T04:15:12.351 INFO:tasks.workunit.client.0.smithi110.stdout:6/252: write d4/d12/d17/d2e/d40/f2f [1630940,116733] 0 2024-09-21T04:15:12.352 INFO:tasks.workunit.client.1.smithi148.stdout:2/302: truncate d0/d9/dc/f24 2475622 0 2024-09-21T04:15:12.352 INFO:tasks.workunit.client.1.smithi148.stdout:2/303: readlink d0/l8 0 2024-09-21T04:15:12.353 INFO:tasks.workunit.client.0.smithi110.stdout:2/166: unlink d2/f8 0 2024-09-21T04:15:12.358 INFO:tasks.workunit.client.1.smithi148.stdout:0/228: creat d5/d6/d7/d22/f46 x:0 0 0 2024-09-21T04:15:12.358 INFO:tasks.workunit.client.1.smithi148.stdout:0/229: fdatasync d5/d6/d7/d11/f34 0 2024-09-21T04:15:12.360 INFO:tasks.workunit.client.0.smithi110.stdout:7/302: rename dd/d1c/d39/c50 to dd/d14/d17/c5c 0 2024-09-21T04:15:12.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/218: dwrite db/d24/f25 [0,4194304] 0 2024-09-21T04:15:12.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/219: write db/dd/d21/f43 [815336,36608] 0 2024-09-21T04:15:12.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/220: chown db/dd/f13 994 1 2024-09-21T04:15:12.361 INFO:tasks.workunit.client.1.smithi148.stdout:1/221: write db/d46/f47 [614826,30565] 0 2024-09-21T04:15:12.361 INFO:tasks.workunit.client.1.smithi148.stdout:1/222: truncate db/d46/f48 873903 0 2024-09-21T04:15:12.362 INFO:tasks.workunit.client.0.smithi110.stdout:7/303: dread dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:12.363 INFO:tasks.workunit.client.0.smithi110.stdout:9/204: dwrite d7/d10/d24/f32 [0,4194304] 0 2024-09-21T04:15:12.363 INFO:tasks.workunit.client.0.smithi110.stdout:9/205: chown d7/d10/d24/d3c/d14/l16 2 1 2024-09-21T04:15:12.363 INFO:tasks.workunit.client.0.smithi110.stdout:5/318: getdents d1/dc 0 2024-09-21T04:15:12.363 INFO:tasks.workunit.client.0.smithi110.stdout:5/319: fsync d1/d7/d18/f41 0 2024-09-21T04:15:12.364 INFO:tasks.workunit.client.1.smithi148.stdout:9/236: mkdir d5/d22/d4c/d5e 0 2024-09-21T04:15:12.365 INFO:tasks.workunit.client.1.smithi148.stdout:9/237: dread - d5/d2b/d32/f58 zero size 2024-09-21T04:15:12.365 INFO:tasks.workunit.client.1.smithi148.stdout:6/188: dwrite d0/d1a/d23/f14 [0,4194304] 0 2024-09-21T04:15:12.365 INFO:tasks.workunit.client.1.smithi148.stdout:6/189: truncate d0/d1a/d23/f12 2924667 0 2024-09-21T04:15:12.365 INFO:tasks.workunit.client.1.smithi148.stdout:6/190: readlink d0/d8/l28 0 2024-09-21T04:15:12.366 INFO:tasks.workunit.client.0.smithi110.stdout:8/339: dwrite d0/d30/f31 [0,4194304] 0 2024-09-21T04:15:12.366 INFO:tasks.workunit.client.0.smithi110.stdout:0/570: getdents d4/dd/d10/d29/d75 0 2024-09-21T04:15:12.366 INFO:tasks.workunit.client.0.smithi110.stdout:0/571: chown d4/f11 219149440 1 2024-09-21T04:15:12.367 INFO:tasks.workunit.client.1.smithi148.stdout:3/215: creat d3/d5/d6/d12/da/d23/d28/f42 x:0 0 0 2024-09-21T04:15:12.367 INFO:tasks.workunit.client.0.smithi110.stdout:6/253: mkdir d4/d12/d16/d52 0 2024-09-21T04:15:12.369 INFO:tasks.workunit.client.0.smithi110.stdout:6/254: dread d4/de/f10 [0,4194304] 0 2024-09-21T04:15:12.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/304: symlink d0/d2f/l5f 0 2024-09-21T04:15:12.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/305: chown d0/d9/d38 385283115 1 2024-09-21T04:15:12.370 INFO:tasks.workunit.client.1.smithi148.stdout:2/306: write d0/d2b/f5e [4530199,62486] 0 2024-09-21T04:15:12.371 INFO:tasks.workunit.client.0.smithi110.stdout:4/384: rename le to d17/d34/d3d/d57/l85 0 2024-09-21T04:15:12.373 INFO:tasks.workunit.client.0.smithi110.stdout:5/320: dread d1/d7/d18/f28 [0,4194304] 0 2024-09-21T04:15:12.373 INFO:tasks.workunit.client.0.smithi110.stdout:6/255: read d4/d12/d17/f2a [336944,7494] 0 2024-09-21T04:15:12.374 INFO:tasks.workunit.client.0.smithi110.stdout:5/321: creat d1/d7/d18/f6d x:0 0 0 2024-09-21T04:15:12.374 INFO:tasks.workunit.client.0.smithi110.stdout:6/256: creat d4/d12/d17/d33/d3e/f53 x:0 0 0 2024-09-21T04:15:12.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/254: rename d4/d12/l1c to d4/d12/d16/l4d 0 2024-09-21T04:15:12.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/255: chown d4/c3b 233801444 1 2024-09-21T04:15:12.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/256: fdatasync d4/d23/d2a/f36 0 2024-09-21T04:15:12.378 INFO:tasks.workunit.client.0.smithi110.stdout:7/304: mkdir dd/d1c/d39/d5d 0 2024-09-21T04:15:12.379 INFO:tasks.workunit.client.1.smithi148.stdout:5/230: sync 2024-09-21T04:15:12.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/257: sync 2024-09-21T04:15:12.379 INFO:tasks.workunit.client.1.smithi148.stdout:5/231: write d3/d24/f43 [778320,120318] 0 2024-09-21T04:15:12.379 INFO:tasks.workunit.client.1.smithi148.stdout:8/215: dwrite dc/f14 [0,4194304] 0 2024-09-21T04:15:12.379 INFO:tasks.workunit.client.1.smithi148.stdout:8/216: read - dc/d2a/f2e zero size 2024-09-21T04:15:12.380 INFO:tasks.workunit.client.0.smithi110.stdout:4/385: dread d17/d3e/d6b/f7d [0,4194304] 0 2024-09-21T04:15:12.380 INFO:tasks.workunit.client.1.smithi148.stdout:0/230: truncate d5/d6/d7/d18/f32 344112 0 2024-09-21T04:15:12.381 INFO:tasks.workunit.client.0.smithi110.stdout:2/167: dwrite d2/d1a/d1f/d2a/d18/f1d [0,4194304] 0 2024-09-21T04:15:12.381 INFO:tasks.workunit.client.0.smithi110.stdout:2/168: fdatasync d2/d1a/d1f/d2a/f21 0 2024-09-21T04:15:12.381 INFO:tasks.workunit.client.1.smithi148.stdout:9/238: write d5/d1e/d37/f3b [1296829,44219] 0 2024-09-21T04:15:12.386 INFO:tasks.workunit.client.1.smithi148.stdout:3/216: write d3/d5/f1e [1137314,1681] 0 2024-09-21T04:15:12.386 INFO:tasks.workunit.client.1.smithi148.stdout:1/223: rmdir db/dd/d21/d3b 0 2024-09-21T04:15:12.390 INFO:tasks.workunit.client.0.smithi110.stdout:0/572: mkdir d4/d1c/d62/d91/da7 0 2024-09-21T04:15:12.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/257: dread d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:12.393 INFO:tasks.workunit.client.1.smithi148.stdout:6/191: dwrite d0/d8/f29 [0,4194304] 0 2024-09-21T04:15:12.393 INFO:tasks.workunit.client.1.smithi148.stdout:6/192: fsync d0/d8/d18/f25 0 2024-09-21T04:15:12.394 INFO:tasks.workunit.client.1.smithi148.stdout:2/307: link d0/d2b/d48/f58 d0/d9/d16/f60 0 2024-09-21T04:15:12.394 INFO:tasks.workunit.client.1.smithi148.stdout:2/308: creat d0/d2b/f61 x:0 0 0 2024-09-21T04:15:12.394 INFO:tasks.workunit.client.1.smithi148.stdout:2/309: stat d0/ff 0 2024-09-21T04:15:12.394 INFO:tasks.workunit.client.0.smithi110.stdout:9/206: rename d7/d10/d24/d3c/d14/d31/l3b to d7/d10/d24/d3c/d1f/d43/l47 0 2024-09-21T04:15:12.397 INFO:tasks.workunit.client.1.smithi148.stdout:7/258: link d5/d14/c45 d5/d14/d25/d40/c51 0 2024-09-21T04:15:12.397 INFO:tasks.workunit.client.1.smithi148.stdout:7/259: truncate d5/d14/d25/f42 1206613 0 2024-09-21T04:15:12.398 INFO:tasks.workunit.client.1.smithi148.stdout:7/260: chown d5/f6 4201794 1 2024-09-21T04:15:12.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/232: symlink d3/d4a/l53 0 2024-09-21T04:15:12.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/322: mkdir d1/d22/d6e 0 2024-09-21T04:15:12.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/323: fsync d1/d49/d4d/f24 0 2024-09-21T04:15:12.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/324: chown d1/d49/d4d/d1e/l37 0 1 2024-09-21T04:15:12.405 INFO:tasks.workunit.client.0.smithi110.stdout:0/573: read f2 [4645272,36284] 0 2024-09-21T04:15:12.408 INFO:tasks.workunit.client.1.smithi148.stdout:8/217: getdents dc/d13 0 2024-09-21T04:15:12.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/193: dread d0/d1a/d23/f14 [0,4194304] 0 2024-09-21T04:15:12.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/194: creat d0/d1a/d23/d16/f30 x:0 0 0 2024-09-21T04:15:12.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/257: mkdir d4/d12/d16/d48/d54 0 2024-09-21T04:15:12.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/258: stat d4/d12/d16/d48 0 2024-09-21T04:15:12.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/259: chown d4/d12/d16/c20 2086501 1 2024-09-21T04:15:12.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/231: mknod d5/d6/d3e/c47 0 2024-09-21T04:15:12.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/232: dread - d5/d6/d7/f43 zero size 2024-09-21T04:15:12.413 INFO:tasks.workunit.client.0.smithi110.stdout:4/386: mkdir d17/d34/d3d/d20/d43/d51/d7a/d86 0 2024-09-21T04:15:12.416 INFO:tasks.workunit.client.0.smithi110.stdout:7/305: dwrite dd/d1c/f34 [4194304,4194304] 0 2024-09-21T04:15:12.417 INFO:tasks.workunit.client.1.smithi148.stdout:3/217: mkdir d3/d5/d6/d12/d43 0 2024-09-21T04:15:12.418 INFO:tasks.workunit.client.1.smithi148.stdout:4/258: symlink d4/d12/d2c/d2d/l4e 0 2024-09-21T04:15:12.418 INFO:tasks.workunit.client.1.smithi148.stdout:4/259: fdatasync d4/d23/f2b 0 2024-09-21T04:15:12.419 INFO:tasks.workunit.client.1.smithi148.stdout:4/260: chown d4/d12 1634904176 1 2024-09-21T04:15:12.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/340: rename d0/d7/l3c to d0/d30/d34/d3e/d5d/l72 0 2024-09-21T04:15:12.422 INFO:tasks.workunit.client.0.smithi110.stdout:9/207: fdatasync d7/d26/f2d 0 2024-09-21T04:15:12.425 INFO:tasks.workunit.client.0.smithi110.stdout:5/325: mkdir d1/d49/d6f 0 2024-09-21T04:15:12.425 INFO:tasks.workunit.client.0.smithi110.stdout:5/326: fdatasync d1/d22/f27 0 2024-09-21T04:15:12.429 INFO:tasks.workunit.client.0.smithi110.stdout:0/574: mknod d4/dd/d53/d96/ca8 0 2024-09-21T04:15:12.430 INFO:tasks.workunit.client.1.smithi148.stdout:7/261: truncate d5/f3d 7561171 0 2024-09-21T04:15:12.431 INFO:tasks.workunit.client.1.smithi148.stdout:7/262: chown d5/d8/cd 434181 1 2024-09-21T04:15:12.433 INFO:tasks.workunit.client.1.smithi148.stdout:2/310: dwrite d0/d2f/f4a [0,4194304] 0 2024-09-21T04:15:12.433 INFO:tasks.workunit.client.1.smithi148.stdout:2/311: truncate d0/d2b/f61 524461 0 2024-09-21T04:15:12.433 INFO:tasks.workunit.client.0.smithi110.stdout:4/387: rename d17/f50 to d17/d34/d3d/d20/d43/d51/d7a/d86/f87 0 2024-09-21T04:15:12.433 INFO:tasks.workunit.client.0.smithi110.stdout:0/575: write d4/d1c/f3c [546511,44081] 0 2024-09-21T04:15:12.436 INFO:tasks.workunit.client.1.smithi148.stdout:5/233: link d3/d4a/f52 d3/dc/f54 0 2024-09-21T04:15:12.436 INFO:tasks.workunit.client.1.smithi148.stdout:5/234: readlink d3/d37/l21 0 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.0.smithi110.stdout:2/169: dwrite d2/d1a/d1f/d2a/d18/f1d [0,4194304] 0 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.0.smithi110.stdout:8/341: dread d0/d30/f31 [0,4194304] 0 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.0.smithi110.stdout:8/342: chown d0/d7/d13/f2d 24709 1 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.0.smithi110.stdout:8/343: truncate d0/d7/d13/d1c/f1d 5201898 0 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.1.smithi148.stdout:8/218: link dc/f17 dc/f46 0 2024-09-21T04:15:12.438 INFO:tasks.workunit.client.1.smithi148.stdout:8/219: chown dc/c32 642191547 1 2024-09-21T04:15:12.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/220: truncate f9 4525719 0 2024-09-21T04:15:12.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/221: creat dc/d43/f47 x:0 0 0 2024-09-21T04:15:12.440 INFO:tasks.workunit.client.0.smithi110.stdout:4/388: read d17/f58 [1516508,129176] 0 2024-09-21T04:15:12.440 INFO:tasks.workunit.client.0.smithi110.stdout:4/389: dread - d17/d1e/d21/f4d zero size 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr fail", "who": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:15:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: Activating manager daemon smithi148.zjrahi 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: osdmap e47: 6 total, 6 up, 6 in 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.14213 172.21.15.110:0/3615406957' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "mgr fail", "who": "smithi110.axmqzb"}]': finished 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: mgrmap e21: smithi148.zjrahi(active, starting, since 0.0129035s) 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:15:12.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:15:12.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:12 smithi148 ceph-mon[34957]: Manager daemon smithi148.zjrahi is now available 2024-09-21T04:15:12.444 INFO:tasks.workunit.client.0.smithi110.stdout:0/576: creat d4/dd/d10/d29/d75/fa9 x:0 0 0 2024-09-21T04:15:12.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/195: symlink d0/d1a/d23/l31 0 2024-09-21T04:15:12.445 INFO:tasks.workunit.client.1.smithi148.stdout:0/233: mkdir d5/d6/d7/d11/d48 0 2024-09-21T04:15:12.445 INFO:tasks.workunit.client.1.smithi148.stdout:1/224: dwrite db/ff [0,4194304] 0 2024-09-21T04:15:12.445 INFO:tasks.workunit.client.1.smithi148.stdout:1/225: chown db/lc 0 1 2024-09-21T04:15:12.447 INFO:tasks.workunit.client.1.smithi148.stdout:9/239: getdents d5/d22/d28/d38 0 2024-09-21T04:15:12.447 INFO:tasks.workunit.client.1.smithi148.stdout:1/226: dread db/d24/d2a/f45 [0,4194304] 0 2024-09-21T04:15:12.448 INFO:tasks.workunit.client.1.smithi148.stdout:1/227: read db/d24/d2a/f45 [36307,54550] 0 2024-09-21T04:15:12.449 INFO:tasks.workunit.client.0.smithi110.stdout:8/344: creat d0/d30/d34/d3e/d32/f73 x:0 0 0 2024-09-21T04:15:12.451 INFO:tasks.workunit.client.1.smithi148.stdout:3/218: rename d3/d5/cd to d3/d1f/c44 0 2024-09-21T04:15:12.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/390: link d17/d1e/d7b/f5e d17/d34/d3d/d20/f88 0 2024-09-21T04:15:12.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/391: readlink d17/d34/d3d/l67 0 2024-09-21T04:15:12.454 INFO:tasks.workunit.client.0.smithi110.stdout:2/170: dread d2/d1a/d1f/d2a/d18/f1d [0,4194304] 0 2024-09-21T04:15:12.454 INFO:tasks.workunit.client.0.smithi110.stdout:2/171: fsync d2/d1a/f20 0 2024-09-21T04:15:12.455 INFO:tasks.workunit.client.1.smithi148.stdout:4/261: creat d4/d12/d16/f4f x:0 0 0 2024-09-21T04:15:12.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/577: link d4/dd/d10/d29/d45/l48 d4/dd/d10/d29/laa 0 2024-09-21T04:15:12.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/578: dread - d4/dd/d53/d66/f81 zero size 2024-09-21T04:15:12.457 INFO:tasks.workunit.client.1.smithi148.stdout:5/235: symlink d3/d24/l55 0 2024-09-21T04:15:12.459 INFO:tasks.workunit.client.0.smithi110.stdout:8/345: mknod d0/d30/c74 0 2024-09-21T04:15:12.459 INFO:tasks.workunit.client.0.smithi110.stdout:8/346: write d0/d7/d23/f33 [336760,50927] 0 2024-09-21T04:15:12.459 INFO:tasks.workunit.client.0.smithi110.stdout:8/347: dread - d0/d7/d23/f69 zero size 2024-09-21T04:15:12.461 INFO:tasks.workunit.client.1.smithi148.stdout:2/312: fsync d0/d2b/f5e 0 2024-09-21T04:15:12.462 INFO:tasks.workunit.client.0.smithi110.stdout:4/392: creat d17/d34/d3d/d20/d43/d51/d7a/d86/f89 x:0 0 0 2024-09-21T04:15:12.464 INFO:tasks.workunit.client.1.smithi148.stdout:6/196: mkdir d0/d1a/d23/d32 0 2024-09-21T04:15:12.467 INFO:tasks.workunit.client.0.smithi110.stdout:4/393: rmdir d17/d34/d3d/d57 39 2024-09-21T04:15:12.467 INFO:tasks.workunit.client.0.smithi110.stdout:4/394: fdatasync d17/d34/d3d/d20/d43/d51/d7a/f7f 0 2024-09-21T04:15:12.468 INFO:tasks.workunit.client.1.smithi148.stdout:9/240: symlink d5/d22/d28/d2f/l5f 0 2024-09-21T04:15:12.469 INFO:tasks.workunit.client.0.smithi110.stdout:8/348: dread d0/d7/fd [0,4194304] 0 2024-09-21T04:15:12.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/228: unlink db/dd/f20 0 2024-09-21T04:15:12.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/229: write db/d24/f26 [424469,52682] 0 2024-09-21T04:15:12.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/230: fdatasync f6 0 2024-09-21T04:15:12.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/231: chown db/d24/d2a 427 1 2024-09-21T04:15:12.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/232: fdatasync db/ff 0 2024-09-21T04:15:12.472 INFO:tasks.workunit.client.0.smithi110.stdout:4/395: stat cf 0 2024-09-21T04:15:12.472 INFO:tasks.workunit.client.1.smithi148.stdout:2/313: dread d0/f23 [0,4194304] 0 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/263: rename d5/f34 to d5/d14/d25/f52 0 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/264: chown d5/dc/d28/f3a 2 1 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/265: write d5/dc/f10 [440038,124765] 0 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/266: truncate d5/d14/f4b 510862 0 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:2/314: dread d0/d9/d16/f60 [0,4194304] 0 2024-09-21T04:15:12.474 INFO:tasks.workunit.client.1.smithi148.stdout:2/315: chown d0/d2b/d48 1975628 1 2024-09-21T04:15:12.475 INFO:tasks.workunit.client.0.smithi110.stdout:8/349: creat d0/d30/d54/d6b/f75 x:0 0 0 2024-09-21T04:15:12.475 INFO:tasks.workunit.client.0.smithi110.stdout:7/306: dwrite dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:12.475 INFO:tasks.workunit.client.0.smithi110.stdout:0/579: dread d4/dd/f1b [0,4194304] 0 2024-09-21T04:15:12.475 INFO:tasks.workunit.client.0.smithi110.stdout:0/580: stat d4/dd/f8e 0 2024-09-21T04:15:12.476 INFO:tasks.workunit.client.1.smithi148.stdout:4/262: symlink d4/d12/d2c/d2d/d44/l50 0 2024-09-21T04:15:12.476 INFO:tasks.workunit.client.0.smithi110.stdout:4/396: unlink d17/d34/d3d/d20/f49 0 2024-09-21T04:15:12.478 INFO:tasks.workunit.client.1.smithi148.stdout:6/197: mkdir d0/d1a/d23/dc/d33 0 2024-09-21T04:15:12.478 INFO:tasks.workunit.client.0.smithi110.stdout:8/350: link d0/d7/d13/d1c/d41/l56 d0/d30/d34/l76 0 2024-09-21T04:15:12.479 INFO:tasks.workunit.client.0.smithi110.stdout:0/581: creat d4/d47/fab x:0 0 0 2024-09-21T04:15:12.480 INFO:tasks.workunit.client.1.smithi148.stdout:1/233: creat db/f4d x:0 0 0 2024-09-21T04:15:12.481 INFO:tasks.workunit.client.0.smithi110.stdout:4/397: creat d17/d34/f8a x:0 0 0 2024-09-21T04:15:12.481 INFO:tasks.workunit.client.0.smithi110.stdout:4/398: chown d17/d34/d3d/d20/d43/d51/d7a/f7f 20900 1 2024-09-21T04:15:12.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/351: write d0/d7/d13/d1c/f50 [969073,42561] 0 2024-09-21T04:15:12.483 INFO:tasks.workunit.client.1.smithi148.stdout:7/267: getdents d5/dc/d24 0 2024-09-21T04:15:12.483 INFO:tasks.workunit.client.1.smithi148.stdout:7/268: chown d5/dc/d28/d4a/l4d 1104181 1 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:4/399: creat d17/d3e/f8b x:0 0 0 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:4/400: dread - d17/d1e/d21/f4d zero size 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:5/327: dwrite d1/dc/f1a [0,4194304] 0 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:0/582: read d4/dd/d53/fa2 [313001,100153] 0 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:5/328: write d1/d49/d4d/d1e/f61 [327003,92597] 0 2024-09-21T04:15:12.484 INFO:tasks.workunit.client.0.smithi110.stdout:5/329: write d1/d49/d4d/d1e/f5f [247229,15353] 0 2024-09-21T04:15:12.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/269: write d5/dc/f10 [258583,13171] 0 2024-09-21T04:15:12.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/270: creat d5/dc/d28/f53 x:0 0 0 2024-09-21T04:15:12.486 INFO:tasks.workunit.client.1.smithi148.stdout:4/263: creat d4/f51 x:0 0 0 2024-09-21T04:15:12.488 INFO:tasks.workunit.client.0.smithi110.stdout:9/208: dwrite d7/d10/d24/d3c/d35/f3a [0,4194304] 0 2024-09-21T04:15:12.488 INFO:tasks.workunit.client.1.smithi148.stdout:1/234: dread db/d46/f48 [0,4194304] 0 2024-09-21T04:15:12.490 INFO:tasks.workunit.client.0.smithi110.stdout:0/583: mknod d4/dd/d10/d8a/cac 0 2024-09-21T04:15:12.493 INFO:tasks.workunit.client.1.smithi148.stdout:0/234: dwrite d5/d6/d7/d22/f46 [0,4194304] 0 2024-09-21T04:15:12.493 INFO:tasks.workunit.client.0.smithi110.stdout:5/330: write d1/f15 [307218,106595] 0 2024-09-21T04:15:12.493 INFO:tasks.workunit.client.0.smithi110.stdout:5/331: write d1/d22/f44 [74388,97819] 0 2024-09-21T04:15:12.494 INFO:tasks.workunit.client.1.smithi148.stdout:3/219: dwrite d3/d5/d6/d12/da/f11 [0,4194304] 0 2024-09-21T04:15:12.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/222: dwrite dc/f46 [0,4194304] 0 2024-09-21T04:15:12.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/172: dwrite d2/d1a/d1f/f2c [0,4194304] 0 2024-09-21T04:15:12.497 INFO:tasks.workunit.client.1.smithi148.stdout:2/316: symlink d0/d9/l62 0 2024-09-21T04:15:12.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/209: truncate f2 1139064 0 2024-09-21T04:15:12.502 INFO:tasks.workunit.client.0.smithi110.stdout:7/307: dwrite dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:12.504 INFO:tasks.workunit.client.1.smithi148.stdout:7/271: link d5/d8/f22 d5/d14/d25/f54 0 2024-09-21T04:15:12.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/272: chown d5/dc/d28/f3a 0 1 2024-09-21T04:15:12.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/273: readlink d5/d8/l11 0 2024-09-21T04:15:12.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/274: fdatasync d5/dc/f19 0 2024-09-21T04:15:12.505 INFO:tasks.workunit.client.1.smithi148.stdout:2/317: dread d0/d2b/f53 [0,4194304] 0 2024-09-21T04:15:12.505 INFO:tasks.workunit.client.1.smithi148.stdout:2/318: fsync d0/f1e 0 2024-09-21T04:15:12.506 INFO:tasks.workunit.client.0.smithi110.stdout:8/352: dread d0/d7/d13/d1c/f1d [0,4194304] 0 2024-09-21T04:15:12.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/198: dwrite d0/f21 [0,4194304] 0 2024-09-21T04:15:12.512 INFO:tasks.workunit.client.0.smithi110.stdout:5/332: link d1/d49/d4d/f24 d1/d22/d51/f70 0 2024-09-21T04:15:12.512 INFO:tasks.workunit.client.0.smithi110.stdout:5/333: creat d1/d49/f71 x:0 0 0 2024-09-21T04:15:12.514 INFO:tasks.workunit.client.0.smithi110.stdout:4/401: dwrite d17/d1e/f33 [0,4194304] 0 2024-09-21T04:15:12.520 INFO:tasks.workunit.client.1.smithi148.stdout:3/220: dwrite d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:12.536 INFO:tasks.workunit.client.1.smithi148.stdout:3/221: dwrite d3/d5/d6/f17 [4194304,4194304] 0 2024-09-21T04:15:12.550 INFO:tasks.workunit.client.1.smithi148.stdout:3/222: dread d3/d5/d6/d12/da/d23/f20 [0,4194304] 0 2024-09-21T04:15:12.590 INFO:tasks.workunit.client.1.smithi148.stdout:1/235: symlink db/d24/d2a/l4e 0 2024-09-21T04:15:12.590 INFO:tasks.workunit.client.1.smithi148.stdout:1/236: chown db/dd/l3f 41140839 1 2024-09-21T04:15:12.597 INFO:tasks.workunit.client.1.smithi148.stdout:0/235: creat d5/d6/d3e/f49 x:0 0 0 2024-09-21T04:15:12.597 INFO:tasks.workunit.client.1.smithi148.stdout:0/236: fsync d5/d6/d7/d11/f38 0 2024-09-21T04:15:12.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/237: dread db/dd/f13 [0,4194304] 0 2024-09-21T04:15:12.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/238: write db/d46/f48 [1124380,24563] 0 2024-09-21T04:15:12.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/236: rename d3/dc to d3/d14/d1f/d26/d2e/d56 0 2024-09-21T04:15:12.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/319: mknod d0/d9/dc/d13/c63 0 2024-09-21T04:15:12.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/320: dread - d0/d9/f47 zero size 2024-09-21T04:15:12.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/321: stat d0/d9/d59 0 2024-09-21T04:15:12.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/264: rmdir d4/d12/d16/d29 39 2024-09-21T04:15:12.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/265: read - d4/d12/d16/f4f zero size 2024-09-21T04:15:12.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/199: truncate d0/d1a/d23/ff 791883 0 2024-09-21T04:15:12.609 INFO:tasks.workunit.client.1.smithi148.stdout:9/241: rename d5/d22 to d5/d60 0 2024-09-21T04:15:12.614 INFO:tasks.workunit.client.1.smithi148.stdout:2/322: dread d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:12.620 INFO:tasks.workunit.client.1.smithi148.stdout:0/237: dwrite d5/d6/d2a/d2b/f36 [0,4194304] 0 2024-09-21T04:15:12.620 INFO:tasks.workunit.client.1.smithi148.stdout:0/238: chown d5/d6/d7/d18/c30 31425 1 2024-09-21T04:15:12.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/237: dwrite d3/d14/f2d [0,4194304] 0 2024-09-21T04:15:12.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/238: write d3/d14/d1f/d26/d2e/d56/f54 [663224,85875] 0 2024-09-21T04:15:12.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/223: rename dc/d13/f19 to dc/d13/d28/f48 0 2024-09-21T04:15:12.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/224: dread - dc/d13/f35 zero size 2024-09-21T04:15:12.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/225: write f8 [1622239,23190] 0 2024-09-21T04:15:12.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/266: link d4/d23/f4a d4/d12/d2c/d2d/f52 0 2024-09-21T04:15:12.627 INFO:tasks.workunit.client.1.smithi148.stdout:2/323: dread d0/d9/dc/d13/d25/d2d/f2e [0,4194304] 0 2024-09-21T04:15:12.628 INFO:tasks.workunit.client.0.smithi110.stdout:9/210: rename d7/d10/d24/d3c/d1f/l3d to d7/d10/d24/d3c/d1f/l48 0 2024-09-21T04:15:12.635 INFO:tasks.workunit.client.1.smithi148.stdout:6/200: mkdir d0/d34 0 2024-09-21T04:15:12.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/584: rmdir d4/d47/d2c 39 2024-09-21T04:15:12.637 INFO:tasks.workunit.client.0.smithi110.stdout:9/211: dread f2 [0,4194304] 0 2024-09-21T04:15:12.639 INFO:tasks.workunit.client.1.smithi148.stdout:1/239: rmdir db/dd 39 2024-09-21T04:15:12.640 INFO:tasks.workunit.client.0.smithi110.stdout:5/334: mkdir d1/d7/d58/d72 0 2024-09-21T04:15:12.642 INFO:tasks.workunit.client.1.smithi148.stdout:5/239: dwrite d3/d14/d1a/f4d [0,4194304] 0 2024-09-21T04:15:12.643 INFO:tasks.workunit.client.1.smithi148.stdout:5/240: readlink d3/d14/d1f/d26/d2e/d56/l22 0 2024-09-21T04:15:12.643 INFO:tasks.workunit.client.1.smithi148.stdout:5/241: write f0 [18298,120056] 0 2024-09-21T04:15:12.645 INFO:tasks.workunit.client.0.smithi110.stdout:4/402: link d17/d34/d3d/d20/f26 d17/d34/d3d/d20/d43/f8c 0 2024-09-21T04:15:12.645 INFO:tasks.workunit.client.0.smithi110.stdout:4/403: write d17/d3e/d6b/f7d [1055623,37744] 0 2024-09-21T04:15:12.645 INFO:tasks.workunit.client.1.smithi148.stdout:0/239: symlink d5/d6/d7/d11/d48/l4a 0 2024-09-21T04:15:12.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/240: read d5/d6/f1f [61396,72655] 0 2024-09-21T04:15:12.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/241: fdatasync d5/d6/d2a/d2b/f39 0 2024-09-21T04:15:12.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/242: chown d5/d6/d7/d11/f17 3 1 2024-09-21T04:15:12.647 INFO:tasks.workunit.client.0.smithi110.stdout:7/308: truncate dd/d1c/f34 259029 0 2024-09-21T04:15:12.647 INFO:tasks.workunit.client.0.smithi110.stdout:7/309: creat dd/d4d/f5e x:0 0 0 2024-09-21T04:15:12.648 INFO:tasks.workunit.client.0.smithi110.stdout:7/310: chown dd/d4d/f59 31415 1 2024-09-21T04:15:12.650 INFO:tasks.workunit.client.0.smithi110.stdout:2/173: truncate d2/d1a/d1f/f2c 2396212 0 2024-09-21T04:15:12.650 INFO:tasks.workunit.client.0.smithi110.stdout:2/174: chown d2/d1a/d1f 1 1 2024-09-21T04:15:12.651 INFO:tasks.workunit.client.0.smithi110.stdout:0/585: creat d4/d1c/d62/d69/d9b/fad x:0 0 0 2024-09-21T04:15:12.654 INFO:tasks.workunit.client.0.smithi110.stdout:9/212: rename d7/d10/d24/d3c/l1e to d7/d10/d24/l49 0 2024-09-21T04:15:12.657 INFO:tasks.workunit.client.1.smithi148.stdout:8/226: creat dc/d13/d3c/f49 x:0 0 0 2024-09-21T04:15:12.657 INFO:tasks.workunit.client.1.smithi148.stdout:8/227: write dc/d2a/f2e [836231,116269] 0 2024-09-21T04:15:12.659 INFO:tasks.workunit.client.1.smithi148.stdout:3/223: truncate d3/d5/d6/d12/da/d23/f20 2636585 0 2024-09-21T04:15:12.659 INFO:tasks.workunit.client.1.smithi148.stdout:3/224: stat d3/d5/c30 0 2024-09-21T04:15:12.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/311: read dd/d1c/d39/f4f [78980,110531] 0 2024-09-21T04:15:12.661 INFO:tasks.workunit.client.1.smithi148.stdout:7/275: rename d5/d14/d25/c3b to d5/c55 0 2024-09-21T04:15:12.664 INFO:tasks.workunit.client.0.smithi110.stdout:4/404: mkdir d17/d34/d3d/d20/d43/d51/d7a/d8d 0 2024-09-21T04:15:12.664 INFO:tasks.workunit.client.0.smithi110.stdout:4/405: chown d17/d1e 14097 1 2024-09-21T04:15:12.666 INFO:tasks.workunit.client.1.smithi148.stdout:9/242: getdents d5/d2b 0 2024-09-21T04:15:12.666 INFO:tasks.workunit.client.0.smithi110.stdout:2/175: dread d2/d1a/d1f/d29/fd [0,4194304] 0 2024-09-21T04:15:12.667 INFO:tasks.workunit.client.0.smithi110.stdout:7/312: dread dd/f1a [0,4194304] 0 2024-09-21T04:15:12.667 INFO:tasks.workunit.client.1.smithi148.stdout:0/243: dwrite d5/d6/d2a/d2b/f39 [0,4194304] 0 2024-09-21T04:15:12.667 INFO:tasks.workunit.client.0.smithi110.stdout:0/586: symlink d4/dd/d53/d66/lae 0 2024-09-21T04:15:12.667 INFO:tasks.workunit.client.0.smithi110.stdout:0/587: write d4/d1c/d62/f39 [1049439,41310] 0 2024-09-21T04:15:12.668 INFO:tasks.workunit.client.0.smithi110.stdout:0/588: truncate d4/dd/d10/d29/d75/fa9 169912 0 2024-09-21T04:15:12.668 INFO:tasks.workunit.client.0.smithi110.stdout:0/589: fdatasync d4/d47/f40 0 2024-09-21T04:15:12.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/201: symlink d0/l35 0 2024-09-21T04:15:12.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/202: chown d0/d8 7237139 1 2024-09-21T04:15:12.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/213: dread d7/d10/d24/d3c/d35/f3a [0,4194304] 0 2024-09-21T04:15:12.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/214: write d7/d10/f37 [249499,63467] 0 2024-09-21T04:15:12.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/215: chown d7/d26/f2c 71 1 2024-09-21T04:15:12.672 INFO:tasks.workunit.client.0.smithi110.stdout:5/335: rename d1/d7/d18/d3f/f6b to d1/d49/d4d/d1e/d5a/d64/f73 0 2024-09-21T04:15:12.675 INFO:tasks.workunit.client.1.smithi148.stdout:1/240: unlink db/f27 0 2024-09-21T04:15:12.676 INFO:tasks.workunit.client.0.smithi110.stdout:7/313: creat dd/d48/d55/f5f x:0 0 0 2024-09-21T04:15:12.679 INFO:tasks.workunit.client.1.smithi148.stdout:7/276: dread d5/f3d [0,4194304] 0 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/244: write d5/d6/d2a/d2b/f39 [361512,76201] 0 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/245: creat d5/d6/d7/f4b x:0 0 0 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/246: chown d5/d6/d7/d22/f24 1839233 1 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/247: stat d5/d6/d3e/f26 0 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/248: chown d5/d6/d7/d11/f1d 1 1 2024-09-21T04:15:12.683 INFO:tasks.workunit.client.0.smithi110.stdout:2/176: read d2/f16 [47027,39355] 0 2024-09-21T04:15:12.684 INFO:tasks.workunit.client.0.smithi110.stdout:7/314: write dd/f31 [1547114,88057] 0 2024-09-21T04:15:12.684 INFO:tasks.workunit.client.0.smithi110.stdout:7/315: chown dd/d14/c27 14885 1 2024-09-21T04:15:12.684 INFO:tasks.workunit.client.0.smithi110.stdout:7/316: fdatasync dd/d14/d17/f3a 0 2024-09-21T04:15:12.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/241: dread db/d24/f2b [0,4194304] 0 2024-09-21T04:15:12.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/242: dread - db/f4d zero size 2024-09-21T04:15:12.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/243: write db/d24/f25 [4484831,41740] 0 2024-09-21T04:15:12.689 INFO:tasks.workunit.client.0.smithi110.stdout:9/216: getdents d7/d10/d24/d3c/d1f/d25 0 2024-09-21T04:15:12.689 INFO:tasks.workunit.client.0.smithi110.stdout:9/217: write d7/d10/d24/d3c/d1f/f23 [4974928,111593] 0 2024-09-21T04:15:12.693 INFO:tasks.workunit.client.1.smithi148.stdout:8/228: dwrite dc/d2a/f40 [0,4194304] 0 2024-09-21T04:15:12.693 INFO:tasks.workunit.client.1.smithi148.stdout:8/229: chown dc/d13/f35 6393 1 2024-09-21T04:15:12.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/243: dwrite d5/d2b/d32/f5b [0,4194304] 0 2024-09-21T04:15:12.693 INFO:tasks.workunit.client.0.smithi110.stdout:5/336: link d1/f4c d1/d7/d18/f74 0 2024-09-21T04:15:12.693 INFO:tasks.workunit.client.0.smithi110.stdout:5/337: chown d1/d7 1759637 1 2024-09-21T04:15:12.694 INFO:tasks.workunit.client.0.smithi110.stdout:5/338: chown d1/d49/d4d/d1e/d5a/d64/f73 5 1 2024-09-21T04:15:12.694 INFO:tasks.workunit.client.0.smithi110.stdout:5/339: write d1/f38 [801212,35652] 0 2024-09-21T04:15:12.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/244: dread db/ff [0,4194304] 0 2024-09-21T04:15:12.694 INFO:tasks.workunit.client.1.smithi148.stdout:5/242: rename d3/d14/d1a/f27 to d3/d14/d1a/f57 0 2024-09-21T04:15:12.696 INFO:tasks.workunit.client.0.smithi110.stdout:4/406: rename l10 to d17/d34/d3d/d57/l8e 0 2024-09-21T04:15:12.696 INFO:tasks.workunit.client.0.smithi110.stdout:4/407: chown d17/f32 121884 1 2024-09-21T04:15:12.698 INFO:tasks.workunit.client.0.smithi110.stdout:2/177: mkdir d2/d1a/d1f/d29/d19/d2e 0 2024-09-21T04:15:12.702 INFO:tasks.workunit.client.0.smithi110.stdout:4/408: read f14 [2204796,92355] 0 2024-09-21T04:15:12.706 INFO:tasks.workunit.client.0.smithi110.stdout:7/317: link dd/d14/d17/f2b dd/d1c/d39/d4a/d52/f60 0 2024-09-21T04:15:12.706 INFO:tasks.workunit.client.0.smithi110.stdout:7/318: chown dd/d1c/d39/d56 116710 1 2024-09-21T04:15:12.714 INFO:tasks.workunit.client.0.smithi110.stdout:3/312: sync 2024-09-21T04:15:12.714 INFO:tasks.workunit.client.0.smithi110.stdout:6/260: sync 2024-09-21T04:15:12.714 INFO:tasks.workunit.client.0.smithi110.stdout:6/261: write d4/d12/f1d [4542531,51448] 0 2024-09-21T04:15:12.715 INFO:tasks.workunit.client.0.smithi110.stdout:1/550: sync 2024-09-21T04:15:12.715 INFO:tasks.workunit.client.0.smithi110.stdout:1/551: write d2/f45 [1915659,9129] 0 2024-09-21T04:15:12.715 INFO:tasks.workunit.client.0.smithi110.stdout:5/340: symlink d1/l75 0 2024-09-21T04:15:12.716 INFO:tasks.workunit.client.0.smithi110.stdout:5/341: stat d1/dc/c55 0 2024-09-21T04:15:12.718 INFO:tasks.workunit.client.0.smithi110.stdout:0/590: rename d4/d1c/d62/d91/da7 to d4/dd/d10/daf 0 2024-09-21T04:15:12.721 INFO:tasks.workunit.client.0.smithi110.stdout:4/409: getdents d17/d34/d3d 0 2024-09-21T04:15:12.722 INFO:tasks.workunit.client.0.smithi110.stdout:7/319: dwrite dd/d4d/f59 [0,4194304] 0 2024-09-21T04:15:12.728 INFO:tasks.workunit.client.0.smithi110.stdout:1/552: mknod d2/d9/d49/d4f/cb5 0 2024-09-21T04:15:12.731 INFO:tasks.workunit.client.0.smithi110.stdout:5/342: symlink d1/d7/d58/l76 0 2024-09-21T04:15:12.731 INFO:tasks.workunit.client.0.smithi110.stdout:5/343: truncate d1/d7/d18/f41 553197 0 2024-09-21T04:15:12.732 INFO:tasks.workunit.client.0.smithi110.stdout:8/353: sync 2024-09-21T04:15:12.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/225: symlink d3/l45 0 2024-09-21T04:15:12.738 INFO:tasks.workunit.client.0.smithi110.stdout:4/410: creat d17/d1e/d64/f8f x:0 0 0 2024-09-21T04:15:12.744 INFO:tasks.workunit.client.1.smithi148.stdout:6/203: mkdir d0/d1a/d23/d16/d36 0 2024-09-21T04:15:12.744 INFO:tasks.workunit.client.1.smithi148.stdout:6/204: dread - d0/d8/f24 zero size 2024-09-21T04:15:12.746 INFO:tasks.workunit.client.0.smithi110.stdout:0/591: dwrite d4/d47/d2c/d3a/f51 [4194304,4194304] 0 2024-09-21T04:15:12.746 INFO:tasks.workunit.client.1.smithi148.stdout:7/277: mknod d5/d14/d25/d40/c56 0 2024-09-21T04:15:12.747 INFO:tasks.workunit.client.0.smithi110.stdout:3/313: dwrite d0/d12/d3c/f5e [0,4194304] 0 2024-09-21T04:15:12.747 INFO:tasks.workunit.client.0.smithi110.stdout:3/314: fsync d0/d12/d3c/f45 0 2024-09-21T04:15:12.748 INFO:tasks.workunit.client.0.smithi110.stdout:7/320: unlink dd/d14/d17/f2b 0 2024-09-21T04:15:12.749 INFO:tasks.workunit.client.0.smithi110.stdout:6/262: dwrite d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:12.749 INFO:tasks.workunit.client.1.smithi148.stdout:7/278: write d5/d14/d25/f42 [664501,48415] 0 2024-09-21T04:15:12.752 INFO:tasks.workunit.client.1.smithi148.stdout:0/249: creat d5/d6/d2a/d3f/f4c x:0 0 0 2024-09-21T04:15:12.752 INFO:tasks.workunit.client.1.smithi148.stdout:0/250: read - d5/d6/d3e/f3a zero size 2024-09-21T04:15:12.758 INFO:tasks.workunit.client.0.smithi110.stdout:5/344: link d1/dc/f33 d1/d7/d58/f77 0 2024-09-21T04:15:12.761 INFO:tasks.workunit.client.0.smithi110.stdout:9/218: rename d7/d10/d24/d3c/d14/l16 to d7/l4a 0 2024-09-21T04:15:12.762 INFO:tasks.workunit.client.0.smithi110.stdout:6/263: dread d4/f6 [4194304,4194304] 0 2024-09-21T04:15:12.762 INFO:tasks.workunit.client.0.smithi110.stdout:6/264: read d4/f13 [4056069,124381] 0 2024-09-21T04:15:12.764 INFO:tasks.workunit.client.1.smithi148.stdout:8/230: creat dc/d2a/d42/f4a x:0 0 0 2024-09-21T04:15:12.764 INFO:tasks.workunit.client.1.smithi148.stdout:8/231: truncate dc/d43/f44 63636 0 2024-09-21T04:15:12.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/315: creat d0/d12/d16/d1a/d6f/f73 x:0 0 0 2024-09-21T04:15:12.767 INFO:tasks.workunit.client.0.smithi110.stdout:8/354: dwrite d0/d30/d54/d6b/f75 [0,4194304] 0 2024-09-21T04:15:12.768 INFO:tasks.workunit.client.1.smithi148.stdout:9/244: symlink d5/d39/l61 0 2024-09-21T04:15:12.768 INFO:tasks.workunit.client.1.smithi148.stdout:9/245: write d5/fe [1188410,91467] 0 2024-09-21T04:15:12.768 INFO:tasks.workunit.client.0.smithi110.stdout:1/553: symlink d2/d9/d49/lb6 0 2024-09-21T04:15:12.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/592: dwrite d4/dd/d53/fa2 [0,4194304] 0 2024-09-21T04:15:12.770 INFO:tasks.workunit.client.0.smithi110.stdout:4/411: dwrite d17/f39 [0,4194304] 0 2024-09-21T04:15:12.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/593: readlink d4/dd/l89 0 2024-09-21T04:15:12.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/594: write d4/f56 [861716,37490] 0 2024-09-21T04:15:12.770 INFO:tasks.workunit.client.1.smithi148.stdout:1/245: mkdir db/d24/d4f 0 2024-09-21T04:15:12.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/321: dwrite dd/d14/d17/f3a [4194304,4194304] 0 2024-09-21T04:15:12.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/322: truncate dd/d1c/f34 933113 0 2024-09-21T04:15:12.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/345: rename d1/dc/f57 to d1/d49/d6f/f78 0 2024-09-21T04:15:12.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/243: creat d3/f58 x:0 0 0 2024-09-21T04:15:12.777 INFO:tasks.workunit.client.1.smithi148.stdout:5/244: dread d3/d14/d1f/d26/d2e/d56/f54 [0,4194304] 0 2024-09-21T04:15:12.780 INFO:tasks.workunit.client.0.smithi110.stdout:7/323: dread dd/d1c/d39/f3c [0,4194304] 0 2024-09-21T04:15:12.782 INFO:tasks.workunit.client.0.smithi110.stdout:6/265: creat d4/d12/d17/d33/f55 x:0 0 0 2024-09-21T04:15:12.782 INFO:tasks.workunit.client.0.smithi110.stdout:6/266: creat d4/d12/d16/f56 x:0 0 0 2024-09-21T04:15:12.784 INFO:tasks.workunit.client.0.smithi110.stdout:0/595: dread d4/d1c/f25 [0,4194304] 0 2024-09-21T04:15:12.787 INFO:tasks.workunit.client.0.smithi110.stdout:3/316: dwrite d0/d12/d16/d1a/d5c/f66 [0,4194304] 0 2024-09-21T04:15:12.790 INFO:tasks.workunit.client.0.smithi110.stdout:8/355: link d0/d7/d13/d1c/d41/l56 d0/d30/d34/d3e/d60/l77 0 2024-09-21T04:15:12.791 INFO:tasks.workunit.client.0.smithi110.stdout:8/356: readlink d0/l6 0 2024-09-21T04:15:12.791 INFO:tasks.workunit.client.0.smithi110.stdout:8/357: truncate d0/d7/d13/d46/f68 120500 0 2024-09-21T04:15:12.791 INFO:tasks.workunit.client.0.smithi110.stdout:8/358: creat d0/d7/f78 x:0 0 0 2024-09-21T04:15:12.792 INFO:tasks.workunit.client.1.smithi148.stdout:3/226: mkdir d3/d5/d6/d12/da/d23/d28/d46 0 2024-09-21T04:15:12.792 INFO:tasks.workunit.client.1.smithi148.stdout:3/227: fsync d3/d1f/f3e 0 2024-09-21T04:15:12.792 INFO:tasks.workunit.client.1.smithi148.stdout:3/228: chown d3/d5/d6/d12 6555048 1 2024-09-21T04:15:12.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/205: creat d0/d1a/d23/f37 x:0 0 0 2024-09-21T04:15:12.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/206: stat d0/d1a/d23/l2b 0 2024-09-21T04:15:12.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/207: chown d0/d1a/d23/d27/f2d 249 1 2024-09-21T04:15:12.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/208: truncate d0/d1a/f1c 822423 0 2024-09-21T04:15:12.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/246: dwrite f1 [4194304,4194304] 0 2024-09-21T04:15:12.798 INFO:tasks.workunit.client.1.smithi148.stdout:0/251: unlink d5/d6/l42 0 2024-09-21T04:15:12.801 INFO:tasks.workunit.client.1.smithi148.stdout:8/232: rename f2 to dc/d13/d3c/f4b 0 2024-09-21T04:15:12.801 INFO:tasks.workunit.client.1.smithi148.stdout:8/233: chown dc/d3e 2 1 2024-09-21T04:15:12.801 INFO:tasks.workunit.client.1.smithi148.stdout:3/229: write d3/d5/d6/d12/da/f3d [4388402,93293] 0 2024-09-21T04:15:12.801 INFO:tasks.workunit.client.1.smithi148.stdout:3/230: creat d3/d1f/f47 x:0 0 0 2024-09-21T04:15:12.802 INFO:tasks.workunit.client.1.smithi148.stdout:1/246: symlink db/d46/l50 0 2024-09-21T04:15:12.803 INFO:tasks.workunit.client.1.smithi148.stdout:1/247: fdatasync db/dd/f13 0 2024-09-21T04:15:12.804 INFO:tasks.workunit.client.1.smithi148.stdout:5/245: mkdir d3/d14/d1f/d26/d59 0 2024-09-21T04:15:12.805 INFO:tasks.workunit.client.0.smithi110.stdout:4/412: creat d17/d34/d3d/d20/f90 x:0 0 0 2024-09-21T04:15:12.806 INFO:tasks.workunit.client.1.smithi148.stdout:6/209: mkdir d0/d34/d38 0 2024-09-21T04:15:12.808 INFO:tasks.workunit.client.1.smithi148.stdout:2/324: sync 2024-09-21T04:15:12.808 INFO:tasks.workunit.client.1.smithi148.stdout:4/267: sync 2024-09-21T04:15:12.808 INFO:tasks.workunit.client.1.smithi148.stdout:2/325: stat d0/d9/dc/d13/d25/f45 0 2024-09-21T04:15:12.809 INFO:tasks.workunit.client.1.smithi148.stdout:5/246: dread f0 [0,4194304] 0 2024-09-21T04:15:12.811 INFO:tasks.workunit.client.1.smithi148.stdout:5/247: write d3/d14/d1f/d26/d2e/d56/f17 [1060587,118351] 0 2024-09-21T04:15:12.811 INFO:tasks.workunit.client.1.smithi148.stdout:7/279: rmdir d5/d14 39 2024-09-21T04:15:12.811 INFO:tasks.workunit.client.1.smithi148.stdout:7/280: creat d5/dc/d28/f57 x:0 0 0 2024-09-21T04:15:12.813 INFO:tasks.workunit.client.0.smithi110.stdout:5/346: rename d1/f38 to d1/d39/f79 0 2024-09-21T04:15:12.813 INFO:tasks.workunit.client.1.smithi148.stdout:7/281: dread d5/f6 [0,4194304] 0 2024-09-21T04:15:12.815 INFO:tasks.workunit.client.0.smithi110.stdout:9/219: getdents d7 0 2024-09-21T04:15:12.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/247: unlink d5/d1f/f3e 0 2024-09-21T04:15:12.817 INFO:tasks.workunit.client.1.smithi148.stdout:4/268: read d4/d23/d2a/f36 [1603284,40677] 0 2024-09-21T04:15:12.817 INFO:tasks.workunit.client.1.smithi148.stdout:4/269: write d4/d12/d16/f19 [1581425,14596] 0 2024-09-21T04:15:12.817 INFO:tasks.workunit.client.1.smithi148.stdout:4/270: truncate d4/d12/d16/f4f 83348 0 2024-09-21T04:15:12.819 INFO:tasks.workunit.client.0.smithi110.stdout:7/324: link dd/f1a dd/d1c/d39/f61 0 2024-09-21T04:15:12.821 INFO:tasks.workunit.client.1.smithi148.stdout:0/252: mknod d5/d6/d7/c4d 0 2024-09-21T04:15:12.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/253: stat d5/d6/d7/d11/f34 0 2024-09-21T04:15:12.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/254: rename d5/d6/d7 to d5/d6/d7/d4e 22 2024-09-21T04:15:12.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/255: chown d5/d6/d7/d22 2 1 2024-09-21T04:15:12.822 INFO:tasks.workunit.client.0.smithi110.stdout:5/347: read d1/dc/f1d [3340760,73885] 0 2024-09-21T04:15:12.823 INFO:tasks.workunit.client.1.smithi148.stdout:6/210: dwrite d0/d1a/d23/d16/f30 [0,4194304] 0 2024-09-21T04:15:12.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/234: mkdir dc/d43/d4c 0 2024-09-21T04:15:12.824 INFO:tasks.workunit.client.1.smithi148.stdout:8/235: readlink dc/l2d 0 2024-09-21T04:15:12.825 INFO:tasks.workunit.client.0.smithi110.stdout:0/596: creat d4/dd/d53/d96/fb0 x:0 0 0 2024-09-21T04:15:12.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/271: write d4/fb [3092792,34829] 0 2024-09-21T04:15:12.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/272: write d4/d23/f2b [1404793,30729] 0 2024-09-21T04:15:12.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/273: fsync d4/d23/d2a/f36 0 2024-09-21T04:15:12.829 INFO:tasks.workunit.client.0.smithi110.stdout:5/348: read d1/d39/f45 [2532002,28670] 0 2024-09-21T04:15:12.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/325: write dd/d4d/f59 [167543,22205] 0 2024-09-21T04:15:12.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/248: rmdir db/d24/d2a 39 2024-09-21T04:15:12.835 INFO:tasks.workunit.client.1.smithi148.stdout:2/326: symlink d0/d2b/l64 0 2024-09-21T04:15:12.835 INFO:tasks.workunit.client.1.smithi148.stdout:2/327: creat d0/d9/d59/f65 x:0 0 0 2024-09-21T04:15:12.836 INFO:tasks.workunit.client.0.smithi110.stdout:3/317: symlink d0/d12/d2a/l74 0 2024-09-21T04:15:12.838 INFO:tasks.workunit.client.1.smithi148.stdout:5/248: symlink d3/d14/l5a 0 2024-09-21T04:15:12.843 INFO:tasks.workunit.client.1.smithi148.stdout:7/282: mknod d5/dc/d24/c58 0 2024-09-21T04:15:12.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/359: mknod d0/d30/d34/d3e/d60/c79 0 2024-09-21T04:15:12.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/360: chown d0/d30/d34/d3e/d60/c79 86046573 1 2024-09-21T04:15:12.849 INFO:tasks.workunit.client.0.smithi110.stdout:6/267: rename d4/d12/d17/c28 to d4/d12/d17/d2e/d38/c57 0 2024-09-21T04:15:12.852 INFO:tasks.workunit.client.0.smithi110.stdout:9/220: symlink d7/d10/d24/d3c/l4b 0 2024-09-21T04:15:12.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/211: rename d0/d1a/f1c to d0/d1a/d23/d16/f39 0 2024-09-21T04:15:12.859 INFO:tasks.workunit.client.0.smithi110.stdout:0/597: symlink d4/d47/d2c/d3a/d84/lb1 0 2024-09-21T04:15:12.861 INFO:tasks.workunit.client.0.smithi110.stdout:5/349: creat d1/d49/d4d/d1e/d5a/f7a x:0 0 0 2024-09-21T04:15:12.861 INFO:tasks.workunit.client.1.smithi148.stdout:2/328: dwrite d0/d9/d59/f5a [0,4194304] 0 2024-09-21T04:15:12.865 INFO:tasks.workunit.client.1.smithi148.stdout:0/256: dwrite d5/d6/d7/d18/f32 [0,4194304] 0 2024-09-21T04:15:12.866 INFO:tasks.workunit.client.1.smithi148.stdout:0/257: chown d5/d6/f3c 0 1 2024-09-21T04:15:12.866 INFO:tasks.workunit.client.0.smithi110.stdout:7/326: dwrite dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:12.867 INFO:tasks.workunit.client.0.smithi110.stdout:3/318: stat d0/ld 0 2024-09-21T04:15:12.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/236: symlink dc/l4d 0 2024-09-21T04:15:12.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/237: readlink dc/l25 0 2024-09-21T04:15:12.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/238: chown dc/d43/f44 22 1 2024-09-21T04:15:12.867 INFO:tasks.workunit.client.1.smithi148.stdout:5/249: dwrite d3/d14/d1f/f42 [0,4194304] 0 2024-09-21T04:15:12.868 INFO:tasks.workunit.client.1.smithi148.stdout:8/239: dread dc/d13/d28/f48 [0,4194304] 0 2024-09-21T04:15:12.868 INFO:tasks.workunit.client.1.smithi148.stdout:8/240: write dc/d43/f47 [967758,96218] 0 2024-09-21T04:15:12.869 INFO:tasks.workunit.client.0.smithi110.stdout:4/413: rename d17/d34/f8a to d17/d34/d3d/f91 0 2024-09-21T04:15:12.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/268: symlink d4/d12/d17/d2e/l58 0 2024-09-21T04:15:12.871 INFO:tasks.workunit.client.0.smithi110.stdout:6/269: write d4/fb [4851405,101319] 0 2024-09-21T04:15:12.874 INFO:tasks.workunit.client.0.smithi110.stdout:9/221: link d7/d10/d24/d3c/c1d d7/d10/d24/d3c/d1f/d43/c4c 0 2024-09-21T04:15:12.874 INFO:tasks.workunit.client.0.smithi110.stdout:9/222: readlink d7/l4a 0 2024-09-21T04:15:12.874 INFO:tasks.workunit.client.0.smithi110.stdout:9/223: dread - d7/d10/f3f zero size 2024-09-21T04:15:12.874 INFO:tasks.workunit.client.0.smithi110.stdout:9/224: truncate d7/d10/d24/d3c/d1f/f41 450264 0 2024-09-21T04:15:12.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/231: getdents d3/d5/d6/d12/da/d23/d28 0 2024-09-21T04:15:12.883 INFO:tasks.workunit.client.0.smithi110.stdout:6/270: write d4/fa [4966413,20211] 0 2024-09-21T04:15:12.883 INFO:tasks.workunit.client.0.smithi110.stdout:6/271: truncate d4/d12/d16/f56 244859 0 2024-09-21T04:15:12.883 INFO:tasks.workunit.client.0.smithi110.stdout:6/272: creat d4/d12/d16/f59 x:0 0 0 2024-09-21T04:15:12.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/319: creat d0/d12/d3c/d5a/d65/f75 x:0 0 0 2024-09-21T04:15:12.884 INFO:tasks.workunit.client.0.smithi110.stdout:4/414: link d17/f27 d17/d34/d3d/d20/d43/f92 0 2024-09-21T04:15:12.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/249: mknod db/d24/d2a/c51 0 2024-09-21T04:15:12.885 INFO:tasks.workunit.client.1.smithi148.stdout:1/250: readlink db/l41 0 2024-09-21T04:15:12.888 INFO:tasks.workunit.client.0.smithi110.stdout:9/225: unlink d7/d10/d24/d3c/d1f/d43/c45 0 2024-09-21T04:15:12.890 INFO:tasks.workunit.client.0.smithi110.stdout:7/327: getdents dd/d14 0 2024-09-21T04:15:12.890 INFO:tasks.workunit.client.0.smithi110.stdout:7/328: dread - dd/d14/d17/f54 zero size 2024-09-21T04:15:12.895 INFO:tasks.workunit.client.0.smithi110.stdout:4/415: write f5 [2920883,108345] 0 2024-09-21T04:15:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:3/320: mkdir d0/d22/d29/d76 0 2024-09-21T04:15:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:3/321: chown d0/d12/d16/d1a 19452 1 2024-09-21T04:15:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:3/322: chown d0/d12/d16/d1a/f39 54 1 2024-09-21T04:15:12.900 INFO:tasks.workunit.client.0.smithi110.stdout:9/226: creat d7/d10/d24/d3c/d14/d3e/f4d x:0 0 0 2024-09-21T04:15:12.902 INFO:tasks.workunit.client.0.smithi110.stdout:4/416: creat d17/d1e/d7b/f93 x:0 0 0 2024-09-21T04:15:12.902 INFO:tasks.workunit.client.0.smithi110.stdout:4/417: fsync fa 0 2024-09-21T04:15:12.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/418: stat c15 0 2024-09-21T04:15:12.906 INFO:tasks.workunit.client.0.smithi110.stdout:5/350: dwrite d1/d7/d18/f28 [0,4194304] 0 2024-09-21T04:15:12.907 INFO:tasks.workunit.client.0.smithi110.stdout:9/227: dread d7/d10/f13 [0,4194304] 0 2024-09-21T04:15:12.907 INFO:tasks.workunit.client.0.smithi110.stdout:9/228: chown d7/d10/d24/d3c/d1f/f41 9479388 1 2024-09-21T04:15:12.908 INFO:tasks.workunit.client.0.smithi110.stdout:3/323: getdents d0 0 2024-09-21T04:15:12.910 INFO:tasks.workunit.client.1.smithi148.stdout:5/250: creat d3/f5b x:0 0 0 2024-09-21T04:15:12.910 INFO:tasks.workunit.client.0.smithi110.stdout:9/229: dread f2 [0,4194304] 0 2024-09-21T04:15:12.912 INFO:tasks.workunit.client.0.smithi110.stdout:5/351: mkdir d1/d49/d4d/d1e/d5a/d64/d7b 0 2024-09-21T04:15:12.912 INFO:tasks.workunit.client.0.smithi110.stdout:4/419: dread d17/d3e/f40 [0,4194304] 0 2024-09-21T04:15:12.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/420: readlink d17/d1e/d21/l76 0 2024-09-21T04:15:12.914 INFO:tasks.workunit.client.0.smithi110.stdout:3/324: creat d0/f77 x:0 0 0 2024-09-21T04:15:12.916 INFO:tasks.workunit.client.0.smithi110.stdout:9/230: creat d7/d10/d24/d3c/d11/f4e x:0 0 0 2024-09-21T04:15:12.917 INFO:tasks.workunit.client.1.smithi148.stdout:2/329: dwrite d0/d9/d38/f41 [0,4194304] 0 2024-09-21T04:15:12.918 INFO:tasks.workunit.client.0.smithi110.stdout:4/421: link l11 d17/d34/d3d/d20/d43/l94 0 2024-09-21T04:15:12.918 INFO:tasks.workunit.client.1.smithi148.stdout:8/241: dwrite dc/d13/d39/f3b [0,4194304] 0 2024-09-21T04:15:12.918 INFO:tasks.workunit.client.1.smithi148.stdout:4/274: rmdir d4/d12 39 2024-09-21T04:15:12.918 INFO:tasks.workunit.client.1.smithi148.stdout:0/258: dwrite d5/d6/d2a/d3f/f4c [0,4194304] 0 2024-09-21T04:15:12.919 INFO:tasks.workunit.client.0.smithi110.stdout:6/273: dwrite d4/f3d [0,4194304] 0 2024-09-21T04:15:12.919 INFO:tasks.workunit.client.0.smithi110.stdout:6/274: chown d4/d12/d17/d2e/d40/f2f 1296 1 2024-09-21T04:15:12.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/232: mkdir d3/d48 0 2024-09-21T04:15:12.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/233: truncate d3/d5/d6/d12/da/d21/f31 257537 0 2024-09-21T04:15:12.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/234: write d3/d5/d6/d12/da/d21/f31 [787154,36744] 0 2024-09-21T04:15:12.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/235: getdents d3/d48 0 2024-09-21T04:15:12.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/236: chown l0 23336 1 2024-09-21T04:15:12.922 INFO:tasks.workunit.client.0.smithi110.stdout:7/329: dwrite dd/f57 [0,4194304] 0 2024-09-21T04:15:12.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/422: link d17/f32 d17/d34/d3d/d20/f95 0 2024-09-21T04:15:12.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/423: write d17/d1e/d64/f8f [478315,26999] 0 2024-09-21T04:15:12.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/424: write d17/d34/d3d/d20/d43/d51/d7a/d86/f89 [73510,19549] 0 2024-09-21T04:15:12.925 INFO:tasks.workunit.client.1.smithi148.stdout:1/251: truncate f6 1021462 0 2024-09-21T04:15:12.927 INFO:tasks.workunit.client.0.smithi110.stdout:6/275: unlink d4/fa 0 2024-09-21T04:15:12.927 INFO:tasks.workunit.client.0.smithi110.stdout:6/276: chown d4/f6 5 1 2024-09-21T04:15:12.928 INFO:tasks.workunit.client.1.smithi148.stdout:7/283: symlink d5/dc/l59 0 2024-09-21T04:15:12.930 INFO:tasks.workunit.client.1.smithi148.stdout:0/259: dread d5/d6/f16 [0,4194304] 0 2024-09-21T04:15:12.930 INFO:tasks.workunit.client.1.smithi148.stdout:0/260: creat d5/d6/d7/f4f x:0 0 0 2024-09-21T04:15:12.931 INFO:tasks.workunit.client.1.smithi148.stdout:5/251: symlink d3/d14/d1f/d26/d2e/d56/l5c 0 2024-09-21T04:15:12.931 INFO:tasks.workunit.client.0.smithi110.stdout:9/231: dread d7/d10/d24/d3c/f20 [0,4194304] 0 2024-09-21T04:15:12.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/232: write d7/d26/f2c [1080957,36597] 0 2024-09-21T04:15:12.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/233: truncate f2 1443563 0 2024-09-21T04:15:12.932 INFO:tasks.workunit.client.0.smithi110.stdout:3/325: getdents d0/d12/d3c 0 2024-09-21T04:15:12.933 INFO:tasks.workunit.client.1.smithi148.stdout:2/330: rmdir d0 39 2024-09-21T04:15:12.933 INFO:tasks.workunit.client.1.smithi148.stdout:2/331: chown d0/d9/d59/f65 12650016 1 2024-09-21T04:15:12.933 INFO:tasks.workunit.client.1.smithi148.stdout:2/332: write d0/d2b/f32 [373598,36069] 0 2024-09-21T04:15:12.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/237: dread d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:12.936 INFO:tasks.workunit.client.0.smithi110.stdout:7/330: getdents dd/d1c/d39/d4a 0 2024-09-21T04:15:12.939 INFO:tasks.workunit.client.1.smithi148.stdout:8/242: dwrite dc/d13/f35 [0,4194304] 0 2024-09-21T04:15:12.942 INFO:tasks.workunit.client.1.smithi148.stdout:4/275: rename d4/d12/d16/f34 to d4/d3a/f53 0 2024-09-21T04:15:12.943 INFO:tasks.workunit.client.1.smithi148.stdout:8/243: dread dc/d43/f47 [0,4194304] 0 2024-09-21T04:15:12.944 INFO:tasks.workunit.client.0.smithi110.stdout:4/425: mkdir d17/d34/d3d/d20/d43/d51/d7a/d96 0 2024-09-21T04:15:12.944 INFO:tasks.workunit.client.0.smithi110.stdout:4/426: fsync d17/d1e/d7b/f93 0 2024-09-21T04:15:12.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/234: mkdir d7/d10/d24/d3c/d4f 0 2024-09-21T04:15:12.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/235: write d7/d10/f3f [688973,66754] 0 2024-09-21T04:15:12.949 INFO:tasks.workunit.client.0.smithi110.stdout:3/326: mknod d0/c78 0 2024-09-21T04:15:12.949 INFO:tasks.workunit.client.0.smithi110.stdout:3/327: stat d0/d12/d3c/d5a/d70 0 2024-09-21T04:15:12.949 INFO:tasks.workunit.client.0.smithi110.stdout:3/328: write d0/d12/f5d [125397,97697] 0 2024-09-21T04:15:12.957 INFO:tasks.workunit.client.1.smithi148.stdout:0/261: creat d5/d6/d2a/d45/f50 x:0 0 0 2024-09-21T04:15:12.958 INFO:tasks.workunit.client.0.smithi110.stdout:3/329: dread d0/fe [0,4194304] 0 2024-09-21T04:15:12.959 INFO:tasks.workunit.client.1.smithi148.stdout:0/262: read d5/d6/d2a/d3f/f41 [489108,98167] 0 2024-09-21T04:15:12.959 INFO:tasks.workunit.client.1.smithi148.stdout:5/252: mkdir d3/d34/d3c/d5d 0 2024-09-21T04:15:12.959 INFO:tasks.workunit.client.1.smithi148.stdout:5/253: chown d3/d14/d1a/f31 1 1 2024-09-21T04:15:12.959 INFO:tasks.workunit.client.0.smithi110.stdout:3/330: read d0/d12/f5d [24550,13151] 0 2024-09-21T04:15:12.960 INFO:tasks.workunit.client.0.smithi110.stdout:3/331: dread - d0/d12/d16/d1a/f68 zero size 2024-09-21T04:15:12.960 INFO:tasks.workunit.client.0.smithi110.stdout:3/332: creat d0/d12/d16/d1a/d5c/f79 x:0 0 0 2024-09-21T04:15:12.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/333: creat d0/d9/dc/d13/d4c/f66 x:0 0 0 2024-09-21T04:15:12.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/238: creat d3/f49 x:0 0 0 2024-09-21T04:15:12.963 INFO:tasks.workunit.client.0.smithi110.stdout:4/427: write d17/d34/d3d/d20/f95 [1826288,14984] 0 2024-09-21T04:15:12.963 INFO:tasks.workunit.client.0.smithi110.stdout:4/428: write fa [1152186,113768] 0 2024-09-21T04:15:12.965 INFO:tasks.workunit.client.0.smithi110.stdout:6/277: dwrite d4/d12/d17/d2e/d40/f2f [4194304,4194304] 0 2024-09-21T04:15:12.965 INFO:tasks.workunit.client.0.smithi110.stdout:9/236: link d7/d10/d24/d3c/d14/d3e/l44 d7/d10/d24/d3c/d4f/l50 0 2024-09-21T04:15:12.965 INFO:tasks.workunit.client.0.smithi110.stdout:9/237: stat d7/d10/d24/d3c/d4f 0 2024-09-21T04:15:12.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/254: write d3/d14/d1f/f23 [882388,75859] 0 2024-09-21T04:15:12.968 INFO:tasks.workunit.client.0.smithi110.stdout:7/331: dwrite dd/d1c/d39/f3c [0,4194304] 0 2024-09-21T04:15:12.968 INFO:tasks.workunit.client.0.smithi110.stdout:7/332: truncate dd/d1c/d39/f61 1487360 0 2024-09-21T04:15:12.968 INFO:tasks.workunit.client.0.smithi110.stdout:7/333: dread - dd/d14/d17/f5b zero size 2024-09-21T04:15:12.969 INFO:tasks.workunit.client.1.smithi148.stdout:4/276: dwrite d4/d12/d2c/f32 [4194304,4194304] 0 2024-09-21T04:15:12.970 INFO:tasks.workunit.client.1.smithi148.stdout:8/244: mknod dc/d2a/d42/c4e 0 2024-09-21T04:15:12.973 INFO:tasks.workunit.client.1.smithi148.stdout:2/334: dread d0/f1d [0,4194304] 0 2024-09-21T04:15:12.974 INFO:tasks.workunit.client.1.smithi148.stdout:2/335: chown d0/d9/d59/f65 8771 1 2024-09-21T04:15:12.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/284: unlink d5/dc/c33 0 2024-09-21T04:15:12.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/285: creat d5/f5a x:0 0 0 2024-09-21T04:15:12.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/286: write d5/f3d [2756502,20926] 0 2024-09-21T04:15:12.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/263: truncate d5/d6/d2a/d2b/f35 1554074 0 2024-09-21T04:15:12.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/264: fdatasync d5/d6/d2a/d3f/f4c 0 2024-09-21T04:15:12.979 INFO:tasks.workunit.client.1.smithi148.stdout:7/287: write d5/d8/f22 [7522,129335] 0 2024-09-21T04:15:12.980 INFO:tasks.workunit.client.0.smithi110.stdout:4/429: creat d17/d1e/d77/f97 x:0 0 0 2024-09-21T04:15:12.981 INFO:tasks.workunit.client.1.smithi148.stdout:3/239: creat d3/d5/d6/d12/da/d19/f4a x:0 0 0 2024-09-21T04:15:12.981 INFO:tasks.workunit.client.1.smithi148.stdout:3/240: dread - d3/d5/d6/d12/da/d19/f4a zero size 2024-09-21T04:15:12.981 INFO:tasks.workunit.client.1.smithi148.stdout:3/241: read d3/d5/d6/f17 [835708,74587] 0 2024-09-21T04:15:12.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/333: getdents d0/d12 0 2024-09-21T04:15:12.988 INFO:tasks.workunit.client.1.smithi148.stdout:0/265: write d5/d6/d7/ff [1426928,32953] 0 2024-09-21T04:15:12.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/252: rename db/d24/c36 to db/c52 0 2024-09-21T04:15:12.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/253: chown db/d24/d2a/c51 16255750 1 2024-09-21T04:15:12.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/254: creat db/d46/f53 x:0 0 0 2024-09-21T04:15:12.991 INFO:tasks.workunit.client.0.smithi110.stdout:4/430: mkdir d17/d34/d3d/d20/d43/d51/d52/d98 0 2024-09-21T04:15:12.993 INFO:tasks.workunit.client.1.smithi148.stdout:8/245: mkdir dc/d43/d4c/d4f 0 2024-09-21T04:15:12.993 INFO:tasks.workunit.client.0.smithi110.stdout:6/278: rmdir d4/d12/d17/d33/d3e 39 2024-09-21T04:15:12.995 INFO:tasks.workunit.client.1.smithi148.stdout:8/246: dread dc/d2a/f2e [0,4194304] 0 2024-09-21T04:15:12.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/336: creat d0/d9/dc/d13/d2c/f67 x:0 0 0 2024-09-21T04:15:12.999 INFO:tasks.workunit.client.0.smithi110.stdout:3/334: link d0/d22/l59 d0/d12/d16/l7a 0 2024-09-21T04:15:12.999 INFO:tasks.workunit.client.1.smithi148.stdout:5/255: dwrite d3/d14/d1a/f33 [0,4194304] 0 2024-09-21T04:15:13.000 INFO:tasks.workunit.client.0.smithi110.stdout:3/335: write d0/fe [338901,70440] 0 2024-09-21T04:15:13.000 INFO:tasks.workunit.client.0.smithi110.stdout:9/238: dwrite d7/d26/f2c [0,4194304] 0 2024-09-21T04:15:13.000 INFO:tasks.workunit.client.1.smithi148.stdout:3/242: dwrite d3/d5/d6/d12/da/d23/f26 [4194304,4194304] 0 2024-09-21T04:15:13.001 INFO:tasks.workunit.client.1.smithi148.stdout:3/243: truncate d3/d5/d6/d12/da/d23/d28/f42 581511 0 2024-09-21T04:15:13.001 INFO:tasks.workunit.client.0.smithi110.stdout:4/431: link d17/d1e/f4c d17/d34/d3d/d20/d43/d51/d52/d98/f99 0 2024-09-21T04:15:13.001 INFO:tasks.workunit.client.0.smithi110.stdout:4/432: creat d17/d3e/f9a x:0 0 0 2024-09-21T04:15:13.001 INFO:tasks.workunit.client.0.smithi110.stdout:4/433: write d17/f1a [212162,18367] 0 2024-09-21T04:15:13.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/288: symlink d5/d14/d25/d40/l5b 0 2024-09-21T04:15:13.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/289: dread - d5/f5a zero size 2024-09-21T04:15:13.004 INFO:tasks.workunit.client.0.smithi110.stdout:6/279: symlink d4/d12/d17/d33/d3e/l5a 0 2024-09-21T04:15:13.004 INFO:tasks.workunit.client.0.smithi110.stdout:6/280: readlink d4/de/l19 0 2024-09-21T04:15:13.004 INFO:tasks.workunit.client.0.smithi110.stdout:4/434: dread d17/f1a [0,4194304] 0 2024-09-21T04:15:13.004 INFO:tasks.workunit.client.0.smithi110.stdout:4/435: write d17/d34/d3d/f4a [394046,30418] 0 2024-09-21T04:15:13.004 INFO:tasks.workunit.client.0.smithi110.stdout:4/436: read - d17/d1e/d7b/f5e zero size 2024-09-21T04:15:13.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/266: symlink d5/d6/d7/l51 0 2024-09-21T04:15:13.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/267: readlink d5/d6/l14 0 2024-09-21T04:15:13.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/268: write d5/d6/d2a/d3f/f4c [5180129,22447] 0 2024-09-21T04:15:13.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/269: chown d5/d6/d7/d11/f1d 499 1 2024-09-21T04:15:13.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/270: chown d5/l25 63572 1 2024-09-21T04:15:13.008 INFO:tasks.workunit.client.1.smithi148.stdout:9/248: sync 2024-09-21T04:15:13.008 INFO:tasks.workunit.client.1.smithi148.stdout:6/212: sync 2024-09-21T04:15:13.011 INFO:tasks.workunit.client.1.smithi148.stdout:3/244: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:13.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/255: mknod db/d46/d4c/c54 0 2024-09-21T04:15:13.014 INFO:tasks.workunit.client.0.smithi110.stdout:9/239: getdents d7/d10/d24/d3c/d14 0 2024-09-21T04:15:13.014 INFO:tasks.workunit.client.0.smithi110.stdout:9/240: chown d7/d10/d24/d3c/d14/l21 11 1 2024-09-21T04:15:13.015 INFO:tasks.workunit.client.1.smithi148.stdout:8/247: mkdir dc/d2a/d50 0 2024-09-21T04:15:13.015 INFO:tasks.workunit.client.1.smithi148.stdout:8/248: read dc/d13/f24 [29856,117919] 0 2024-09-21T04:15:13.019 INFO:tasks.workunit.client.1.smithi148.stdout:0/271: dread d5/d6/d7/d18/f32 [0,4194304] 0 2024-09-21T04:15:13.019 INFO:tasks.workunit.client.0.smithi110.stdout:4/437: link d17/d34/d3d/d20/d43/d51/d52/d98/f99 d17/d34/d3d/d20/d43/d51/d7a/d86/f9b 0 2024-09-21T04:15:13.021 INFO:tasks.workunit.client.0.smithi110.stdout:6/281: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:13.022 INFO:tasks.workunit.client.0.smithi110.stdout:9/241: mknod d7/d10/d24/d3c/d35/c51 0 2024-09-21T04:15:13.022 INFO:tasks.workunit.client.0.smithi110.stdout:9/242: getdents d7/d10/d24/d3c/d11/d33 0 2024-09-21T04:15:13.022 INFO:tasks.workunit.client.0.smithi110.stdout:9/243: creat d7/d10/d24/f52 x:0 0 0 2024-09-21T04:15:13.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/244: readlink d7/d10/d24/l27 0 2024-09-21T04:15:13.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/245: fdatasync d7/d10/d24/d3c/d35/f3a 0 2024-09-21T04:15:13.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/246: fsync d7/d10/d24/f32 0 2024-09-21T04:15:13.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/247: write f2 [150123,29591] 0 2024-09-21T04:15:13.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/248: read d7/d10/d24/d3c/f20 [1840437,100182] 0 2024-09-21T04:15:13.030 INFO:tasks.workunit.client.1.smithi148.stdout:6/213: dwrite d0/f21 [4194304,4194304] 0 2024-09-21T04:15:13.032 INFO:tasks.workunit.client.1.smithi148.stdout:7/290: creat d5/dc/d28/d4a/f5c x:0 0 0 2024-09-21T04:15:13.032 INFO:tasks.workunit.client.1.smithi148.stdout:7/291: dread - d5/dc/f50 zero size 2024-09-21T04:15:13.033 INFO:tasks.workunit.client.0.smithi110.stdout:9/249: creat d7/d10/d24/d3c/d14/d31/f53 x:0 0 0 2024-09-21T04:15:13.033 INFO:tasks.workunit.client.0.smithi110.stdout:9/250: fdatasync d7/d26/f2c 0 2024-09-21T04:15:13.033 INFO:tasks.workunit.client.0.smithi110.stdout:6/282: dread d4/de/f44 [0,4194304] 0 2024-09-21T04:15:13.034 INFO:tasks.workunit.client.1.smithi148.stdout:9/249: creat d5/d60/d28/d2f/f62 x:0 0 0 2024-09-21T04:15:13.036 INFO:tasks.workunit.client.1.smithi148.stdout:4/277: rename d4/d12/d2c/d2d/d44/c45 to d4/d12/c54 0 2024-09-21T04:15:13.036 INFO:tasks.workunit.client.1.smithi148.stdout:5/256: rename d3/d34 to d3/d34/d3c/d5d/d5e 22 2024-09-21T04:15:13.037 INFO:tasks.workunit.client.0.smithi110.stdout:6/283: link d4/d12/d17/d33/d3e/f53 d4/d12/d16/f5b 0 2024-09-21T04:15:13.037 INFO:tasks.workunit.client.0.smithi110.stdout:6/284: chown d4/d12/d17/d33 201854333 1 2024-09-21T04:15:13.037 INFO:tasks.workunit.client.0.smithi110.stdout:6/285: fdatasync d4/d12/f2b 0 2024-09-21T04:15:13.038 INFO:tasks.workunit.client.1.smithi148.stdout:1/256: creat db/d24/d4f/f55 x:0 0 0 2024-09-21T04:15:13.039 INFO:tasks.workunit.client.1.smithi148.stdout:8/249: creat dc/d13/d39/f51 x:0 0 0 2024-09-21T04:15:13.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/286: mkdir d4/de/d5c 0 2024-09-21T04:15:13.040 INFO:tasks.workunit.client.1.smithi148.stdout:0/272: truncate d5/d6/d7/d11/f29 2977305 0 2024-09-21T04:15:13.042 INFO:tasks.workunit.client.1.smithi148.stdout:7/292: creat d5/dc/f5d x:0 0 0 2024-09-21T04:15:13.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/287: creat d4/d12/d17/d2e/d40/f5d x:0 0 0 2024-09-21T04:15:13.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/288: write d4/f13 [3514221,71448] 0 2024-09-21T04:15:13.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/289: truncate d4/d12/d16/f59 985879 0 2024-09-21T04:15:13.044 INFO:tasks.workunit.client.1.smithi148.stdout:9/250: creat d5/f63 x:0 0 0 2024-09-21T04:15:13.045 INFO:tasks.workunit.client.0.smithi110.stdout:4/438: dwrite d17/d34/d3d/d20/d43/d51/d52/f59 [0,4194304] 0 2024-09-21T04:15:13.045 INFO:tasks.workunit.client.0.smithi110.stdout:4/439: creat d17/d3e/f9c x:0 0 0 2024-09-21T04:15:13.046 INFO:tasks.workunit.client.1.smithi148.stdout:3/245: rename d3/d1f/c44 to d3/d5/d6/d12/da/d21/c4b 0 2024-09-21T04:15:13.047 INFO:tasks.workunit.client.1.smithi148.stdout:5/257: creat d3/d37/f5f x:0 0 0 2024-09-21T04:15:13.047 INFO:tasks.workunit.client.1.smithi148.stdout:5/258: truncate d3/d14/d1f/f42 5041661 0 2024-09-21T04:15:13.048 INFO:tasks.workunit.client.1.smithi148.stdout:1/257: getdents db/dd 0 2024-09-21T04:15:13.049 INFO:tasks.workunit.client.1.smithi148.stdout:8/250: getdents dc/d13/d28 0 2024-09-21T04:15:13.051 INFO:tasks.workunit.client.0.smithi110.stdout:4/440: rename d17/d3e/d6b/f6e to d17/d1e/d64/f9d 0 2024-09-21T04:15:13.051 INFO:tasks.workunit.client.0.smithi110.stdout:4/441: write d17/f58 [1565225,72585] 0 2024-09-21T04:15:13.051 INFO:tasks.workunit.client.1.smithi148.stdout:0/273: creat d5/f52 x:0 0 0 2024-09-21T04:15:13.052 INFO:tasks.workunit.client.1.smithi148.stdout:7/293: symlink d5/d14/l5e 0 2024-09-21T04:15:13.053 INFO:tasks.workunit.client.1.smithi148.stdout:3/246: truncate d3/d5/f1e 3277560 0 2024-09-21T04:15:13.055 INFO:tasks.workunit.client.0.smithi110.stdout:4/442: link d17/d34/d3d/d57/l8e d17/d1e/d21/l9e 0 2024-09-21T04:15:13.055 INFO:tasks.workunit.client.0.smithi110.stdout:4/443: chown d17/d3e/f8b 12622264 1 2024-09-21T04:15:13.055 INFO:tasks.workunit.client.0.smithi110.stdout:4/444: truncate d17/d1e/d7b/f5e 1031873 0 2024-09-21T04:15:13.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/259: getdents d3/d14/d1f/d26 0 2024-09-21T04:15:13.056 INFO:tasks.workunit.client.1.smithi148.stdout:9/251: write d5/f23 [5273121,79484] 0 2024-09-21T04:15:13.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/251: link dc/d2a/c3f dc/d2a/d50/c52 0 2024-09-21T04:15:13.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/252: creat dc/d2a/d42/f53 x:0 0 0 2024-09-21T04:15:13.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/253: creat dc/d43/f54 x:0 0 0 2024-09-21T04:15:13.057 INFO:tasks.workunit.client.1.smithi148.stdout:5/260: dread d3/d4a/f52 [0,4194304] 0 2024-09-21T04:15:13.057 INFO:tasks.workunit.client.1.smithi148.stdout:5/261: dread - d3/d14/f29 zero size 2024-09-21T04:15:13.058 INFO:tasks.workunit.client.1.smithi148.stdout:8/254: dread dc/d13/f24 [0,4194304] 0 2024-09-21T04:15:13.058 INFO:tasks.workunit.client.0.smithi110.stdout:4/445: rename d17/d34/f65 to d17/d34/d3d/d57/f9f 0 2024-09-21T04:15:13.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/274: truncate f4 8263465 0 2024-09-21T04:15:13.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/275: chown d5/d6/d7/ff 41 1 2024-09-21T04:15:13.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/276: chown d5/d6/d3e/c47 41 1 2024-09-21T04:15:13.060 INFO:tasks.workunit.client.1.smithi148.stdout:3/247: unlink d3/l2e 0 2024-09-21T04:15:13.061 INFO:tasks.workunit.client.0.smithi110.stdout:4/446: mknod d17/d34/d3d/d20/d43/d51/ca0 0 2024-09-21T04:15:13.062 INFO:tasks.workunit.client.1.smithi148.stdout:3/248: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:13.063 INFO:tasks.workunit.client.1.smithi148.stdout:9/252: chown d5/l13 6854 1 2024-09-21T04:15:13.065 INFO:tasks.workunit.client.0.smithi110.stdout:4/447: unlink d17/l36 0 2024-09-21T04:15:13.065 INFO:tasks.workunit.client.0.smithi110.stdout:4/448: write d17/d34/d3d/d20/d43/f8c [4545279,39812] 0 2024-09-21T04:15:13.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/255: dread dc/f18 [0,4194304] 0 2024-09-21T04:15:13.069 INFO:tasks.workunit.client.1.smithi148.stdout:5/262: unlink d3/l13 0 2024-09-21T04:15:13.070 INFO:tasks.workunit.client.1.smithi148.stdout:0/277: unlink d5/d6/d7/l51 0 2024-09-21T04:15:13.071 INFO:tasks.workunit.client.1.smithi148.stdout:0/278: write d5/d6/d7/ff [1477999,55769] 0 2024-09-21T04:15:13.071 INFO:tasks.workunit.client.1.smithi148.stdout:0/279: fdatasync d5/d6/d7/d11/f29 0 2024-09-21T04:15:13.071 INFO:tasks.workunit.client.1.smithi148.stdout:0/280: chown d5/d6/d7/d11/f38 9 1 2024-09-21T04:15:13.071 INFO:tasks.workunit.client.1.smithi148.stdout:9/253: dread d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:13.071 INFO:tasks.workunit.client.1.smithi148.stdout:7/294: dread d5/d8/ff [0,4194304] 0 2024-09-21T04:15:13.072 INFO:tasks.workunit.client.1.smithi148.stdout:3/249: rename d3/d5/d6/d12/da/d21/f31 to d3/f4c 0 2024-09-21T04:15:13.072 INFO:tasks.workunit.client.1.smithi148.stdout:3/250: chown d3/d5/d6/d12/l1c 5 1 2024-09-21T04:15:13.072 INFO:tasks.workunit.client.1.smithi148.stdout:8/256: symlink dc/d13/d39/l55 0 2024-09-21T04:15:13.073 INFO:tasks.workunit.client.1.smithi148.stdout:5/263: creat d3/d14/d1f/d26/d2e/d3d/f60 x:0 0 0 2024-09-21T04:15:13.074 INFO:tasks.workunit.client.1.smithi148.stdout:0/281: mknod d5/d6/d2a/d2b/c53 0 2024-09-21T04:15:13.078 INFO:tasks.workunit.client.1.smithi148.stdout:9/254: creat d5/d1e/d37/f64 x:0 0 0 2024-09-21T04:15:13.078 INFO:tasks.workunit.client.1.smithi148.stdout:6/214: dwrite d0/d1a/d23/fb [0,4194304] 0 2024-09-21T04:15:13.080 INFO:tasks.workunit.client.0.smithi110.stdout:9/251: dwrite f6 [0,4194304] 0 2024-09-21T04:15:13.080 INFO:tasks.workunit.client.1.smithi148.stdout:7/295: dread - d5/dc/f3e zero size 2024-09-21T04:15:13.080 INFO:tasks.workunit.client.1.smithi148.stdout:7/296: write d5/d14/d25/f54 [1466562,15151] 0 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:5/264: truncate d3/d14/d1f/d26/d2e/d56/f15 2609551 0 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:0/282: dread d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:5/265: creat d3/d14/d1f/d26/d2e/d3d/f61 x:0 0 0 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:0/283: dread - d5/f52 zero size 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:0/284: chown d5/d6/d7/d22/f46 352997548 1 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:0/285: stat d5/d6/d7 0 2024-09-21T04:15:13.083 INFO:tasks.workunit.client.1.smithi148.stdout:0/286: fsync d5/d6/d2a/d2b/f39 0 2024-09-21T04:15:13.084 INFO:tasks.workunit.client.1.smithi148.stdout:0/287: fdatasync d5/d6/d2a/d2b/f35 0 2024-09-21T04:15:13.084 INFO:tasks.workunit.client.1.smithi148.stdout:4/278: dwrite d4/d12/d2c/d2d/d44/f4b [0,4194304] 0 2024-09-21T04:15:13.084 INFO:tasks.workunit.client.1.smithi148.stdout:4/279: write d4/d12/d16/f19 [1908658,103614] 0 2024-09-21T04:15:13.084 INFO:tasks.workunit.client.1.smithi148.stdout:4/280: write d4/fe [526445,126279] 0 2024-09-21T04:15:13.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/215: creat d0/d34/d38/f3a x:0 0 0 2024-09-21T04:15:13.087 INFO:tasks.workunit.client.1.smithi148.stdout:8/257: dread fb [0,4194304] 0 2024-09-21T04:15:13.087 INFO:tasks.workunit.client.1.smithi148.stdout:8/258: write dc/f31 [841196,59626] 0 2024-09-21T04:15:13.091 INFO:tasks.workunit.client.1.smithi148.stdout:8/259: dread dc/d13/f24 [0,4194304] 0 2024-09-21T04:15:13.095 INFO:tasks.workunit.client.1.smithi148.stdout:5/266: mknod d3/d14/d1f/d26/d2e/c62 0 2024-09-21T04:15:13.095 INFO:tasks.workunit.client.1.smithi148.stdout:5/267: fsync d3/d14/d1a/f33 0 2024-09-21T04:15:13.095 INFO:tasks.workunit.client.1.smithi148.stdout:5/268: truncate d3/d14/d1a/f4f 7529 0 2024-09-21T04:15:13.096 INFO:tasks.workunit.client.0.smithi110.stdout:9/252: creat d7/d10/d24/d3c/d14/f54 x:0 0 0 2024-09-21T04:15:13.096 INFO:tasks.workunit.client.0.smithi110.stdout:9/253: dread - d7/d10/d24/d3c/d14/d31/f53 zero size 2024-09-21T04:15:13.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/288: truncate d5/d6/d7/d22/f24 1523894 0 2024-09-21T04:15:13.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/289: write d5/d6/d7/d22/f37 [393888,14321] 0 2024-09-21T04:15:13.104 INFO:tasks.workunit.client.1.smithi148.stdout:3/251: dwrite d3/d1f/f47 [0,4194304] 0 2024-09-21T04:15:13.109 INFO:tasks.workunit.client.1.smithi148.stdout:6/216: symlink d0/d1a/d23/d16/d36/l3b 0 2024-09-21T04:15:13.111 INFO:tasks.workunit.client.1.smithi148.stdout:5/269: symlink d3/d34/d3c/l63 0 2024-09-21T04:15:13.112 INFO:tasks.workunit.client.1.smithi148.stdout:0/290: mknod d5/c54 0 2024-09-21T04:15:13.116 INFO:tasks.workunit.client.1.smithi148.stdout:3/252: unlink d3/d5/d6/d12/da/l10 0 2024-09-21T04:15:13.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/270: rename d3/d14/f29 to d3/d34/f64 0 2024-09-21T04:15:13.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/271: write d3/f1b [1230763,119398] 0 2024-09-21T04:15:13.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/272: chown d3/d24/c4c 80144353 1 2024-09-21T04:15:13.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/273: write d3/d4a/f52 [924712,129625] 0 2024-09-21T04:15:13.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/291: creat d5/d6/d2a/d2b/f55 x:0 0 0 2024-09-21T04:15:13.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/292: truncate d5/d6/d2a/d2b/f39 4719414 0 2024-09-21T04:15:13.119 INFO:tasks.workunit.client.1.smithi148.stdout:3/253: mkdir d3/d1f/d4d 0 2024-09-21T04:15:13.121 INFO:tasks.workunit.client.1.smithi148.stdout:5/274: rename d3/d14/d1a/l2b to d3/d14/d1f/l65 0 2024-09-21T04:15:13.122 INFO:tasks.workunit.client.1.smithi148.stdout:0/293: mknod d5/d6/d2a/d45/c56 0 2024-09-21T04:15:13.123 INFO:tasks.workunit.client.0.smithi110.stdout:4/449: dwrite d17/d34/d3d/d20/f26 [0,4194304] 0 2024-09-21T04:15:13.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/297: dwrite d5/dc/f5d [0,4194304] 0 2024-09-21T04:15:13.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/298: write d5/d14/d25/f52 [545540,70718] 0 2024-09-21T04:15:13.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/299: fsync d5/d14/d25/f4c 0 2024-09-21T04:15:13.124 INFO:tasks.workunit.client.1.smithi148.stdout:0/294: symlink d5/d6/d2a/d45/l57 0 2024-09-21T04:15:13.125 INFO:tasks.workunit.client.0.smithi110.stdout:4/450: dread d17/d1e/d64/f8f [0,4194304] 0 2024-09-21T04:15:13.125 INFO:tasks.workunit.client.0.smithi110.stdout:4/451: chown d17/f2a 463557 1 2024-09-21T04:15:13.128 INFO:tasks.workunit.client.1.smithi148.stdout:4/281: dwrite d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:13.128 INFO:tasks.workunit.client.1.smithi148.stdout:8/260: dwrite f6 [0,4194304] 0 2024-09-21T04:15:13.129 INFO:tasks.workunit.client.0.smithi110.stdout:4/452: creat d17/d34/d3d/d20/d43/d51/fa1 x:0 0 0 2024-09-21T04:15:13.129 INFO:tasks.workunit.client.0.smithi110.stdout:4/453: chown d17/d34/d3d/d20/d43/c68 23 1 2024-09-21T04:15:13.132 INFO:tasks.workunit.client.1.smithi148.stdout:6/217: dwrite d0/d1a/d23/d16/f1f [4194304,4194304] 0 2024-09-21T04:15:13.134 INFO:tasks.workunit.client.0.smithi110.stdout:4/454: creat d17/d34/d3d/d20/d43/fa2 x:0 0 0 2024-09-21T04:15:13.134 INFO:tasks.workunit.client.0.smithi110.stdout:4/455: chown d17/d34/d3d/d20/f95 31903403 1 2024-09-21T04:15:13.134 INFO:tasks.workunit.client.0.smithi110.stdout:4/456: readlink d17/d34/d3d/d20/d43/l63 0 2024-09-21T04:15:13.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/282: unlink d4/c13 0 2024-09-21T04:15:13.136 INFO:tasks.workunit.client.1.smithi148.stdout:5/275: dread d3/f12 [0,4194304] 0 2024-09-21T04:15:13.136 INFO:tasks.workunit.client.1.smithi148.stdout:5/276: chown d3/d37/l21 939778789 1 2024-09-21T04:15:13.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/261: link dc/d2a/d50/c52 dc/d43/c56 0 2024-09-21T04:15:13.137 INFO:tasks.workunit.client.1.smithi148.stdout:8/262: fdatasync f7 0 2024-09-21T04:15:13.143 INFO:tasks.workunit.client.1.smithi148.stdout:6/218: mkdir d0/d34/d3c 0 2024-09-21T04:15:13.145 INFO:tasks.workunit.client.1.smithi148.stdout:4/283: unlink d4/d23/f40 0 2024-09-21T04:15:13.145 INFO:tasks.workunit.client.1.smithi148.stdout:4/284: dread - d4/f51 zero size 2024-09-21T04:15:13.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/295: getdents d5/d6/d2a/d2b 0 2024-09-21T04:15:13.149 INFO:tasks.workunit.client.1.smithi148.stdout:7/300: dwrite d5/dc/d28/f53 [0,4194304] 0 2024-09-21T04:15:13.150 INFO:tasks.workunit.client.1.smithi148.stdout:3/254: dwrite f1 [0,4194304] 0 2024-09-21T04:15:13.155 INFO:tasks.workunit.client.0.smithi110.stdout:4/457: dwrite d17/d34/f37 [0,4194304] 0 2024-09-21T04:15:13.160 INFO:tasks.workunit.client.1.smithi148.stdout:5/277: link d3/d24/f43 d3/d14/d1f/d26/f66 0 2024-09-21T04:15:13.161 INFO:tasks.workunit.client.1.smithi148.stdout:8/263: rmdir dc/d13 39 2024-09-21T04:15:13.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/285: getdents d4/d12/d2c/d2d/d44 0 2024-09-21T04:15:13.163 INFO:tasks.workunit.client.1.smithi148.stdout:0/296: dwrite d5/d6/d7/f4b [0,4194304] 0 2024-09-21T04:15:13.165 INFO:tasks.workunit.client.1.smithi148.stdout:5/278: dread d3/d24/f43 [0,4194304] 0 2024-09-21T04:15:13.167 INFO:tasks.workunit.client.0.smithi110.stdout:4/458: write d17/d34/d3d/d20/d43/d51/d52/f59 [2504246,57836] 0 2024-09-21T04:15:13.168 INFO:tasks.workunit.client.1.smithi148.stdout:3/255: creat d3/d5/d6/d12/da/d21/f4e x:0 0 0 2024-09-21T04:15:13.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/256: rename d3 to d3/d5/d4f 22 2024-09-21T04:15:13.171 INFO:tasks.workunit.client.0.smithi110.stdout:4/459: truncate d17/d34/d3d/d20/f80 1018534 0 2024-09-21T04:15:13.172 INFO:tasks.workunit.client.1.smithi148.stdout:8/264: creat dc/d2a/f57 x:0 0 0 2024-09-21T04:15:13.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/286: rmdir d4/d3a 39 2024-09-21T04:15:13.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/287: dread - d4/d12/d2c/d2d/f52 zero size 2024-09-21T04:15:13.174 INFO:tasks.workunit.client.1.smithi148.stdout:5/279: dread d3/f1b [0,4194304] 0 2024-09-21T04:15:13.174 INFO:tasks.workunit.client.1.smithi148.stdout:5/280: fsync d3/d14/d1a/f4d 0 2024-09-21T04:15:13.175 INFO:tasks.workunit.client.0.smithi110.stdout:4/460: creat d17/d34/d3d/fa3 x:0 0 0 2024-09-21T04:15:13.175 INFO:tasks.workunit.client.0.smithi110.stdout:4/461: read d17/f32 [490495,37611] 0 2024-09-21T04:15:13.179 INFO:tasks.workunit.client.1.smithi148.stdout:7/301: rmdir d5/dc 39 2024-09-21T04:15:13.179 INFO:tasks.workunit.client.1.smithi148.stdout:7/302: readlink d5/d8/l11 0 2024-09-21T04:15:13.180 INFO:tasks.workunit.client.1.smithi148.stdout:3/257: rename d3/d5/d6/l39 to d3/d5/d6/d12/d43/l50 0 2024-09-21T04:15:13.180 INFO:tasks.workunit.client.1.smithi148.stdout:3/258: dread - d3/f49 zero size 2024-09-21T04:15:13.183 INFO:tasks.workunit.client.0.smithi110.stdout:4/462: dread d17/d34/d3d/d20/d43/f8c [0,4194304] 0 2024-09-21T04:15:13.183 INFO:tasks.workunit.client.0.smithi110.stdout:4/463: write d17/d34/d3d/d20/d43/d51/d7a/d86/f87 [932815,65978] 0 2024-09-21T04:15:13.183 INFO:tasks.workunit.client.1.smithi148.stdout:4/288: truncate d4/d23/d2a/f36 562352 0 2024-09-21T04:15:13.184 INFO:tasks.workunit.client.1.smithi148.stdout:5/281: truncate d3/d34/d3c/f48 471652 0 2024-09-21T04:15:13.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/337: sync 2024-09-21T04:15:13.186 INFO:tasks.workunit.client.1.smithi148.stdout:7/303: symlink d5/dc/d28/d4a/l5f 0 2024-09-21T04:15:13.186 INFO:tasks.workunit.client.1.smithi148.stdout:7/304: chown d5/dc/d28/d4a 2 1 2024-09-21T04:15:13.192 INFO:tasks.workunit.client.1.smithi148.stdout:3/259: rename l0 to d3/l51 0 2024-09-21T04:15:13.192 INFO:tasks.workunit.client.1.smithi148.stdout:8/265: rmdir dc/d13/d39 39 2024-09-21T04:15:13.192 INFO:tasks.workunit.client.1.smithi148.stdout:3/260: fsync d3/d5/d6/d12/da/d23/f26 0 2024-09-21T04:15:13.193 INFO:tasks.workunit.client.1.smithi148.stdout:4/289: stat d4/d12/l1a 0 2024-09-21T04:15:13.196 INFO:tasks.workunit.client.1.smithi148.stdout:2/338: creat d0/f68 x:0 0 0 2024-09-21T04:15:13.197 INFO:tasks.workunit.client.1.smithi148.stdout:7/305: symlink d5/l60 0 2024-09-21T04:15:13.198 INFO:tasks.workunit.client.1.smithi148.stdout:8/266: symlink dc/d3e/l58 0 2024-09-21T04:15:13.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/290: creat d4/d12/d16/d26/f55 x:0 0 0 2024-09-21T04:15:13.203 INFO:tasks.workunit.client.1.smithi148.stdout:5/282: dwrite d3/d14/d41/f4b [0,4194304] 0 2024-09-21T04:15:13.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/464: dwrite d17/d34/d3d/d20/d43/d51/d7a/f7f [0,4194304] 0 2024-09-21T04:15:13.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/465: fsync d17/f27 0 2024-09-21T04:15:13.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/466: truncate d17/d34/d3d/d20/f80 1611038 0 2024-09-21T04:15:13.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/467: chown d17/d3e/f8b 0 1 2024-09-21T04:15:13.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/306: symlink d5/dc/d28/d4a/l61 0 2024-09-21T04:15:13.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/267: link dc/l27 dc/d43/d4c/d4f/l59 0 2024-09-21T04:15:13.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/268: fdatasync dc/d2a/d42/f53 0 2024-09-21T04:15:13.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/269: creat dc/d43/f5a x:0 0 0 2024-09-21T04:15:13.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/270: stat dc/l30 0 2024-09-21T04:15:13.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/339: getdents d0/d9/dc/d13/d2c 0 2024-09-21T04:15:13.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/340: creat d0/d9/dc/d13/f69 x:0 0 0 2024-09-21T04:15:13.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/341: creat d0/d9/d16/f6a x:0 0 0 2024-09-21T04:15:13.217 INFO:tasks.workunit.client.0.smithi110.stdout:4/468: dread d17/d1e/f48 [0,4194304] 0 2024-09-21T04:15:13.217 INFO:tasks.workunit.client.0.smithi110.stdout:4/469: truncate d17/d34/d3d/d20/d43/f92 6345831 0 2024-09-21T04:15:13.221 INFO:tasks.workunit.client.1.smithi148.stdout:7/307: rename d5/dc/d28/f3a to d5/d8/f62 0 2024-09-21T04:15:13.225 INFO:tasks.workunit.client.1.smithi148.stdout:3/261: rename d3/d5/d6/d12/da/d19 to d3/d5/d6/d12/da/d52 0 2024-09-21T04:15:13.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/291: dwrite d4/d23/f37 [0,4194304] 0 2024-09-21T04:15:13.232 INFO:tasks.workunit.client.1.smithi148.stdout:4/292: mkdir d4/d23/d56 0 2024-09-21T04:15:13.232 INFO:tasks.workunit.client.0.smithi110.stdout:4/470: dread d17/f39 [0,4194304] 0 2024-09-21T04:15:13.232 INFO:tasks.workunit.client.1.smithi148.stdout:8/271: rename dc/d43/f5a to dc/d2a/f5b 0 2024-09-21T04:15:13.234 INFO:tasks.workunit.client.1.smithi148.stdout:4/293: creat d4/d12/d2c/f57 x:0 0 0 2024-09-21T04:15:13.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/272: write f8 [1418492,74043] 0 2024-09-21T04:15:13.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/273: readlink dc/d13/l37 0 2024-09-21T04:15:13.236 INFO:tasks.workunit.client.0.smithi110.stdout:4/471: dread d17/d34/d3d/d20/f26 [0,4194304] 0 2024-09-21T04:15:13.239 INFO:tasks.workunit.client.1.smithi148.stdout:3/262: write d3/d5/f41 [3743550,8722] 0 2024-09-21T04:15:13.240 INFO:tasks.workunit.client.1.smithi148.stdout:5/283: dwrite d3/d14/d1a/f57 [0,4194304] 0 2024-09-21T04:15:13.242 INFO:tasks.workunit.client.0.smithi110.stdout:2/178: sync 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:2/179: write d2/d1a/d1f/d2a/d18/f2b [681564,35349] 0 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:2/180: rename d2/d1a/d1f/d29/d19/d2d to d2/d1a/d1f/d29/d19/d2d/d2f 22 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:2/181: readlink d2/d1a/d1f/d2a/l11 0 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:8/361: sync 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/554: sync 2024-09-21T04:15:13.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/555: chown d2/d9/d26/f4d 269 1 2024-09-21T04:15:13.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/263: truncate d3/d1f/f47 1370110 0 2024-09-21T04:15:13.249 INFO:tasks.workunit.client.1.smithi148.stdout:2/342: getdents d0/d9/d16 0 2024-09-21T04:15:13.250 INFO:tasks.workunit.client.0.smithi110.stdout:2/182: symlink d2/d1a/d1f/d29/l30 0 2024-09-21T04:15:13.250 INFO:tasks.workunit.client.1.smithi148.stdout:8/274: dread f6 [0,4194304] 0 2024-09-21T04:15:13.254 INFO:tasks.workunit.client.1.smithi148.stdout:2/343: dread d0/d9/f1b [0,4194304] 0 2024-09-21T04:15:13.254 INFO:tasks.workunit.client.1.smithi148.stdout:2/344: chown d0/d9/dc/d13/d2c 1906348611 1 2024-09-21T04:15:13.255 INFO:tasks.workunit.client.0.smithi110.stdout:2/183: mkdir d2/d1a/d1f/d2a/d18/d31 0 2024-09-21T04:15:13.256 INFO:tasks.workunit.client.1.smithi148.stdout:3/264: mknod d3/d5/d6/c53 0 2024-09-21T04:15:13.258 INFO:tasks.workunit.client.1.smithi148.stdout:8/275: truncate f7 756266 0 2024-09-21T04:15:13.259 INFO:tasks.workunit.client.0.smithi110.stdout:8/362: write d0/d30/f31 [1403750,69442] 0 2024-09-21T04:15:13.259 INFO:tasks.workunit.client.0.smithi110.stdout:8/363: write d0/d7/f36 [3779097,92803] 0 2024-09-21T04:15:13.260 INFO:tasks.workunit.client.1.smithi148.stdout:2/345: creat d0/d9/dc/d13/d25/d4e/f6b x:0 0 0 2024-09-21T04:15:13.260 INFO:tasks.workunit.client.1.smithi148.stdout:2/346: stat d0/d9/l62 0 2024-09-21T04:15:13.261 INFO:tasks.workunit.client.1.smithi148.stdout:8/276: truncate dc/f18 2879148 0 2024-09-21T04:15:13.261 INFO:tasks.workunit.client.1.smithi148.stdout:8/277: read dc/d13/f24 [64739,88617] 0 2024-09-21T04:15:13.261 INFO:tasks.workunit.client.1.smithi148.stdout:8/278: creat dc/d2a/d42/f5c x:0 0 0 2024-09-21T04:15:13.265 INFO:tasks.workunit.client.0.smithi110.stdout:8/364: mknod d0/d7/c7a 0 2024-09-21T04:15:13.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/365: symlink d0/d7/d13/l7b 0 2024-09-21T04:15:13.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/366: creat d0/d30/d34/d3e/d32/f7c x:0 0 0 2024-09-21T04:15:13.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/367: chown d0/f3d 0 1 2024-09-21T04:15:13.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/279: mkdir dc/d2a/d5d 0 2024-09-21T04:15:13.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/280: read - dc/d2a/d42/f5c zero size 2024-09-21T04:15:13.268 INFO:tasks.workunit.client.1.smithi148.stdout:2/347: mknod d0/d2f/c6c 0 2024-09-21T04:15:13.268 INFO:tasks.workunit.client.1.smithi148.stdout:2/348: read - d0/d9/dc/d13/f69 zero size 2024-09-21T04:15:13.269 INFO:tasks.workunit.client.0.smithi110.stdout:8/368: creat d0/d7/d13/d4b/f7d x:0 0 0 2024-09-21T04:15:13.269 INFO:tasks.workunit.client.0.smithi110.stdout:8/369: fdatasync d0/d30/d34/d3e/d32/f73 0 2024-09-21T04:15:13.269 INFO:tasks.workunit.client.1.smithi148.stdout:3/265: write f1 [3239787,85232] 0 2024-09-21T04:15:13.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/281: symlink dc/l5e 0 2024-09-21T04:15:13.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/282: stat dc/l12 0 2024-09-21T04:15:13.271 INFO:tasks.workunit.client.1.smithi148.stdout:2/349: creat d0/d9/d38/f6d x:0 0 0 2024-09-21T04:15:13.275 INFO:tasks.workunit.client.1.smithi148.stdout:8/283: truncate dc/d2a/f40 4008114 0 2024-09-21T04:15:13.275 INFO:tasks.workunit.client.1.smithi148.stdout:8/284: creat dc/d13/d3c/f5f x:0 0 0 2024-09-21T04:15:13.278 INFO:tasks.workunit.client.1.smithi148.stdout:7/308: dwrite d5/d14/f3c [0,4194304] 0 2024-09-21T04:15:13.282 INFO:tasks.workunit.client.0.smithi110.stdout:8/370: rename d0/d7/f1a to d0/d7/d13/d4b/f7e 0 2024-09-21T04:15:13.282 INFO:tasks.workunit.client.0.smithi110.stdout:8/371: fsync d0/d30/d54/f66 0 2024-09-21T04:15:13.282 INFO:tasks.workunit.client.1.smithi148.stdout:2/350: creat d0/d9/f6e x:0 0 0 2024-09-21T04:15:13.282 INFO:tasks.workunit.client.1.smithi148.stdout:2/351: dread d0/d2b/f61 [0,4194304] 0 2024-09-21T04:15:13.284 INFO:tasks.workunit.client.0.smithi110.stdout:4/472: dwrite d17/d3e/f40 [0,4194304] 0 2024-09-21T04:15:13.286 INFO:tasks.workunit.client.0.smithi110.stdout:1/556: dwrite d2/d9/d49/d4f/f67 [0,4194304] 0 2024-09-21T04:15:13.286 INFO:tasks.workunit.client.1.smithi148.stdout:3/266: write d3/d5/d6/d12/da/f11 [2623149,16282] 0 2024-09-21T04:15:13.287 INFO:tasks.workunit.client.0.smithi110.stdout:8/372: dread d0/d30/d54/f59 [0,4194304] 0 2024-09-21T04:15:13.287 INFO:tasks.workunit.client.1.smithi148.stdout:2/352: rename d0/d9/dc/d13/d2c to d0/d9/d16/d55/d6f 0 2024-09-21T04:15:13.287 INFO:tasks.workunit.client.1.smithi148.stdout:4/294: dwrite d4/f11 [0,4194304] 0 2024-09-21T04:15:13.289 INFO:tasks.workunit.client.0.smithi110.stdout:2/184: dwrite d2/d1a/d1f/d2a/d18/f1d [0,4194304] 0 2024-09-21T04:15:13.289 INFO:tasks.workunit.client.0.smithi110.stdout:4/473: symlink d17/d1e/d77/la4 0 2024-09-21T04:15:13.290 INFO:tasks.workunit.client.1.smithi148.stdout:5/284: dwrite d3/d4a/f52 [0,4194304] 0 2024-09-21T04:15:13.290 INFO:tasks.workunit.client.1.smithi148.stdout:5/285: creat d3/d14/d1f/d26/d2e/d3d/f67 x:0 0 0 2024-09-21T04:15:13.290 INFO:tasks.workunit.client.1.smithi148.stdout:5/286: write d3/f58 [34819,99986] 0 2024-09-21T04:15:13.293 INFO:tasks.workunit.client.1.smithi148.stdout:3/267: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:13.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/353: truncate d0/f4 3217710 0 2024-09-21T04:15:13.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/354: dread - d0/d9/dc/d13/d25/f45 zero size 2024-09-21T04:15:13.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/373: unlink d0/fb 0 2024-09-21T04:15:13.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/374: write d0/d7/d23/f33 [531240,61747] 0 2024-09-21T04:15:13.303 INFO:tasks.workunit.client.1.smithi148.stdout:4/295: dread d4/d12/d16/d26/f3e [0,4194304] 0 2024-09-21T04:15:13.303 INFO:tasks.workunit.client.1.smithi148.stdout:4/296: rename d4/d12 to d4/d12/d16/d29/d58 22 2024-09-21T04:15:13.303 INFO:tasks.workunit.client.1.smithi148.stdout:4/297: fdatasync d4/d23/f37 0 2024-09-21T04:15:13.303 INFO:tasks.workunit.client.0.smithi110.stdout:2/185: mknod d2/d1a/c32 0 2024-09-21T04:15:13.304 INFO:tasks.workunit.client.1.smithi148.stdout:1/258: sync 2024-09-21T04:15:13.306 INFO:tasks.workunit.client.1.smithi148.stdout:5/287: dread d3/fa [0,4194304] 0 2024-09-21T04:15:13.306 INFO:tasks.workunit.client.1.smithi148.stdout:5/288: dread - d3/d14/d1f/d26/d2e/d3d/f61 zero size 2024-09-21T04:15:13.306 INFO:tasks.workunit.client.1.smithi148.stdout:5/289: chown d3/d14/d1f/d26/l38 29457 1 2024-09-21T04:15:13.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/309: dwrite d5/dc/d24/f2b [0,4194304] 0 2024-09-21T04:15:13.307 INFO:tasks.workunit.client.0.smithi110.stdout:4/474: mknod d17/d34/d3d/d20/d43/ca5 0 2024-09-21T04:15:13.307 INFO:tasks.workunit.client.0.smithi110.stdout:4/475: fsync d17/d34/f37 0 2024-09-21T04:15:13.307 INFO:tasks.workunit.client.0.smithi110.stdout:4/476: creat d17/d34/d3d/fa6 x:0 0 0 2024-09-21T04:15:13.308 INFO:tasks.workunit.client.1.smithi148.stdout:5/290: read d3/d14/f44 [474421,69823] 0 2024-09-21T04:15:13.308 INFO:tasks.workunit.client.1.smithi148.stdout:5/291: chown d3/d14/d1f/d26/d2e/d3d 54 1 2024-09-21T04:15:13.310 INFO:tasks.workunit.client.0.smithi110.stdout:1/557: rename d2/d9/d23/d3b/d5e to d2/db7 0 2024-09-21T04:15:13.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/268: mkdir d3/d5/d6/d12/d43/d54 0 2024-09-21T04:15:13.313 INFO:tasks.workunit.client.0.smithi110.stdout:8/375: mknod d0/d30/d34/d3e/c7f 0 2024-09-21T04:15:13.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/355: creat d0/d9/dc/d13/d50/f70 x:0 0 0 2024-09-21T04:15:13.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/356: dread - d0/d9/d16/f5b zero size 2024-09-21T04:15:13.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/357: write d0/f11 [3156416,76980] 0 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/477: link d17/d1e/d7b/f5e d17/d34/d3d/d20/d43/d51/d7a/fa7 0 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/478: chown d17/d34/d3d/d20/f88 57 1 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/479: read f14 [1923390,73754] 0 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/480: dread - d17/d1e/d21/f4d zero size 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/481: chown d17/d34/d3d/d20/d43/d51/c53 2 1 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/482: creat d17/d34/d3d/d20/fa8 x:0 0 0 2024-09-21T04:15:13.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/483: write f14 [1295868,74329] 0 2024-09-21T04:15:13.321 INFO:tasks.workunit.client.0.smithi110.stdout:4/484: dread - d17/d34/d3d/f91 zero size 2024-09-21T04:15:13.321 INFO:tasks.workunit.client.1.smithi148.stdout:5/292: mknod d3/d14/c68 0 2024-09-21T04:15:13.324 INFO:tasks.workunit.client.0.smithi110.stdout:1/558: link d2/d9/d23/d4e/d57/c73 d2/d9/d23/d3b/cb8 0 2024-09-21T04:15:13.325 INFO:tasks.workunit.client.0.smithi110.stdout:1/559: chown d2/d9/d23/d4e/d57 31149584 1 2024-09-21T04:15:13.325 INFO:tasks.workunit.client.0.smithi110.stdout:1/560: chown d2/db7/f5f 9 1 2024-09-21T04:15:13.326 INFO:tasks.workunit.client.1.smithi148.stdout:2/358: dread d0/d9/f17 [0,4194304] 0 2024-09-21T04:15:13.326 INFO:tasks.workunit.client.1.smithi148.stdout:2/359: stat d0/d9/dc/d13/f21 0 2024-09-21T04:15:13.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/186: dwrite d2/d1a/d1f/d2a/f21 [0,4194304] 0 2024-09-21T04:15:13.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/187: fsync d2/d1a/d1f/d2a/f21 0 2024-09-21T04:15:13.327 INFO:tasks.workunit.client.0.smithi110.stdout:4/485: dread d17/d3e/d6b/f7d [0,4194304] 0 2024-09-21T04:15:13.328 INFO:tasks.workunit.client.1.smithi148.stdout:2/360: write d0/d9/f1b [232128,62435] 0 2024-09-21T04:15:13.328 INFO:tasks.workunit.client.1.smithi148.stdout:2/361: write d0/d2b/f32 [1215347,39331] 0 2024-09-21T04:15:13.328 INFO:tasks.workunit.client.1.smithi148.stdout:2/362: chown d0/l15 748532432 1 2024-09-21T04:15:13.328 INFO:tasks.workunit.client.0.smithi110.stdout:8/376: symlink d0/d7/d13/d46/l80 0 2024-09-21T04:15:13.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/259: rmdir db/d24/d2e 39 2024-09-21T04:15:13.331 INFO:tasks.workunit.client.1.smithi148.stdout:3/269: dread d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:13.333 INFO:tasks.workunit.client.1.smithi148.stdout:7/310: dwrite d5/dc/f3e [0,4194304] 0 2024-09-21T04:15:13.335 INFO:tasks.workunit.client.0.smithi110.stdout:2/188: dread d2/d1a/f20 [0,4194304] 0 2024-09-21T04:15:13.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/293: link d3/d14/l3a d3/d14/d1f/d26/d2e/l69 0 2024-09-21T04:15:13.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/294: fdatasync d3/d14/f44 0 2024-09-21T04:15:13.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/561: creat d2/db7/d86/fb9 x:0 0 0 2024-09-21T04:15:13.340 INFO:tasks.workunit.client.1.smithi148.stdout:7/311: dread d5/f6 [0,4194304] 0 2024-09-21T04:15:13.347 INFO:tasks.workunit.client.1.smithi148.stdout:2/363: mkdir d0/d9/d16/d55/d71 0 2024-09-21T04:15:13.353 INFO:tasks.workunit.client.1.smithi148.stdout:5/295: symlink d3/d14/d1a/l6a 0 2024-09-21T04:15:13.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/312: mkdir d5/dc/d28/d63 0 2024-09-21T04:15:13.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/313: write d5/d8/f9 [1355419,14896] 0 2024-09-21T04:15:13.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/364: dread d0/ff [0,4194304] 0 2024-09-21T04:15:13.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/365: truncate d0/d9/dc/f46 1065433 0 2024-09-21T04:15:13.360 INFO:tasks.workunit.client.1.smithi148.stdout:7/314: mkdir d5/d8/d64 0 2024-09-21T04:15:13.362 INFO:tasks.workunit.client.1.smithi148.stdout:2/366: symlink d0/d9/d16/d55/d6f/l72 0 2024-09-21T04:15:13.363 INFO:tasks.workunit.client.0.smithi110.stdout:4/486: mknod d17/d1e/d77/ca9 0 2024-09-21T04:15:13.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/315: rmdir d5/dc/d28/d4a 39 2024-09-21T04:15:13.369 INFO:tasks.workunit.client.1.smithi148.stdout:3/270: dwrite d3/d5/f35 [0,4194304] 0 2024-09-21T04:15:13.369 INFO:tasks.workunit.client.1.smithi148.stdout:3/271: chown d3/d5/d6/d12/da/l1b 252224 1 2024-09-21T04:15:13.369 INFO:tasks.workunit.client.1.smithi148.stdout:3/272: creat d3/f55 x:0 0 0 2024-09-21T04:15:13.370 INFO:tasks.workunit.client.0.smithi110.stdout:2/189: mknod d2/d1a/d1f/d2a/d18/d31/c33 0 2024-09-21T04:15:13.371 INFO:tasks.workunit.client.0.smithi110.stdout:1/562: mknod d2/d9/d23/d4e/d57/cba 0 2024-09-21T04:15:13.372 INFO:tasks.workunit.client.0.smithi110.stdout:1/563: write d2/f45 [2942109,8977] 0 2024-09-21T04:15:13.373 INFO:tasks.workunit.client.0.smithi110.stdout:4/487: write d17/d34/d3d/d57/f5a [444571,776] 0 2024-09-21T04:15:13.374 INFO:tasks.workunit.client.0.smithi110.stdout:4/488: write d17/d1e/d64/f9d [585812,96229] 0 2024-09-21T04:15:13.374 INFO:tasks.workunit.client.0.smithi110.stdout:2/190: mknod d2/d1a/c34 0 2024-09-21T04:15:13.376 INFO:tasks.workunit.client.1.smithi148.stdout:1/260: dwrite db/ff [0,4194304] 0 2024-09-21T04:15:13.376 INFO:tasks.workunit.client.0.smithi110.stdout:0/598: sync 2024-09-21T04:15:13.376 INFO:tasks.workunit.client.0.smithi110.stdout:0/599: creat d4/d47/fb2 x:0 0 0 2024-09-21T04:15:13.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/377: dwrite d0/d7/f1b [0,4194304] 0 2024-09-21T04:15:13.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/378: truncate d0/d7/f2a 639968 0 2024-09-21T04:15:13.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/379: creat d0/d30/d54/d6b/f81 x:0 0 0 2024-09-21T04:15:13.378 INFO:tasks.workunit.client.1.smithi148.stdout:5/296: dwrite f0 [0,4194304] 0 2024-09-21T04:15:13.379 INFO:tasks.workunit.client.0.smithi110.stdout:1/564: mkdir d2/d9/d23/d3b/d93/dbb 0 2024-09-21T04:15:13.380 INFO:tasks.workunit.client.1.smithi148.stdout:2/367: dwrite d0/d9/dc/d13/d4c/f66 [0,4194304] 0 2024-09-21T04:15:13.380 INFO:tasks.workunit.client.1.smithi148.stdout:2/368: dread - d0/d9/d59/f65 zero size 2024-09-21T04:15:13.380 INFO:tasks.workunit.client.1.smithi148.stdout:2/369: chown d0/d9/dc/d13/f21 37071 1 2024-09-21T04:15:13.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/370: stat d0/d9/l62 0 2024-09-21T04:15:13.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/371: stat d0/ff 0 2024-09-21T04:15:13.381 INFO:tasks.workunit.client.1.smithi148.stdout:2/372: fsync d0/d9/dc/f24 0 2024-09-21T04:15:13.385 INFO:tasks.workunit.client.1.smithi148.stdout:7/316: getdents d5 0 2024-09-21T04:15:13.385 INFO:tasks.workunit.client.0.smithi110.stdout:4/489: mknod d17/d3e/d6b/caa 0 2024-09-21T04:15:13.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/191: getdents d2/d1a/d1f/d2a 0 2024-09-21T04:15:13.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/192: chown d2/d1a/d1f/d29/c13 2 1 2024-09-21T04:15:13.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/193: stat d2/d1a/d1f/d2a/d18/f2b 0 2024-09-21T04:15:13.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/194: creat d2/f35 x:0 0 0 2024-09-21T04:15:13.395 INFO:tasks.workunit.client.0.smithi110.stdout:0/600: mknod d4/dd/d10/cb3 0 2024-09-21T04:15:13.511 INFO:tasks.workunit.client.0.smithi110.stdout:0/601: read d4/dd/fe [2759165,71056] 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/490: symlink d17/d34/d3d/d20/d43/d51/d7a/lab 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/491: mknod d17/d3e/cac 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/602: unlink d4/c14 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/603: readlink d4/dd/d10/d29/l95 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/604: chown d4/dd/d10/d29/d75/fa9 4 1 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/492: symlink d17/d1e/d21/lad 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/493: creat d17/d34/d3d/d20/d43/d51/d7a/d86/fae x:0 0 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/605: link d4/d47/d2c/f88 d4/d8b/fb4 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:8/380: dwrite d0/d7/f36 [0,4194304] 0 2024-09-21T04:15:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/494: mkdir d17/d1e/d7b/daf 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/565: dwrite d2/d9/d23/f38 [0,4194304] 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:0/606: truncate d4/f85 901217 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:0/607: chown d4/dd/d10/l26 19 1 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/566: link d2/d9/d26/f31 d2/d9/d26/d3d/d8f/fbc 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/567: write d2/d9/d23/d3b/f55 [1596964,33580] 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:8/381: dwrite d0/d7/d13/d46/f51 [0,4194304] 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:4/495: getdents d17/d34/d3d/d20 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/568: rename d2/c37 to d2/d9/d23/d3b/cbd 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:8/382: mkdir d0/d7/d23/d82 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:0/608: dwrite d4/d1c/d62/f39 [0,4194304] 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:8/383: creat d0/d7/d23/d82/f83 x:0 0 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/569: rename d2/d9/d26/f34 to d2/d9/d49/fbe 0 2024-09-21T04:15:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:8/384: write d0/d7/d13/d4b/f17 [2868670,10559] 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:8/385: symlink d0/d7/d23/l84 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:4/496: dwrite d17/d34/d3d/d20/f90 [0,4194304] 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:8/386: mkdir d0/d7/d23/d82/d85 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/609: dwrite d4/d1c/d62/d69/d9b/fad [0,4194304] 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:4/497: getdents d17/d34/d3d 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/610: mkdir d4/dd/d10/d8a/db5 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:1/570: dwrite d2/d9/d23/d4e/f7f [4194304,4194304] 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/611: link d4/dd/f8e d4/dd/d10/d8a/d92/fb6 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/612: write d4/dd/d10/d29/f5d [1446157,114668] 0 2024-09-21T04:15:13.520 INFO:tasks.workunit.client.0.smithi110.stdout:1/571: truncate d2/d9/d26/f2d 486005 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:8/387: dwrite d0/f11 [4194304,4194304] 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:8/388: mkdir d0/d7/d6f/d86 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:8/389: write d0/d7/d23/d82/f83 [566597,47578] 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:4/498: dwrite d17/d34/d3d/fa3 [0,4194304] 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:4/499: creat d17/d34/fb0 x:0 0 0 2024-09-21T04:15:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:8/390: symlink d0/d7/d13/l87 0 2024-09-21T04:15:13.522 INFO:tasks.workunit.client.1.smithi148.stdout:1/261: link db/d24/c39 db/d24/c56 0 2024-09-21T04:15:13.522 INFO:tasks.workunit.client.1.smithi148.stdout:1/262: chown db/d24/c40 9510 1 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:2/373: symlink d0/d2b/d48/l73 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:6/219: sync 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:9/255: sync 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:2/374: dread d0/d9/f17 [0,4194304] 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:5/297: symlink d3/l6b 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:9/256: creat d5/d1e/f65 x:0 0 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:7/317: rename d5/dc/l12 to d5/d14/l65 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:1/263: dwrite db/d24/d2e/f3c [0,4194304] 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:1/264: creat db/d46/f57 x:0 0 0 2024-09-21T04:15:13.523 INFO:tasks.workunit.client.1.smithi148.stdout:6/220: write d0/d1a/d23/d16/f1f [5307179,47748] 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/221: creat d0/d1a/d23/f3d x:0 0 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:5/298: link d3/l30 d3/d4a/l6c 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/257: symlink d5/d1e/d37/d50/l66 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/258: readlink d5/d60/d28/d2f/l5f 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/318: rename d5/l1c to d5/d8/d64/l66 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:2/375: getdents d0 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:1/265: dwrite db/d46/f57 [0,4194304] 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/222: dread d0/f21 [0,4194304] 0 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/223: chown d0/d1a/d23/f3d 72 1 2024-09-21T04:15:13.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/319: rmdir d5/dc/d28/d63 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:7/320: chown d5/d8/f9 1385 1 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:7/321: truncate d5/d14/d25/f42 1459555 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:5/299: symlink d3/l6d 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:9/259: truncate d5/ff 2498145 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/376: dwrite d0/d9/d59/f65 [0,4194304] 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/377: creat d0/d9/dc/d13/f74 x:0 0 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/378: write d0/d9/dc/d13/d25/f45 [50249,115354] 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/266: write db/d46/f57 [1494533,121504] 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/224: creat d0/d1a/d23/dc/f3e x:0 0 0 2024-09-21T04:15:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/225: creat d0/d1a/d23/d16/f3f x:0 0 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:6/226: chown d0/d1a/d23/d16/f1f 395 1 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:6/227: fdatasync d0/f21 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:2/379: link d0/la d0/d9/dc/d13/d25/l75 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:7/322: dwrite d5/dc/f3e [0,4194304] 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:1/267: unlink db/d24/d2e/f3c 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:1/268: chown db/d46/l50 0 1 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:2/380: dread d0/d9/d38/f4f [0,4194304] 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:1/269: creat db/f58 x:0 0 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:2/381: rename d0/d9/dc/d13/d4c/f66 to d0/d2b/f76 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:1/270: stat db/d24/f25 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:1/271: mkdir db/d24/d2e/d59 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/300: dwrite d3/d14/d1f/d26/d2e/d56/f3f [0,4194304] 0 2024-09-21T04:15:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/301: fdatasync d3/d34/f47 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/382: mknod d0/d9/d16/d55/c77 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/383: stat d0/d2b/l51 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:1/272: mknod db/d24/d2e/d59/c5a 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:6/228: dwrite d0/d8/f24 [0,4194304] 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:6/229: write d0/d8/d18/f25 [736994,45845] 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:6/230: write d0/d34/d38/f3a [182767,117419] 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:1/273: dread db/d24/f2b [0,4194304] 0 2024-09-21T04:15:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/323: dwrite d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:13.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/500: rename d17/d34/d3d/d20/f90 to d17/d34/d3d/d20/d43/d51/d52/fb1 0 2024-09-21T04:15:13.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/501: creat d17/d34/fb2 x:0 0 0 2024-09-21T04:15:13.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/572: dwrite d2/d9/d26/f4d [0,4194304] 0 2024-09-21T04:15:13.528 INFO:tasks.workunit.client.0.smithi110.stdout:8/391: rmdir d0/d7/d23/d52 39 2024-09-21T04:15:13.531 INFO:tasks.workunit.client.1.smithi148.stdout:2/384: unlink d0/d9/d16/d55/d6f/l3c 0 2024-09-21T04:15:13.536 INFO:tasks.workunit.client.0.smithi110.stdout:1/573: mknod d2/db7/cbf 0 2024-09-21T04:15:13.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/574: chown d2/d9/d26/c42 35419047 1 2024-09-21T04:15:13.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/575: fdatasync d2/d9/d26/f2d 0 2024-09-21T04:15:13.538 INFO:tasks.workunit.client.1.smithi148.stdout:6/231: link d0/d1a/d23/d16/c1e d0/d8/c40 0 2024-09-21T04:15:13.538 INFO:tasks.workunit.client.1.smithi148.stdout:6/232: chown d0/d1a/d23/f19 866 1 2024-09-21T04:15:13.539 INFO:tasks.workunit.client.0.smithi110.stdout:8/392: creat d0/d30/d34/d3e/f88 x:0 0 0 2024-09-21T04:15:13.541 INFO:tasks.workunit.client.0.smithi110.stdout:1/576: mknod d2/d9/d49/d5c/cc0 0 2024-09-21T04:15:13.542 INFO:tasks.workunit.client.1.smithi148.stdout:8/285: sync 2024-09-21T04:15:13.543 INFO:tasks.workunit.client.0.smithi110.stdout:8/393: chown d0/d7/d13/d4b/c24 62297 1 2024-09-21T04:15:13.543 INFO:tasks.workunit.client.0.smithi110.stdout:4/502: dwrite d17/d1e/f79 [0,4194304] 0 2024-09-21T04:15:13.543 INFO:tasks.workunit.client.1.smithi148.stdout:0/297: sync 2024-09-21T04:15:13.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/274: dwrite db/d46/f48 [0,4194304] 0 2024-09-21T04:15:13.544 INFO:tasks.workunit.client.0.smithi110.stdout:1/577: write d2/d9/d26/f8b [2558476,45136] 0 2024-09-21T04:15:13.544 INFO:tasks.workunit.client.0.smithi110.stdout:1/578: write d2/d9/d49/d7a/f7c [4610554,66133] 0 2024-09-21T04:15:13.544 INFO:tasks.workunit.client.0.smithi110.stdout:1/579: creat d2/d9/d26/d3d/fc1 x:0 0 0 2024-09-21T04:15:13.545 INFO:tasks.workunit.client.0.smithi110.stdout:8/394: mkdir d0/d7/d89 0 2024-09-21T04:15:13.545 INFO:tasks.workunit.client.0.smithi110.stdout:8/395: readlink d0/d7/d13/d46/l80 0 2024-09-21T04:15:13.546 INFO:tasks.workunit.client.1.smithi148.stdout:7/324: rename d5/d8/l37 to d5/dc/d28/l67 0 2024-09-21T04:15:13.547 INFO:tasks.workunit.client.1.smithi148.stdout:5/302: truncate d3/d14/d1f/d26/d2e/d56/f3f 3490550 0 2024-09-21T04:15:13.547 INFO:tasks.workunit.client.1.smithi148.stdout:5/303: dread - d3/d34/f64 zero size 2024-09-21T04:15:13.551 INFO:tasks.workunit.client.1.smithi148.stdout:2/385: mknod d0/d2b/d48/c78 0 2024-09-21T04:15:13.554 INFO:tasks.workunit.client.1.smithi148.stdout:6/233: symlink d0/d1a/d23/d32/l41 0 2024-09-21T04:15:13.556 INFO:tasks.workunit.client.1.smithi148.stdout:8/286: unlink f6 0 2024-09-21T04:15:13.558 INFO:tasks.workunit.client.0.smithi110.stdout:1/580: dread d2/d9/d49/fbe [0,4194304] 0 2024-09-21T04:15:13.558 INFO:tasks.workunit.client.1.smithi148.stdout:1/275: dwrite db/d46/f57 [4194304,4194304] 0 2024-09-21T04:15:13.563 INFO:tasks.workunit.client.0.smithi110.stdout:4/503: read d17/d34/d3d/d20/d43/d51/d52/fb1 [1459078,114184] 0 2024-09-21T04:15:13.563 INFO:tasks.workunit.client.1.smithi148.stdout:4/298: sync 2024-09-21T04:15:13.564 INFO:tasks.workunit.client.1.smithi148.stdout:3/273: sync 2024-09-21T04:15:13.564 INFO:tasks.workunit.client.1.smithi148.stdout:4/299: fsync d4/d12/d16/f19 0 2024-09-21T04:15:13.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/325: creat d5/dc/d28/d4a/f68 x:0 0 0 2024-09-21T04:15:13.565 INFO:tasks.workunit.client.1.smithi148.stdout:7/326: fdatasync d5/d8/f16 0 2024-09-21T04:15:13.567 INFO:tasks.workunit.client.1.smithi148.stdout:7/327: dread d5/d14/d25/f42 [0,4194304] 0 2024-09-21T04:15:13.568 INFO:tasks.workunit.client.0.smithi110.stdout:8/396: stat d0/d7/d13/d4b/c71 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/298: rename d5/d6/d7/l40 to d5/d6/d2a/d2b/l58 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:5/304: symlink d3/d14/d1f/d26/d2e/l6e 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/299: creat d5/d6/d7/d22/f59 x:0 0 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/300: creat d5/d6/f5a x:0 0 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:5/305: write d3/f58 [995586,95506] 0 2024-09-21T04:15:13.569 INFO:tasks.workunit.client.1.smithi148.stdout:5/306: chown d3/f8 961 1 2024-09-21T04:15:13.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/234: symlink d0/d1a/d23/d32/l42 0 2024-09-21T04:15:13.571 INFO:tasks.workunit.client.0.smithi110.stdout:1/581: dread d2/d9/d23/d4e/f97 [0,4194304] 0 2024-09-21T04:15:13.572 INFO:tasks.workunit.client.0.smithi110.stdout:8/397: link d0/l2e d0/d30/d34/d3e/l8a 0 2024-09-21T04:15:13.572 INFO:tasks.workunit.client.0.smithi110.stdout:4/504: dread d17/f58 [0,4194304] 0 2024-09-21T04:15:13.573 INFO:tasks.workunit.client.1.smithi148.stdout:2/386: dwrite d0/d9/dc/d13/d25/f2a [4194304,4194304] 0 2024-09-21T04:15:13.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/287: rmdir dc/d13 39 2024-09-21T04:15:13.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/288: chown dc/l27 28 1 2024-09-21T04:15:13.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/289: fdatasync dc/d13/d3c/f49 0 2024-09-21T04:15:13.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/290: dread - dc/d13/d39/f51 zero size 2024-09-21T04:15:13.575 INFO:tasks.workunit.client.1.smithi148.stdout:8/291: write dc/f46 [725401,19712] 0 2024-09-21T04:15:13.578 INFO:tasks.workunit.client.0.smithi110.stdout:8/398: mkdir d0/d7/d23/d52/d8b 0 2024-09-21T04:15:13.578 INFO:tasks.workunit.client.0.smithi110.stdout:8/399: stat d0/d7/d23/f69 0 2024-09-21T04:15:13.580 INFO:tasks.workunit.client.1.smithi148.stdout:1/276: creat db/d46/d4c/f5b x:0 0 0 2024-09-21T04:15:13.580 INFO:tasks.workunit.client.1.smithi148.stdout:9/260: sync 2024-09-21T04:15:13.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/582: mknod d2/d9/d23/cc2 0 2024-09-21T04:15:13.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/583: chown d2/d9/d23/d3b/d93/fab 93 1 2024-09-21T04:15:13.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/584: fsync d2/d9/d26/d3d/f43 0 2024-09-21T04:15:13.582 INFO:tasks.workunit.client.1.smithi148.stdout:4/300: creat d4/d12/d16/d29/f59 x:0 0 0 2024-09-21T04:15:13.585 INFO:tasks.workunit.client.0.smithi110.stdout:8/400: unlink d0/d30/d34/d3e/d60/l77 0 2024-09-21T04:15:13.585 INFO:tasks.workunit.client.0.smithi110.stdout:8/401: creat d0/d30/d34/d3e/f8c x:0 0 0 2024-09-21T04:15:13.585 INFO:tasks.workunit.client.1.smithi148.stdout:3/274: symlink d3/d5/d6/d2b/l56 0 2024-09-21T04:15:13.586 INFO:tasks.workunit.client.0.smithi110.stdout:1/585: symlink d2/db7/lc3 0 2024-09-21T04:15:13.588 INFO:tasks.workunit.client.1.smithi148.stdout:7/328: fsync d5/dc/d28/d4a/f68 0 2024-09-21T04:15:13.588 INFO:tasks.workunit.client.0.smithi110.stdout:4/505: dwrite d17/d1e/d7b/f5e [0,4194304] 0 2024-09-21T04:15:13.589 INFO:tasks.workunit.client.1.smithi148.stdout:7/329: read d5/d14/d25/f52 [374008,28150] 0 2024-09-21T04:15:13.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/301: creat d5/d6/d2a/d2b/f5b x:0 0 0 2024-09-21T04:15:13.596 INFO:tasks.workunit.client.1.smithi148.stdout:5/307: mkdir d3/d14/d41/d6f 0 2024-09-21T04:15:13.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/235: mkdir d0/d34/d38/d43 0 2024-09-21T04:15:13.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/236: truncate d0/d1a/d23/d16/f2c 1087399 0 2024-09-21T04:15:13.599 INFO:tasks.workunit.client.0.smithi110.stdout:8/402: mknod d0/d7/d13/d4b/c8d 0 2024-09-21T04:15:13.601 INFO:tasks.workunit.client.0.smithi110.stdout:1/586: dwrite d2/d9/d26/d3d/f6f [0,4194304] 0 2024-09-21T04:15:13.602 INFO:tasks.workunit.client.1.smithi148.stdout:8/292: mknod dc/d43/c60 0 2024-09-21T04:15:13.602 INFO:tasks.workunit.client.1.smithi148.stdout:8/293: creat dc/d13/d3c/f61 x:0 0 0 2024-09-21T04:15:13.602 INFO:tasks.workunit.client.1.smithi148.stdout:8/294: write dc/d2a/f5b [336202,65449] 0 2024-09-21T04:15:13.602 INFO:tasks.workunit.client.1.smithi148.stdout:8/295: write dc/d2a/f57 [1036156,113704] 0 2024-09-21T04:15:13.603 INFO:tasks.workunit.client.0.smithi110.stdout:4/506: rmdir d17/d1e/d7b 39 2024-09-21T04:15:13.603 INFO:tasks.workunit.client.0.smithi110.stdout:4/507: read - d17/d1e/d21/f54 zero size 2024-09-21T04:15:13.604 INFO:tasks.workunit.client.0.smithi110.stdout:8/403: rename d0/d30/d34/d3e/c5f to d0/d30/d34/d3e/d60/c8e 0 2024-09-21T04:15:13.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/404: dread d0/d7/d23/d82/f83 [0,4194304] 0 2024-09-21T04:15:13.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/405: chown d0/d7/d13/l87 123 1 2024-09-21T04:15:13.609 INFO:tasks.workunit.client.1.smithi148.stdout:0/302: dwrite d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:13.610 INFO:tasks.workunit.client.0.smithi110.stdout:1/587: symlink d2/d88/lc4 0 2024-09-21T04:15:13.610 INFO:tasks.workunit.client.0.smithi110.stdout:1/588: readlink d2/d9/d49/lb6 0 2024-09-21T04:15:13.610 INFO:tasks.workunit.client.0.smithi110.stdout:1/589: creat d2/d9/d26/d3d/fc5 x:0 0 0 2024-09-21T04:15:13.612 INFO:tasks.workunit.client.1.smithi148.stdout:1/277: unlink db/f4d 0 2024-09-21T04:15:13.612 INFO:tasks.workunit.client.1.smithi148.stdout:1/278: fsync db/dd/f12 0 2024-09-21T04:15:13.612 INFO:tasks.workunit.client.1.smithi148.stdout:0/303: write d5/d6/f16 [2304367,52519] 0 2024-09-21T04:15:13.614 INFO:tasks.workunit.client.0.smithi110.stdout:4/508: link c9 d17/d34/d3d/d20/d43/d51/d7a/d8d/cb3 0 2024-09-21T04:15:13.614 INFO:tasks.workunit.client.0.smithi110.stdout:4/509: read - d17/d34/fb0 zero size 2024-09-21T04:15:13.614 INFO:tasks.workunit.client.0.smithi110.stdout:4/510: chown d17/d1e/f33 927817 1 2024-09-21T04:15:13.614 INFO:tasks.workunit.client.0.smithi110.stdout:4/511: chown f4 11 1 2024-09-21T04:15:13.617 INFO:tasks.workunit.client.1.smithi148.stdout:9/261: mkdir d5/d2b/d32/d67 0 2024-09-21T04:15:13.617 INFO:tasks.workunit.client.1.smithi148.stdout:9/262: readlink d5/d39/l61 0 2024-09-21T04:15:13.617 INFO:tasks.workunit.client.1.smithi148.stdout:9/263: stat d5/d60/d28/f41 0 2024-09-21T04:15:13.619 INFO:tasks.workunit.client.1.smithi148.stdout:4/301: link d4/d12/f41 d4/d12/d16/d29/f5a 0 2024-09-21T04:15:13.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/590: unlink d2/d9/d49/d5c/l7d 0 2024-09-21T04:15:13.626 INFO:tasks.workunit.client.1.smithi148.stdout:5/308: mkdir d3/d14/d41/d70 0 2024-09-21T04:15:13.626 INFO:tasks.workunit.client.1.smithi148.stdout:5/309: truncate d3/d14/d1f/d26/d2e/d3d/f67 336057 0 2024-09-21T04:15:13.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/237: rmdir d0/d8/d18 39 2024-09-21T04:15:13.628 INFO:tasks.workunit.client.0.smithi110.stdout:1/591: dread d2/d9/d23/f29 [0,4194304] 0 2024-09-21T04:15:13.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/279: link db/ff db/dd/d21/f5c 0 2024-09-21T04:15:13.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/280: write db/d24/d4f/f55 [814715,114894] 0 2024-09-21T04:15:13.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/281: chown db/f15 66186 1 2024-09-21T04:15:13.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/282: chown db/c3e 116 1 2024-09-21T04:15:13.632 INFO:tasks.workunit.client.0.smithi110.stdout:1/592: creat d2/d9/d49/d7a/da8/fc6 x:0 0 0 2024-09-21T04:15:13.632 INFO:tasks.workunit.client.0.smithi110.stdout:1/593: truncate d2/d9/d49/d5c/d91/fa5 807862 0 2024-09-21T04:15:13.633 INFO:tasks.workunit.client.1.smithi148.stdout:4/302: creat d4/d12/f5b x:0 0 0 2024-09-21T04:15:13.636 INFO:tasks.workunit.client.1.smithi148.stdout:7/330: symlink d5/dc/l69 0 2024-09-21T04:15:13.638 INFO:tasks.workunit.client.1.smithi148.stdout:5/310: dread d3/f1b [0,4194304] 0 2024-09-21T04:15:13.638 INFO:tasks.workunit.client.1.smithi148.stdout:0/304: dwrite d5/d6/d7/f31 [0,4194304] 0 2024-09-21T04:15:13.639 INFO:tasks.workunit.client.1.smithi148.stdout:6/238: truncate d0/d1a/d23/f14 646842 0 2024-09-21T04:15:13.639 INFO:tasks.workunit.client.1.smithi148.stdout:8/296: rmdir dc/d2a/d42 39 2024-09-21T04:15:13.641 INFO:tasks.workunit.client.0.smithi110.stdout:5/352: sync 2024-09-21T04:15:13.641 INFO:tasks.workunit.client.1.smithi148.stdout:5/311: dread d3/d14/f44 [0,4194304] 0 2024-09-21T04:15:13.641 INFO:tasks.workunit.client.1.smithi148.stdout:5/312: stat d3/d14/d1f/f23 0 2024-09-21T04:15:13.642 INFO:tasks.workunit.client.0.smithi110.stdout:5/353: read d1/d22/f4f [128936,89769] 0 2024-09-21T04:15:13.642 INFO:tasks.workunit.client.1.smithi148.stdout:1/283: unlink db/lc 0 2024-09-21T04:15:13.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/331: unlink d5/f6 0 2024-09-21T04:15:13.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/332: fsync d5/dc/d24/f2b 0 2024-09-21T04:15:13.645 INFO:tasks.workunit.client.0.smithi110.stdout:5/354: rmdir d1/d49/d4d 39 2024-09-21T04:15:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/239: mknod d0/d1a/d23/d16/d36/c44 0 2024-09-21T04:15:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/240: write d0/d1a/d23/ff [353143,15436] 0 2024-09-21T04:15:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/241: stat d0/d1a/d23/l31 0 2024-09-21T04:15:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/303: dread d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/304: readlink d4/l43 0 2024-09-21T04:15:13.650 INFO:tasks.workunit.client.1.smithi148.stdout:8/297: mkdir dc/d13/d28/d62 0 2024-09-21T04:15:13.651 INFO:tasks.workunit.client.1.smithi148.stdout:8/298: dread dc/d43/f44 [0,4194304] 0 2024-09-21T04:15:13.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/313: mknod d3/d34/d3c/c71 0 2024-09-21T04:15:13.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/333: creat d5/f6a x:0 0 0 2024-09-21T04:15:13.653 INFO:tasks.workunit.client.0.smithi110.stdout:4/512: dwrite d17/f1a [0,4194304] 0 2024-09-21T04:15:13.653 INFO:tasks.workunit.client.0.smithi110.stdout:4/513: write d17/d34/d3d/d20/d43/f92 [4926314,6118] 0 2024-09-21T04:15:13.654 INFO:tasks.workunit.client.0.smithi110.stdout:5/355: dread d1/d49/d4d/f48 [0,4194304] 0 2024-09-21T04:15:13.654 INFO:tasks.workunit.client.0.smithi110.stdout:5/356: creat d1/d7/d58/f7c x:0 0 0 2024-09-21T04:15:13.654 INFO:tasks.workunit.client.0.smithi110.stdout:5/357: write d1/d7/f4e [384212,105031] 0 2024-09-21T04:15:13.654 INFO:tasks.workunit.client.0.smithi110.stdout:5/358: rename d1 to d1/d7d 22 2024-09-21T04:15:13.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/305: rmdir d5/d6/d2a/d45 39 2024-09-21T04:15:13.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/306: truncate d5/d6/d3e/f1b 1224584 0 2024-09-21T04:15:13.657 INFO:tasks.workunit.client.1.smithi148.stdout:6/242: mkdir d0/d34/d45 0 2024-09-21T04:15:13.661 INFO:tasks.workunit.client.0.smithi110.stdout:8/406: dwrite d0/d7/d13/d4b/f67 [0,4194304] 0 2024-09-21T04:15:13.663 INFO:tasks.workunit.client.1.smithi148.stdout:4/305: rmdir d4/d23/d56 0 2024-09-21T04:15:13.663 INFO:tasks.workunit.client.1.smithi148.stdout:4/306: stat d4/f8 0 2024-09-21T04:15:13.663 INFO:tasks.workunit.client.1.smithi148.stdout:8/299: write fb [1298873,64880] 0 2024-09-21T04:15:13.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/300: dread - dc/d13/d3c/f61 zero size 2024-09-21T04:15:13.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/314: mkdir d3/d14/d1f/d72 0 2024-09-21T04:15:13.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/315: truncate d3/d37/f5f 106860 0 2024-09-21T04:15:13.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/334: mkdir d5/dc/d24/d6b 0 2024-09-21T04:15:13.666 INFO:tasks.workunit.client.0.smithi110.stdout:5/359: rename d1/d7/d18/d3f/c5e to d1/d49/d4d/d1e/d5a/d64/c7e 0 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: mgrmap e22: smithi148.zjrahi(active, since 1.03933s) 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: Checking for cert/key for grafana.smithi110 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: Migrating grafana.smithi110 cert to cert store 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: Migrating grafana.smithi110 key to cert store 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:13.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/trash_purge_schedule"}]: dispatch 2024-09-21T04:15:13.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:13 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/trash_purge_schedule"}]: dispatch 2024-09-21T04:15:13.668 INFO:tasks.workunit.client.1.smithi148.stdout:0/307: mknod d5/d6/d7/d11/c5c 0 2024-09-21T04:15:13.668 INFO:tasks.workunit.client.1.smithi148.stdout:6/243: write d0/d1a/d23/f14 [624199,43171] 0 2024-09-21T04:15:13.668 INFO:tasks.workunit.client.1.smithi148.stdout:6/244: fsync d0/d1a/d23/d16/f39 0 2024-09-21T04:15:13.668 INFO:tasks.workunit.client.1.smithi148.stdout:6/245: fsync d0/d1a/d23/d16/f1f 0 2024-09-21T04:15:13.669 INFO:tasks.workunit.client.1.smithi148.stdout:1/284: dwrite db/d24/f25 [0,4194304] 0 2024-09-21T04:15:13.669 INFO:tasks.workunit.client.1.smithi148.stdout:1/285: fsync db/ff 0 2024-09-21T04:15:13.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/316: creat d3/d14/d1f/d26/d2e/d56/f73 x:0 0 0 2024-09-21T04:15:13.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/317: write d3/d14/d1f/d26/d2e/d56/f19 [530565,52218] 0 2024-09-21T04:15:13.670 INFO:tasks.workunit.client.0.smithi110.stdout:1/594: dwrite d2/f45 [0,4194304] 0 2024-09-21T04:15:13.671 INFO:tasks.workunit.client.0.smithi110.stdout:1/595: fdatasync d2/d9/d49/d4f/d53/f8c 0 2024-09-21T04:15:13.671 INFO:tasks.workunit.client.0.smithi110.stdout:1/596: dread - d2/db7/d86/fb9 zero size 2024-09-21T04:15:13.671 INFO:tasks.workunit.client.1.smithi148.stdout:7/335: unlink d5/d8/f1a 0 2024-09-21T04:15:13.672 INFO:tasks.workunit.client.0.smithi110.stdout:8/407: mknod d0/d7/d13/d1c/d41/c8f 0 2024-09-21T04:15:13.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/308: dread d5/d6/d2a/d2b/f35 [0,4194304] 0 2024-09-21T04:15:13.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/309: write d5/d6/d7/d11/f38 [873952,118916] 0 2024-09-21T04:15:13.673 INFO:tasks.workunit.client.0.smithi110.stdout:4/514: dwrite d17/d34/d3d/d20/d43/d51/fa1 [0,4194304] 0 2024-09-21T04:15:13.675 INFO:tasks.workunit.client.0.smithi110.stdout:5/360: symlink d1/d49/d69/l7f 0 2024-09-21T04:15:13.677 INFO:tasks.workunit.client.1.smithi148.stdout:6/246: symlink d0/d1a/d23/d32/l46 0 2024-09-21T04:15:13.677 INFO:tasks.workunit.client.1.smithi148.stdout:6/247: read - d0/d1a/d23/d16/f3f zero size 2024-09-21T04:15:13.680 INFO:tasks.workunit.client.0.smithi110.stdout:1/597: write d2/d9/d26/d3d/f6f [2417916,94726] 0 2024-09-21T04:15:13.681 INFO:tasks.workunit.client.0.smithi110.stdout:8/408: creat d0/d7/d23/d6c/f90 x:0 0 0 2024-09-21T04:15:13.681 INFO:tasks.workunit.client.0.smithi110.stdout:8/409: chown d0/d30/d54/l63 1 1 2024-09-21T04:15:13.684 INFO:tasks.workunit.client.1.smithi148.stdout:6/248: dread d0/d1a/d23/f19 [0,4194304] 0 2024-09-21T04:15:13.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/249: chown d0/f20 1992882774 1 2024-09-21T04:15:13.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/250: write d0/d8/d18/f25 [1030698,53307] 0 2024-09-21T04:15:13.690 INFO:tasks.workunit.client.1.smithi148.stdout:4/307: dwrite d4/d12/d16/d29/f59 [0,4194304] 0 2024-09-21T04:15:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: mgrmap e22: smithi148.zjrahi(active, since 1.03933s) 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: Checking for cert/key for grafana.smithi110 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: Migrating grafana.smithi110 cert to cert store 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: Migrating grafana.smithi110 key to cert store 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/trash_purge_schedule"}]: dispatch 2024-09-21T04:15:13.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:13 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.zjrahi/trash_purge_schedule"}]: dispatch 2024-09-21T04:15:13.693 INFO:tasks.workunit.client.1.smithi148.stdout:8/301: dwrite dc/f17 [0,4194304] 0 2024-09-21T04:15:13.693 INFO:tasks.workunit.client.1.smithi148.stdout:8/302: write f8 [1095555,40971] 0 2024-09-21T04:15:13.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/286: link db/dd/l19 db/d24/d2e/d59/l5d 0 2024-09-21T04:15:13.700 INFO:tasks.workunit.client.1.smithi148.stdout:5/318: rmdir d3/d14/d41/d70 0 2024-09-21T04:15:13.700 INFO:tasks.workunit.client.1.smithi148.stdout:5/319: dread - d3/f5b zero size 2024-09-21T04:15:13.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/254: sync 2024-09-21T04:15:13.702 INFO:tasks.workunit.client.0.smithi110.stdout:6/290: sync 2024-09-21T04:15:13.702 INFO:tasks.workunit.client.0.smithi110.stdout:3/336: sync 2024-09-21T04:15:13.702 INFO:tasks.workunit.client.1.smithi148.stdout:1/287: dread db/dd/d21/f43 [0,4194304] 0 2024-09-21T04:15:13.703 INFO:tasks.workunit.client.0.smithi110.stdout:7/334: sync 2024-09-21T04:15:13.703 INFO:tasks.workunit.client.0.smithi110.stdout:7/335: truncate dd/d1c/d39/f5a 332348 0 2024-09-21T04:15:13.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/336: symlink d5/dc/d28/l6c 0 2024-09-21T04:15:13.704 INFO:tasks.workunit.client.0.smithi110.stdout:4/515: truncate d17/d1e/f33 1793862 0 2024-09-21T04:15:13.705 INFO:tasks.workunit.client.0.smithi110.stdout:4/516: write d17/d1e/d64/f8f [703997,109666] 0 2024-09-21T04:15:13.705 INFO:tasks.workunit.client.0.smithi110.stdout:4/517: fdatasync d17/f32 0 2024-09-21T04:15:13.708 INFO:tasks.workunit.client.1.smithi148.stdout:0/310: rename d5/d6/d7/d11/c2e to d5/d6/d2a/d2b/c5d 0 2024-09-21T04:15:13.708 INFO:tasks.workunit.client.0.smithi110.stdout:1/598: dwrite d2/d9/d49/d5c/f84 [0,4194304] 0 2024-09-21T04:15:13.708 INFO:tasks.workunit.client.0.smithi110.stdout:2/195: sync 2024-09-21T04:15:13.710 INFO:tasks.workunit.client.0.smithi110.stdout:8/410: dwrite d0/d7/d13/d4b/f17 [4194304,4194304] 0 2024-09-21T04:15:13.711 INFO:tasks.workunit.client.0.smithi110.stdout:5/361: creat d1/d49/d4d/d1e/d5a/d64/f80 x:0 0 0 2024-09-21T04:15:13.711 INFO:tasks.workunit.client.1.smithi148.stdout:1/288: dread db/d24/d4f/f55 [0,4194304] 0 2024-09-21T04:15:13.711 INFO:tasks.workunit.client.1.smithi148.stdout:6/251: symlink d0/l47 0 2024-09-21T04:15:13.713 INFO:tasks.workunit.client.0.smithi110.stdout:5/362: dread d1/d22/d51/f65 [0,4194304] 0 2024-09-21T04:15:13.715 INFO:tasks.workunit.client.0.smithi110.stdout:9/255: rename d7 to d7/d26/d55 22 2024-09-21T04:15:13.719 INFO:tasks.workunit.client.0.smithi110.stdout:6/291: creat d4/d12/d16/d48/d4b/f5e x:0 0 0 2024-09-21T04:15:13.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/613: sync 2024-09-21T04:15:13.725 INFO:tasks.workunit.client.1.smithi148.stdout:7/337: dwrite d5/dc/f5d [0,4194304] 0 2024-09-21T04:15:13.725 INFO:tasks.workunit.client.1.smithi148.stdout:7/338: chown d5/dc/d28/d4a 26972570 1 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.1.smithi148.stdout:7/339: chown d5/d14/d25/d40/c47 18 1 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.1.smithi148.stdout:7/340: fdatasync d5/dc/d28/f53 0 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.1.smithi148.stdout:7/341: readlink d5/l1f 0 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/337: dwrite d0/d12/d16/d1a/d6f/f73 [0,4194304] 0 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/338: dread - d0/d22/d29/f5f zero size 2024-09-21T04:15:13.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/339: write d0/d12/f6c [674149,25323] 0 2024-09-21T04:15:13.731 INFO:tasks.workunit.client.0.smithi110.stdout:1/599: symlink d2/d9/d26/d3d/d8f/lc7 0 2024-09-21T04:15:13.731 INFO:tasks.workunit.client.1.smithi148.stdout:8/303: mknod dc/d3e/c63 0 2024-09-21T04:15:13.733 INFO:tasks.workunit.client.0.smithi110.stdout:2/196: rmdir d2/d1a/d1f/d29/d19/d2d 0 2024-09-21T04:15:13.734 INFO:tasks.workunit.client.1.smithi148.stdout:0/311: dwrite d5/d6/f1f [0,4194304] 0 2024-09-21T04:15:13.735 INFO:tasks.workunit.client.1.smithi148.stdout:4/308: rename d4/d12/d16/d26/f55 to d4/d12/d2c/f5c 0 2024-09-21T04:15:13.736 INFO:tasks.workunit.client.0.smithi110.stdout:5/363: rmdir d1/dc 39 2024-09-21T04:15:13.736 INFO:tasks.workunit.client.0.smithi110.stdout:5/364: dread - d1/d22/d51/f54 zero size 2024-09-21T04:15:13.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/252: symlink d0/d1a/d23/d16/d36/l48 0 2024-09-21T04:15:13.738 INFO:tasks.workunit.client.0.smithi110.stdout:9/256: unlink d7/d26/f2d 0 2024-09-21T04:15:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:9/264: sync 2024-09-21T04:15:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:3/275: sync 2024-09-21T04:15:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:2/387: sync 2024-09-21T04:15:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:9/265: truncate d5/fe 1538813 0 2024-09-21T04:15:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:2/388: creat d0/d9/dc/d13/d25/f79 x:0 0 0 2024-09-21T04:15:13.742 INFO:tasks.workunit.client.1.smithi148.stdout:7/342: rmdir d5/dc/d24/d6b 0 2024-09-21T04:15:13.742 INFO:tasks.workunit.client.0.smithi110.stdout:7/336: rmdir dd/d4d 39 2024-09-21T04:15:13.742 INFO:tasks.workunit.client.0.smithi110.stdout:7/337: stat dd/d1c/d39/f4f 0 2024-09-21T04:15:13.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/304: truncate dc/d13/d28/f2b 3095011 0 2024-09-21T04:15:13.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/305: getdents dc/d13/d28/d62 0 2024-09-21T04:15:13.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/306: readlink dc/l23 0 2024-09-21T04:15:13.743 INFO:tasks.workunit.client.1.smithi148.stdout:3/276: dread d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:13.747 INFO:tasks.workunit.client.1.smithi148.stdout:4/309: unlink d4/d12/d16/l4c 0 2024-09-21T04:15:13.747 INFO:tasks.workunit.client.0.smithi110.stdout:6/292: dwrite d4/d12/d16/f5b [0,4194304] 0 2024-09-21T04:15:13.748 INFO:tasks.workunit.client.1.smithi148.stdout:1/289: dwrite db/dd/d21/f5c [0,4194304] 0 2024-09-21T04:15:13.748 INFO:tasks.workunit.client.0.smithi110.stdout:6/293: read d4/d12/d16/f59 [124768,70673] 0 2024-09-21T04:15:13.748 INFO:tasks.workunit.client.0.smithi110.stdout:6/294: chown d4/d12/d17/l35 62 1 2024-09-21T04:15:13.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/253: creat d0/d8/f49 x:0 0 0 2024-09-21T04:15:13.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/254: dread - d0/d1a/d23/f37 zero size 2024-09-21T04:15:13.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/255: chown d0/d1a/d23/d27/f2d 19 1 2024-09-21T04:15:13.752 INFO:tasks.workunit.client.1.smithi148.stdout:9/266: symlink d5/d1e/l68 0 2024-09-21T04:15:13.755 INFO:tasks.workunit.client.1.smithi148.stdout:2/389: symlink d0/d9/d16/d55/d71/l7a 0 2024-09-21T04:15:13.755 INFO:tasks.workunit.client.1.smithi148.stdout:2/390: dread - d0/d9/f6e zero size 2024-09-21T04:15:13.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/391: write d0/d9/dc/f46 [2029058,106244] 0 2024-09-21T04:15:13.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/392: write d0/d2b/f53 [1409801,86379] 0 2024-09-21T04:15:13.756 INFO:tasks.workunit.client.1.smithi148.stdout:7/343: getdents d5/dc/d28 0 2024-09-21T04:15:13.759 INFO:tasks.workunit.client.1.smithi148.stdout:7/344: dread d5/dc/f10 [0,4194304] 0 2024-09-21T04:15:13.759 INFO:tasks.workunit.client.1.smithi148.stdout:8/307: rename dc/d2a/c3f to dc/d43/c64 0 2024-09-21T04:15:13.760 INFO:tasks.workunit.client.1.smithi148.stdout:3/277: mknod d3/d1f/d4d/c57 0 2024-09-21T04:15:13.762 INFO:tasks.workunit.client.1.smithi148.stdout:0/312: rmdir d5/d6/d2a/d2b 39 2024-09-21T04:15:13.762 INFO:tasks.workunit.client.1.smithi148.stdout:0/313: fsync d5/d6/d2a/d2b/f35 0 2024-09-21T04:15:13.764 INFO:tasks.workunit.client.1.smithi148.stdout:4/310: mknod d4/d23/d2a/c5d 0 2024-09-21T04:15:13.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/290: rmdir db/d24 39 2024-09-21T04:15:13.768 INFO:tasks.workunit.client.1.smithi148.stdout:9/267: truncate d5/d1e/f2d 2400782 0 2024-09-21T04:15:13.769 INFO:tasks.workunit.client.1.smithi148.stdout:9/268: read - d5/d60/d28/d2f/f62 zero size 2024-09-21T04:15:13.769 INFO:tasks.workunit.client.0.smithi110.stdout:4/518: rmdir d17 39 2024-09-21T04:15:13.769 INFO:tasks.workunit.client.0.smithi110.stdout:4/519: write d17/d34/d3d/d20/d43/d51/d7a/d86/fae [157919,81491] 0 2024-09-21T04:15:13.770 INFO:tasks.workunit.client.1.smithi148.stdout:6/256: dwrite d0/d1a/d23/d27/f2d [0,4194304] 0 2024-09-21T04:15:13.772 INFO:tasks.workunit.client.0.smithi110.stdout:6/295: dwrite d4/d12/d17/d2e/d40/f2f [0,4194304] 0 2024-09-21T04:15:13.772 INFO:tasks.workunit.client.0.smithi110.stdout:6/296: truncate d4/d12/d17/d33/f55 619355 0 2024-09-21T04:15:13.772 INFO:tasks.workunit.client.0.smithi110.stdout:6/297: chown d4/d3c/l3f 352 1 2024-09-21T04:15:13.772 INFO:tasks.workunit.client.1.smithi148.stdout:2/393: mkdir d0/d9/d16/d55/d6f/d7b 0 2024-09-21T04:15:13.772 INFO:tasks.workunit.client.1.smithi148.stdout:2/394: chown d0/d9/d38/f4f 6562367 1 2024-09-21T04:15:13.773 INFO:tasks.workunit.client.1.smithi148.stdout:1/291: read db/d24/f2b [2112332,36246] 0 2024-09-21T04:15:13.774 INFO:tasks.workunit.client.1.smithi148.stdout:7/345: creat d5/dc/d28/f6d x:0 0 0 2024-09-21T04:15:13.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/340: unlink d0/d12/f58 0 2024-09-21T04:15:13.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/341: write d0/d12/d3c/f45 [1062635,1721] 0 2024-09-21T04:15:13.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/342: chown d0/d12/d3c/d5a/f64 2709 1 2024-09-21T04:15:13.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/314: mknod d5/d6/d7/d22/c5e 0 2024-09-21T04:15:13.781 INFO:tasks.workunit.client.0.smithi110.stdout:1/600: mknod d2/d9/d23/d3b/cc8 0 2024-09-21T04:15:13.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/395: write d0/d9/dc/d13/d25/d2d/f2e [922875,94243] 0 2024-09-21T04:15:13.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/396: readlink d0/d9/d16/d55/d6f/l72 0 2024-09-21T04:15:13.783 INFO:tasks.workunit.client.1.smithi148.stdout:9/269: mkdir d5/d60/d69 0 2024-09-21T04:15:13.783 INFO:tasks.workunit.client.1.smithi148.stdout:9/270: fdatasync d5/d60/d28/f2c 0 2024-09-21T04:15:13.786 INFO:tasks.workunit.client.1.smithi148.stdout:6/257: link d0/d1a/d23/fb d0/d1a/d23/dc/d33/f4a 0 2024-09-21T04:15:13.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/308: dwrite dc/d13/d28/f48 [0,4194304] 0 2024-09-21T04:15:13.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/309: fsync f9 0 2024-09-21T04:15:13.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/310: fsync dc/d13/d28/f2f 0 2024-09-21T04:15:13.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/311: write f7 [4243123,32683] 0 2024-09-21T04:15:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:1/292: symlink db/dd/l5e 0 2024-09-21T04:15:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:1/293: creat db/d46/f5f x:0 0 0 2024-09-21T04:15:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:3/278: dwrite d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:3/279: write d3/d5/d6/d12/da/d23/d28/f42 [881179,22207] 0 2024-09-21T04:15:13.789 INFO:tasks.workunit.client.1.smithi148.stdout:3/280: write d3/d5/d6/d12/da/d23/f26 [43081,115108] 0 2024-09-21T04:15:13.789 INFO:tasks.workunit.client.1.smithi148.stdout:3/281: write d3/d5/d6/f17 [668260,129615] 0 2024-09-21T04:15:13.789 INFO:tasks.workunit.client.1.smithi148.stdout:0/315: symlink d5/d6/d7/d11/l5f 0 2024-09-21T04:15:13.789 INFO:tasks.workunit.client.1.smithi148.stdout:4/311: rmdir d4/d12/d16/d29 39 2024-09-21T04:15:13.791 INFO:tasks.workunit.client.1.smithi148.stdout:9/271: creat d5/d60/d28/f6a x:0 0 0 2024-09-21T04:15:13.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/272: truncate d5/d1e/d37/f4d 438110 0 2024-09-21T04:15:13.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/258: mkdir d0/d1a/d4b 0 2024-09-21T04:15:13.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/273: dread d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:13.796 INFO:tasks.workunit.client.0.smithi110.stdout:0/614: rmdir d4/dd 39 2024-09-21T04:15:13.796 INFO:tasks.workunit.client.0.smithi110.stdout:0/615: fdatasync d4/dd/d53/d96/fb0 0 2024-09-21T04:15:13.798 INFO:tasks.workunit.client.0.smithi110.stdout:7/338: mknod dd/d1c/d39/d5d/c62 0 2024-09-21T04:15:13.804 INFO:tasks.workunit.client.0.smithi110.stdout:4/520: symlink d17/d34/d3d/d20/lb4 0 2024-09-21T04:15:13.805 INFO:tasks.workunit.client.0.smithi110.stdout:6/298: mknod d4/d12/d16/d48/d54/c5f 0 2024-09-21T04:15:13.807 INFO:tasks.workunit.client.1.smithi148.stdout:7/346: dwrite d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:13.807 INFO:tasks.workunit.client.1.smithi148.stdout:7/347: chown d5/dc/d28/d4a/f68 329824 1 2024-09-21T04:15:13.807 INFO:tasks.workunit.client.1.smithi148.stdout:7/348: readlink d5/dc/l59 0 2024-09-21T04:15:13.807 INFO:tasks.workunit.client.0.smithi110.stdout:3/343: mkdir d0/d51/d7b 0 2024-09-21T04:15:13.808 INFO:tasks.workunit.client.0.smithi110.stdout:3/344: dread d0/d12/d16/d1a/f3b [0,4194304] 0 2024-09-21T04:15:13.808 INFO:tasks.workunit.client.0.smithi110.stdout:1/601: mkdir d2/d9/d23/d4e/d57/dc9 0 2024-09-21T04:15:13.811 INFO:tasks.workunit.client.1.smithi148.stdout:2/397: dwrite d0/d9/dc/f46 [0,4194304] 0 2024-09-21T04:15:13.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/294: dwrite db/d24/d4f/f55 [0,4194304] 0 2024-09-21T04:15:13.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/295: write f6 [1081791,69406] 0 2024-09-21T04:15:13.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/296: chown db/d24/d2e/l3d 0 1 2024-09-21T04:15:13.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/602: dread d2/d9/d49/d4f/d53/f85 [0,4194304] 0 2024-09-21T04:15:13.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/603: fdatasync d2/d9/d26/d3d/f6f 0 2024-09-21T04:15:13.812 INFO:tasks.workunit.client.0.smithi110.stdout:0/616: symlink d4/dd/d60/d72/lb7 0 2024-09-21T04:15:13.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/604: dread d2/d9/d49/d5c/d91/fa5 [0,4194304] 0 2024-09-21T04:15:13.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/605: read d2/d9/d26/d3d/f6f [1145328,109669] 0 2024-09-21T04:15:13.814 INFO:tasks.workunit.client.0.smithi110.stdout:7/339: mknod dd/c63 0 2024-09-21T04:15:13.816 INFO:tasks.workunit.client.0.smithi110.stdout:4/521: rmdir d17/d34/d3d/d20/d43/d51 39 2024-09-21T04:15:13.818 INFO:tasks.workunit.client.0.smithi110.stdout:6/299: rmdir d4/de/d5c 0 2024-09-21T04:15:13.819 INFO:tasks.workunit.client.1.smithi148.stdout:2/398: write d0/d9/d59/f5a [1944558,19396] 0 2024-09-21T04:15:13.820 INFO:tasks.workunit.client.0.smithi110.stdout:3/345: symlink d0/d12/d16/l7c 0 2024-09-21T04:15:13.820 INFO:tasks.workunit.client.0.smithi110.stdout:3/346: getdents d0/d12/d3c/d5a/d70 0 2024-09-21T04:15:13.820 INFO:tasks.workunit.client.0.smithi110.stdout:3/347: fdatasync d0/d12/f1b 0 2024-09-21T04:15:13.820 INFO:tasks.workunit.client.0.smithi110.stdout:0/617: write d4/dd/d10/f1a [835975,16371] 0 2024-09-21T04:15:13.822 INFO:tasks.workunit.client.0.smithi110.stdout:2/197: rename d2/d1a/d1f/d2a/d18 to d2/d1a/d1f/d29/d19/d36 0 2024-09-21T04:15:13.824 INFO:tasks.workunit.client.1.smithi148.stdout:0/316: mkdir d5/d6/d7/d11/d60 0 2024-09-21T04:15:13.824 INFO:tasks.workunit.client.0.smithi110.stdout:4/522: read d17/f3c [1810889,26018] 0 2024-09-21T04:15:13.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/312: mknod d4/d23/c5e 0 2024-09-21T04:15:13.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/313: write d4/d23/f4a [901381,56253] 0 2024-09-21T04:15:13.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/314: chown d4/d12/d2c/f5c 11720834 1 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/259: mknod d0/d34/d38/d43/c4c 0 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/260: chown d0/d1a/d23/d16/f30 5582804 1 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/274: creat d5/d60/d4c/f6b x:0 0 0 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/275: readlink d5/l7 0 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:8/312: truncate dc/d13/d28/f48 3020096 0 2024-09-21T04:15:13.828 INFO:tasks.workunit.client.1.smithi148.stdout:8/313: fsync dc/d13/d3c/f49 0 2024-09-21T04:15:13.832 INFO:tasks.workunit.client.1.smithi148.stdout:7/349: creat d5/dc/f6e x:0 0 0 2024-09-21T04:15:13.833 INFO:tasks.workunit.client.1.smithi148.stdout:6/261: dread d0/d1a/d23/f14 [0,4194304] 0 2024-09-21T04:15:13.833 INFO:tasks.workunit.client.0.smithi110.stdout:7/340: symlink dd/d1c/l64 0 2024-09-21T04:15:13.833 INFO:tasks.workunit.client.0.smithi110.stdout:7/341: dread - dd/d14/d17/f54 zero size 2024-09-21T04:15:13.834 INFO:tasks.workunit.client.1.smithi148.stdout:7/350: write d5/dc/f10 [82839,30312] 0 2024-09-21T04:15:13.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/300: symlink d4/d12/d16/d52/l60 0 2024-09-21T04:15:13.838 INFO:tasks.workunit.client.0.smithi110.stdout:3/348: mknod d0/d22/c7d 0 2024-09-21T04:15:13.839 INFO:tasks.workunit.client.1.smithi148.stdout:2/399: dwrite d0/d9/d16/f37 [0,4194304] 0 2024-09-21T04:15:13.839 INFO:tasks.workunit.client.1.smithi148.stdout:1/297: symlink db/d46/d4c/l60 0 2024-09-21T04:15:13.839 INFO:tasks.workunit.client.1.smithi148.stdout:1/298: fdatasync db/dd/f12 0 2024-09-21T04:15:13.839 INFO:tasks.workunit.client.1.smithi148.stdout:1/299: fdatasync db/d24/f26 0 2024-09-21T04:15:13.840 INFO:tasks.workunit.client.0.smithi110.stdout:0/618: mknod d4/dd/d53/cb8 0 2024-09-21T04:15:13.840 INFO:tasks.workunit.client.0.smithi110.stdout:0/619: write d4/dd/d10/f2a [1701583,1330] 0 2024-09-21T04:15:13.843 INFO:tasks.workunit.client.1.smithi148.stdout:3/282: getdents d3/d1f 0 2024-09-21T04:15:13.843 INFO:tasks.workunit.client.1.smithi148.stdout:3/283: fdatasync d3/f4c 0 2024-09-21T04:15:13.844 INFO:tasks.workunit.client.0.smithi110.stdout:2/198: mknod d2/d1a/d1f/c37 0 2024-09-21T04:15:13.846 INFO:tasks.workunit.client.1.smithi148.stdout:9/276: unlink d5/d2b/f53 0 2024-09-21T04:15:13.849 INFO:tasks.workunit.client.0.smithi110.stdout:5/365: rename d1/d22/d51/f54 to d1/d49/d4d/d1e/f81 0 2024-09-21T04:15:13.849 INFO:tasks.workunit.client.0.smithi110.stdout:5/366: chown d1/d49 10274374 1 2024-09-21T04:15:13.849 INFO:tasks.workunit.client.0.smithi110.stdout:5/367: dread - d1/d7/d58/f60 zero size 2024-09-21T04:15:13.849 INFO:tasks.workunit.client.0.smithi110.stdout:5/368: creat d1/d7/d58/f82 x:0 0 0 2024-09-21T04:15:13.852 INFO:tasks.workunit.client.1.smithi148.stdout:8/314: rename dc/l16 to dc/d43/d4c/d4f/l65 0 2024-09-21T04:15:13.852 INFO:tasks.workunit.client.1.smithi148.stdout:8/315: write f9 [2943780,128599] 0 2024-09-21T04:15:13.853 INFO:tasks.workunit.client.0.smithi110.stdout:4/523: rmdir d17/d34/d3d/d20/d43/d51/d7a/d86 39 2024-09-21T04:15:13.853 INFO:tasks.workunit.client.0.smithi110.stdout:4/524: read - d17/d1e/d21/f54 zero size 2024-09-21T04:15:13.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/351: dwrite d5/dc/d28/f49 [0,4194304] 0 2024-09-21T04:15:13.854 INFO:tasks.workunit.client.1.smithi148.stdout:6/262: truncate d0/d8/f24 2684964 0 2024-09-21T04:15:13.855 INFO:tasks.workunit.client.0.smithi110.stdout:1/606: rmdir d2/d9/d49/d4f/d53 39 2024-09-21T04:15:13.857 INFO:tasks.workunit.client.1.smithi148.stdout:0/317: dwrite d5/d6/d3e/f49 [0,4194304] 0 2024-09-21T04:15:13.859 INFO:tasks.workunit.client.0.smithi110.stdout:7/342: link dd/d48/d55/f5f dd/f65 0 2024-09-21T04:15:13.860 INFO:tasks.workunit.client.1.smithi148.stdout:5/320: sync 2024-09-21T04:15:13.861 INFO:tasks.workunit.client.1.smithi148.stdout:5/321: dread d3/d37/f5f [0,4194304] 0 2024-09-21T04:15:13.861 INFO:tasks.workunit.client.0.smithi110.stdout:3/349: creat d0/d51/f7e x:0 0 0 2024-09-21T04:15:13.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/400: rmdir d0/d9/d16/d55/d6f/d7b 0 2024-09-21T04:15:13.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/401: getdents d0 0 2024-09-21T04:15:13.865 INFO:tasks.workunit.client.1.smithi148.stdout:4/315: dwrite d4/d12/d16/d29/f59 [4194304,4194304] 0 2024-09-21T04:15:13.865 INFO:tasks.workunit.client.1.smithi148.stdout:3/284: dwrite d3/d5/d6/d12/da/d52/f4a [0,4194304] 0 2024-09-21T04:15:13.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/301: dwrite d4/d12/d17/d33/d3e/f42 [0,4194304] 0 2024-09-21T04:15:13.867 INFO:tasks.workunit.client.1.smithi148.stdout:1/300: link db/d24/d2e/l44 db/d24/d2a/l61 0 2024-09-21T04:15:13.867 INFO:tasks.workunit.client.0.smithi110.stdout:0/620: creat d4/dd/d53/fb9 x:0 0 0 2024-09-21T04:15:13.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/277: link d5/d1e/f49 d5/d1e/d37/d46/f6c 0 2024-09-21T04:15:13.869 INFO:tasks.workunit.client.0.smithi110.stdout:2/199: rename d2/d1a/d1f/f2c to d2/d1a/d1f/f38 0 2024-09-21T04:15:13.871 INFO:tasks.workunit.client.1.smithi148.stdout:8/316: link dc/d2a/d42/f5c dc/d13/d39/f66 0 2024-09-21T04:15:13.872 INFO:tasks.workunit.client.0.smithi110.stdout:9/257: rename d7/d10/d24/c42 to d7/d10/c56 0 2024-09-21T04:15:13.877 INFO:tasks.workunit.client.0.smithi110.stdout:5/369: link d1/d49/d4d/d1e/d5a/f67 d1/d49/d4d/d1e/d4a/f83 0 2024-09-21T04:15:13.877 INFO:tasks.workunit.client.0.smithi110.stdout:5/370: creat d1/d7/d58/f84 x:0 0 0 2024-09-21T04:15:13.879 INFO:tasks.workunit.client.1.smithi148.stdout:7/352: truncate d5/dc/f3e 1056499 0 2024-09-21T04:15:13.882 INFO:tasks.workunit.client.0.smithi110.stdout:4/525: link d17/d34/d3d/d57/f9f d17/d34/d3d/d20/d43/d51/fb5 0 2024-09-21T04:15:13.883 INFO:tasks.workunit.client.1.smithi148.stdout:6/263: rmdir d0/d34 39 2024-09-21T04:15:13.887 INFO:tasks.workunit.client.0.smithi110.stdout:3/350: dwrite d0/d12/d3c/d5a/f64 [0,4194304] 0 2024-09-21T04:15:13.887 INFO:tasks.workunit.client.0.smithi110.stdout:3/351: fsync d0/f7 0 2024-09-21T04:15:13.888 INFO:tasks.workunit.client.0.smithi110.stdout:7/343: mkdir dd/d1c/d39/d56/d66 0 2024-09-21T04:15:13.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/344: write dd/d14/f36 [109670,42169] 0 2024-09-21T04:15:13.889 INFO:tasks.workunit.client.1.smithi148.stdout:3/285: dwrite d3/d1f/f47 [0,4194304] 0 2024-09-21T04:15:13.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/318: creat d5/d6/d2a/d2b/f61 x:0 0 0 2024-09-21T04:15:13.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/322: symlink d3/d34/d3c/l74 0 2024-09-21T04:15:13.896 INFO:tasks.workunit.client.0.smithi110.stdout:6/302: mknod d4/d12/d17/d33/d3e/c61 0 2024-09-21T04:15:13.896 INFO:tasks.workunit.client.0.smithi110.stdout:6/303: chown d4/l1a 2495 1 2024-09-21T04:15:13.898 INFO:tasks.workunit.client.1.smithi148.stdout:2/402: unlink d0/d9/l36 0 2024-09-21T04:15:13.898 INFO:tasks.workunit.client.1.smithi148.stdout:2/403: chown d0/d9/d16/d55 31561 1 2024-09-21T04:15:13.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/621: mknod d4/d1c/d62/d69/d9b/cba 0 2024-09-21T04:15:13.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/622: write d4/d47/d2c/d3a/f43 [1593973,71107] 0 2024-09-21T04:15:13.901 INFO:tasks.workunit.client.1.smithi148.stdout:5/323: dread d3/f9 [0,4194304] 0 2024-09-21T04:15:13.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/200: mknod d2/d1a/d1f/d2a/c39 0 2024-09-21T04:15:13.902 INFO:tasks.workunit.client.1.smithi148.stdout:3/286: dwrite d3/d5/d6/d12/da/d23/d28/f42 [0,4194304] 0 2024-09-21T04:15:13.902 INFO:tasks.workunit.client.1.smithi148.stdout:3/287: dread - d3/d1f/f3e zero size 2024-09-21T04:15:13.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/316: creat d4/d3a/f5f x:0 0 0 2024-09-21T04:15:13.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/317: chown d4/c14 12559868 1 2024-09-21T04:15:13.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/318: stat d4/d12/d16/f19 0 2024-09-21T04:15:13.905 INFO:tasks.workunit.client.0.smithi110.stdout:9/258: symlink d7/d10/l57 0 2024-09-21T04:15:13.905 INFO:tasks.workunit.client.0.smithi110.stdout:9/259: chown d7/d10/d24/d3c/d14/d31/f53 31992 1 2024-09-21T04:15:13.905 INFO:tasks.workunit.client.0.smithi110.stdout:9/260: read f2 [1076587,73026] 0 2024-09-21T04:15:13.910 INFO:tasks.workunit.client.0.smithi110.stdout:5/371: mknod d1/d49/d4d/d1e/d4a/c85 0 2024-09-21T04:15:13.910 INFO:tasks.workunit.client.0.smithi110.stdout:5/372: creat d1/d7/d18/d3f/f86 x:0 0 0 2024-09-21T04:15:13.910 INFO:tasks.workunit.client.0.smithi110.stdout:5/373: stat d1/d39/l2a 0 2024-09-21T04:15:13.911 INFO:tasks.workunit.client.0.smithi110.stdout:5/374: chown d1/d49/d4d/d1e/f5f 1 1 2024-09-21T04:15:13.911 INFO:tasks.workunit.client.0.smithi110.stdout:5/375: dread - d1/d49/d4d/d1e/d4a/f83 zero size 2024-09-21T04:15:13.912 INFO:tasks.workunit.client.1.smithi148.stdout:1/301: mknod db/d24/d4f/c62 0 2024-09-21T04:15:13.912 INFO:tasks.workunit.client.1.smithi148.stdout:1/302: readlink db/d24/d2e/d59/l5d 0 2024-09-21T04:15:13.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/526: getdents d17/d34/d3d/d20/d43/d51/d52 0 2024-09-21T04:15:13.913 INFO:tasks.workunit.client.0.smithi110.stdout:1/607: write d2/d9/d23/f75 [386094,69575] 0 2024-09-21T04:15:13.914 INFO:tasks.workunit.client.0.smithi110.stdout:1/608: fdatasync d2/d9/d23/d3b/f55 0 2024-09-21T04:15:13.914 INFO:tasks.workunit.client.0.smithi110.stdout:1/609: write d2/d9/d23/d4e/f61 [1823638,86926] 0 2024-09-21T04:15:13.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/278: creat d5/d1e/d37/d46/f6d x:0 0 0 2024-09-21T04:15:13.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/353: mkdir d5/d8/d6f 0 2024-09-21T04:15:13.922 INFO:tasks.workunit.client.0.smithi110.stdout:3/352: rename d0/d12/d16/d1a/f68 to d0/d12/d3c/f7f 0 2024-09-21T04:15:13.924 INFO:tasks.workunit.client.0.smithi110.stdout:7/345: symlink dd/d1c/d39/d5d/l67 0 2024-09-21T04:15:13.926 INFO:tasks.workunit.client.0.smithi110.stdout:5/376: dwrite d1/d39/f2d [0,4194304] 0 2024-09-21T04:15:13.926 INFO:tasks.workunit.client.0.smithi110.stdout:5/377: write d1/d39/f50 [5145290,118163] 0 2024-09-21T04:15:13.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/261: dwrite d7/d10/f12 [0,4194304] 0 2024-09-21T04:15:13.926 INFO:tasks.workunit.client.0.smithi110.stdout:5/378: dread d1/d7/d58/f77 [0,4194304] 0 2024-09-21T04:15:13.927 INFO:tasks.workunit.client.0.smithi110.stdout:5/379: creat d1/d7/d18/f87 x:0 0 0 2024-09-21T04:15:13.929 INFO:tasks.workunit.client.0.smithi110.stdout:0/623: mknod d4/dd/d10/d29/d75/cbb 0 2024-09-21T04:15:13.929 INFO:tasks.workunit.client.0.smithi110.stdout:0/624: truncate d4/d47/fb2 1041205 0 2024-09-21T04:15:13.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/380: dread d1/d39/f79 [0,4194304] 0 2024-09-21T04:15:13.930 INFO:tasks.workunit.client.0.smithi110.stdout:5/381: write d1/d39/f45 [2333284,66377] 0 2024-09-21T04:15:13.930 INFO:tasks.workunit.client.0.smithi110.stdout:5/382: creat d1/d39/f88 x:0 0 0 2024-09-21T04:15:13.931 INFO:tasks.workunit.client.0.smithi110.stdout:2/201: creat d2/d1a/d1f/d29/d19/d36/f3a x:0 0 0 2024-09-21T04:15:13.931 INFO:tasks.workunit.client.0.smithi110.stdout:2/202: fdatasync d2/d1a/d1f/d29/d19/d36/f3a 0 2024-09-21T04:15:13.938 INFO:tasks.workunit.client.0.smithi110.stdout:1/610: mkdir d2/d9/d23/d3b/dca 0 2024-09-21T04:15:13.938 INFO:tasks.workunit.client.0.smithi110.stdout:1/611: fsync d2/d9/d26/d3d/d8f/fbc 0 2024-09-21T04:15:13.940 INFO:tasks.workunit.client.0.smithi110.stdout:4/527: rename c15 to d17/d1e/d77/cb6 0 2024-09-21T04:15:13.942 INFO:tasks.workunit.client.0.smithi110.stdout:3/353: dwrite d0/d12/d3c/d5a/f64 [0,4194304] 0 2024-09-21T04:15:13.942 INFO:tasks.workunit.client.0.smithi110.stdout:7/346: creat dd/d4d/f68 x:0 0 0 2024-09-21T04:15:13.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/319: rename d5/d6/d7/d22/f37 to d5/d6/f62 0 2024-09-21T04:15:13.948 INFO:tasks.workunit.client.0.smithi110.stdout:2/203: dwrite d2/d1a/d1f/f38 [0,4194304] 0 2024-09-21T04:15:13.949 INFO:tasks.workunit.client.1.smithi148.stdout:5/324: mkdir d3/d34/d3c/d5d/d75 0 2024-09-21T04:15:13.950 INFO:tasks.workunit.client.0.smithi110.stdout:6/304: getdents d4/d12/d17/d2e/d40 0 2024-09-21T04:15:13.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/288: creat d3/d48/f58 x:0 0 0 2024-09-21T04:15:13.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/319: unlink d4/d12/d2c/d2d/d44/l50 0 2024-09-21T04:15:13.957 INFO:tasks.workunit.client.1.smithi148.stdout:1/303: creat db/d24/d2e/f63 x:0 0 0 2024-09-21T04:15:13.957 INFO:tasks.workunit.client.1.smithi148.stdout:1/304: write db/d24/f2b [2775145,9669] 0 2024-09-21T04:15:13.959 INFO:tasks.workunit.client.1.smithi148.stdout:9/279: mknod d5/d60/d28/c6e 0 2024-09-21T04:15:13.959 INFO:tasks.workunit.client.1.smithi148.stdout:9/280: read - d5/d2b/d32/f58 zero size 2024-09-21T04:15:13.960 INFO:tasks.workunit.client.0.smithi110.stdout:5/383: creat d1/d49/d4d/f89 x:0 0 0 2024-09-21T04:15:13.962 INFO:tasks.workunit.client.1.smithi148.stdout:7/354: unlink d5/d14/c2f 0 2024-09-21T04:15:13.965 INFO:tasks.workunit.client.0.smithi110.stdout:1/612: unlink d2/d9/d23/d4e/la7 0 2024-09-21T04:15:13.965 INFO:tasks.workunit.client.0.smithi110.stdout:1/613: chown d2/d9/d49/d5c/laa 767886707 1 2024-09-21T04:15:13.965 INFO:tasks.workunit.client.0.smithi110.stdout:1/614: chown d2/d9/d23/d3b/l74 33519227 1 2024-09-21T04:15:13.965 INFO:tasks.workunit.client.0.smithi110.stdout:1/615: write d2/d9/d49/d5c/d91/f96 [46387,121953] 0 2024-09-21T04:15:13.967 INFO:tasks.workunit.client.1.smithi148.stdout:5/325: dwrite d3/f7 [4194304,4194304] 0 2024-09-21T04:15:13.970 INFO:tasks.workunit.client.0.smithi110.stdout:9/262: rename d7/d10/d24/l27 to d7/d10/d24/d3c/d1f/d25/l58 0 2024-09-21T04:15:13.971 INFO:tasks.workunit.client.0.smithi110.stdout:4/528: symlink d17/d34/d3d/d20/d43/d51/d52/lb7 0 2024-09-21T04:15:13.972 INFO:tasks.workunit.client.0.smithi110.stdout:4/529: write d17/d34/d3d/d20/d43/d51/d7a/f7f [4779620,106962] 0 2024-09-21T04:15:13.972 INFO:tasks.workunit.client.0.smithi110.stdout:4/530: write d17/f58 [4760801,83264] 0 2024-09-21T04:15:13.978 INFO:tasks.workunit.client.1.smithi148.stdout:6/264: rmdir d0 39 2024-09-21T04:15:13.981 INFO:tasks.workunit.client.1.smithi148.stdout:2/404: rename d0/d2b/f61 to d0/d2b/d48/f7c 0 2024-09-21T04:15:13.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/384: dwrite d1/d7/d58/f82 [0,4194304] 0 2024-09-21T04:15:13.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/385: truncate d1/d7/f2b 1036302 0 2024-09-21T04:15:13.982 INFO:tasks.workunit.client.0.smithi110.stdout:4/531: write d17/d34/d3d/d20/f88 [99445,86046] 0 2024-09-21T04:15:13.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/305: dwrite db/dd/f13 [4194304,4194304] 0 2024-09-21T04:15:13.984 INFO:tasks.workunit.client.1.smithi148.stdout:3/289: creat d3/d5/d6/d12/da/d52/f59 x:0 0 0 2024-09-21T04:15:13.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/306: read db/dd/d21/f43 [508030,130089] 0 2024-09-21T04:15:13.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/307: fsync db/dd/f23 0 2024-09-21T04:15:13.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/308: fdatasync db/d24/d2e/f63 0 2024-09-21T04:15:13.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/309: fsync db/d24/d4f/f55 0 2024-09-21T04:15:13.990 INFO:tasks.workunit.client.1.smithi148.stdout:9/281: unlink d5/d60/d28/d2f/c4a 0 2024-09-21T04:15:13.991 INFO:tasks.workunit.client.0.smithi110.stdout:6/305: creat d4/d12/d16/d48/d54/f62 x:0 0 0 2024-09-21T04:15:13.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/625: rmdir d4/d1c/d62/d69/d9b 39 2024-09-21T04:15:13.994 INFO:tasks.workunit.client.0.smithi110.stdout:1/616: truncate d2/d9/d23/f40 692407 0 2024-09-21T04:15:13.995 INFO:tasks.workunit.client.0.smithi110.stdout:9/263: rename d7/d10/d24/d3c/d11/d33 to d7/d10/d24/d3c/d14/d3e/d59 0 2024-09-21T04:15:13.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/265: truncate d0/d8/f24 426083 0 2024-09-21T04:15:13.999 INFO:tasks.workunit.client.1.smithi148.stdout:0/320: rename d5/d6/d7/f31 to d5/d6/f63 0 2024-09-21T04:15:14.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/411: sync 2024-09-21T04:15:14.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/412: truncate d0/d7/f20 5631707 0 2024-09-21T04:15:14.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/405: truncate d0/d9/dc/d13/d25/f2a 2898021 0 2024-09-21T04:15:14.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/406: chown d0/d9/d38 80143 1 2024-09-21T04:15:14.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/407: chown d0/d9/dc/d13/c63 11900 1 2024-09-21T04:15:14.002 INFO:tasks.workunit.client.1.smithi148.stdout:2/408: creat d0/d2b/f7d x:0 0 0 2024-09-21T04:15:14.004 INFO:tasks.workunit.client.0.smithi110.stdout:5/386: dwrite d1/d22/f4f [0,4194304] 0 2024-09-21T04:15:14.006 INFO:tasks.workunit.client.0.smithi110.stdout:7/347: mknod dd/d1c/d39/d4a/d52/c69 0 2024-09-21T04:15:14.006 INFO:tasks.workunit.client.1.smithi148.stdout:3/290: getdents d3/d5/d6 0 2024-09-21T04:15:14.006 INFO:tasks.workunit.client.1.smithi148.stdout:6/266: dread d0/d8/f29 [0,4194304] 0 2024-09-21T04:15:14.007 INFO:tasks.workunit.client.0.smithi110.stdout:7/348: dread dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:14.007 INFO:tasks.workunit.client.0.smithi110.stdout:7/349: readlink dd/d14/l53 0 2024-09-21T04:15:14.007 INFO:tasks.workunit.client.0.smithi110.stdout:7/350: fdatasync dd/d1c/d39/f61 0 2024-09-21T04:15:14.008 INFO:tasks.workunit.client.1.smithi148.stdout:8/317: sync 2024-09-21T04:15:14.008 INFO:tasks.workunit.client.1.smithi148.stdout:8/318: truncate dc/d13/d3c/f5f 968112 0 2024-09-21T04:15:14.008 INFO:tasks.workunit.client.1.smithi148.stdout:8/319: chown dc/d2a/d42/f53 23 1 2024-09-21T04:15:14.012 INFO:tasks.workunit.client.0.smithi110.stdout:9/264: dwrite d7/d10/d24/d3c/d14/f54 [0,4194304] 0 2024-09-21T04:15:14.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/320: getdents d4/d12/d16 0 2024-09-21T04:15:14.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/321: readlink d4/d12/d16/l1d 0 2024-09-21T04:15:14.032 INFO:tasks.workunit.client.1.smithi148.stdout:1/310: truncate db/d46/f57 5621819 0 2024-09-21T04:15:14.033 INFO:tasks.workunit.client.0.smithi110.stdout:6/306: truncate d4/d12/d16/f56 107560 0 2024-09-21T04:15:14.034 INFO:tasks.workunit.client.0.smithi110.stdout:9/265: dwrite d7/d10/f37 [0,4194304] 0 2024-09-21T04:15:14.035 INFO:tasks.workunit.client.1.smithi148.stdout:8/320: dwrite f5 [4194304,4194304] 0 2024-09-21T04:15:14.035 INFO:tasks.workunit.client.1.smithi148.stdout:9/282: link d5/d60/f3c d5/d39/f6f 0 2024-09-21T04:15:14.037 INFO:tasks.workunit.client.0.smithi110.stdout:0/626: creat d4/fbc x:0 0 0 2024-09-21T04:15:14.039 INFO:tasks.workunit.client.1.smithi148.stdout:7/355: rename d5/f6a to d5/dc/d28/f70 0 2024-09-21T04:15:14.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/617: mknod d2/ccb 0 2024-09-21T04:15:14.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/618: stat d2/d9/d23/f82 0 2024-09-21T04:15:14.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/321: link d5/d6/d2a/d2b/f2c d5/d6/d3e/f64 0 2024-09-21T04:15:14.048 INFO:tasks.workunit.client.1.smithi148.stdout:2/409: creat d0/d9/d16/d55/d6f/f7e x:0 0 0 2024-09-21T04:15:14.050 INFO:tasks.workunit.client.0.smithi110.stdout:9/266: write d7/d10/f37 [2849287,130] 0 2024-09-21T04:15:14.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/267: creat d0/d8/d18/f4d x:0 0 0 2024-09-21T04:15:14.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/268: chown d0/d1a/d23/d32 107248 1 2024-09-21T04:15:14.051 INFO:tasks.workunit.client.1.smithi148.stdout:9/283: dwrite d5/d60/f3c [0,4194304] 0 2024-09-21T04:15:14.055 INFO:tasks.workunit.client.1.smithi148.stdout:4/322: mknod d4/d23/c60 0 2024-09-21T04:15:14.058 INFO:tasks.workunit.client.1.smithi148.stdout:9/284: write d5/d1f/f24 [1217118,99436] 0 2024-09-21T04:15:14.058 INFO:tasks.workunit.client.1.smithi148.stdout:9/285: fdatasync d5/d60/d28/d38/f52 0 2024-09-21T04:15:14.060 INFO:tasks.workunit.client.1.smithi148.stdout:1/311: unlink db/d24/d2a/c51 0 2024-09-21T04:15:14.060 INFO:tasks.workunit.client.0.smithi110.stdout:5/387: unlink d1/d22/c36 0 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/321: mknod dc/d2a/c67 0 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/322: creat dc/f68 x:0 0 0 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/269: write d0/d8/f29 [1392099,107572] 0 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/323: truncate dc/d13/f34 642452 0 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/270: chown d0/d1a/d23/d27/d2a 7 1 2024-09-21T04:15:14.063 INFO:tasks.workunit.client.1.smithi148.stdout:8/324: fsync dc/f46 0 2024-09-21T04:15:14.064 INFO:tasks.workunit.client.0.smithi110.stdout:7/351: creat dd/d1c/d39/d5d/f6a x:0 0 0 2024-09-21T04:15:14.066 INFO:tasks.workunit.client.0.smithi110.stdout:5/388: read d1/d49/d4d/d1e/f61 [165258,30440] 0 2024-09-21T04:15:14.068 INFO:tasks.workunit.client.1.smithi148.stdout:5/326: rename d3/d14/d1a/f31 to d3/d24/f76 0 2024-09-21T04:15:14.073 INFO:tasks.workunit.client.1.smithi148.stdout:0/322: mknod d5/d6/d2a/d45/c65 0 2024-09-21T04:15:14.081 INFO:tasks.workunit.client.1.smithi148.stdout:8/325: mknod dc/d13/c69 0 2024-09-21T04:15:14.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/291: rename d3/d5/d6/d12/da/d52/l29 to d3/d5/d6/d12/d43/l5a 0 2024-09-21T04:15:14.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/292: dread - d3/d48/f58 zero size 2024-09-21T04:15:14.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/293: write d3/d1f/f3e [263753,20539] 0 2024-09-21T04:15:14.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/294: getdents d3/d5/d6/d12/da/d23/d28 0 2024-09-21T04:15:14.085 INFO:tasks.workunit.client.0.smithi110.stdout:6/307: mkdir d4/d12/d16/d48/d63 0 2024-09-21T04:15:14.087 INFO:tasks.workunit.client.1.smithi148.stdout:0/323: unlink d5/d6/d2a/d45/l57 0 2024-09-21T04:15:14.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/324: chown d5/d6/d2a/d2b/c5d 109 1 2024-09-21T04:15:14.089 INFO:tasks.workunit.client.0.smithi110.stdout:1/619: dwrite d2/d9/d23/fa2 [0,4194304] 0 2024-09-21T04:15:14.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/627: mknod d4/dd/d10/d29/d45/cbd 0 2024-09-21T04:15:14.094 INFO:tasks.workunit.client.0.smithi110.stdout:6/308: read d4/d12/d17/d2e/d40/f2f [2255849,114156] 0 2024-09-21T04:15:14.094 INFO:tasks.workunit.client.0.smithi110.stdout:8/413: mkdir d0/d7/d13/d4b/d91 0 2024-09-21T04:15:14.094 INFO:tasks.workunit.client.0.smithi110.stdout:8/414: write d0/d30/f58 [1025524,4404] 0 2024-09-21T04:15:14.095 INFO:tasks.workunit.client.0.smithi110.stdout:9/267: link f6 d7/d10/d24/d3c/f5a 0 2024-09-21T04:15:14.098 INFO:tasks.workunit.client.0.smithi110.stdout:5/389: rename d1/dc/f1d to d1/f8a 0 2024-09-21T04:15:14.098 INFO:tasks.workunit.client.0.smithi110.stdout:5/390: creat d1/d7/d18/f8b x:0 0 0 2024-09-21T04:15:14.098 INFO:tasks.workunit.client.0.smithi110.stdout:5/391: fdatasync d1/f8a 0 2024-09-21T04:15:14.099 INFO:tasks.workunit.client.0.smithi110.stdout:1/620: symlink d2/d9/d49/d4f/d53/lcc 0 2024-09-21T04:15:14.100 INFO:tasks.workunit.client.0.smithi110.stdout:0/628: getdents d4/d47/d2c/d3a/d84 0 2024-09-21T04:15:14.102 INFO:tasks.workunit.client.0.smithi110.stdout:6/309: truncate d4/f6 744996 0 2024-09-21T04:15:14.102 INFO:tasks.workunit.client.0.smithi110.stdout:6/310: write d4/f7 [2410009,46299] 0 2024-09-21T04:15:14.103 INFO:tasks.workunit.client.0.smithi110.stdout:6/311: write d4/d12/d17/d2e/d38/f4f [706341,4161] 0 2024-09-21T04:15:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/415: mkdir d0/d7/d23/d82/d92 0 2024-09-21T04:15:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/416: fsync d0/d7/d13/f1e 0 2024-09-21T04:15:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/417: stat d0/d7/d13/d1c/c55 0 2024-09-21T04:15:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/418: chown d0/d7/d13/d46/f68 6560 1 2024-09-21T04:15:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/419: write d0/d7/d23/f33 [1840153,90164] 0 2024-09-21T04:15:14.105 INFO:tasks.workunit.client.0.smithi110.stdout:8/420: chown d0/d7/d89 6 1 2024-09-21T04:15:14.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/268: creat d7/d26/f5b x:0 0 0 2024-09-21T04:15:14.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/269: readlink d7/d10/d24/d3c/d14/l21 0 2024-09-21T04:15:14.109 INFO:tasks.workunit.client.0.smithi110.stdout:1/621: rename d2/d9/d23/d3b/d93/fab to d2/db7/fcd 0 2024-09-21T04:15:14.109 INFO:tasks.workunit.client.0.smithi110.stdout:1/622: fsync d2/d9/d49/d4f/d53/f8c 0 2024-09-21T04:15:14.112 INFO:tasks.workunit.client.1.smithi148.stdout:0/325: dread d5/d6/f63 [0,4194304] 0 2024-09-21T04:15:14.112 INFO:tasks.workunit.client.1.smithi148.stdout:4/323: dwrite d4/d12/d16/d29/f5a [0,4194304] 0 2024-09-21T04:15:14.112 INFO:tasks.workunit.client.1.smithi148.stdout:4/324: dread - d4/d12/d2c/f57 zero size 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:8/421: read d0/f3d [2076762,28508] 0 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:5/392: symlink d1/d7/l8c 0 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:5/393: creat d1/f8d x:0 0 0 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:5/394: dread - d1/d7/d18/d3f/f86 zero size 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:0/629: rename d4/dd/d60/d72/lb7 to d4/dd/lbe 0 2024-09-21T04:15:14.113 INFO:tasks.workunit.client.0.smithi110.stdout:5/395: rename d1/d49 to d1/d49/d69/d8e 22 2024-09-21T04:15:14.114 INFO:tasks.workunit.client.1.smithi148.stdout:3/295: write d3/d5/d6/d12/da/d23/f26 [8113556,106947] 0 2024-09-21T04:15:14.115 INFO:tasks.workunit.client.0.smithi110.stdout:1/623: fsync d2/d9/d23/f64 0 2024-09-21T04:15:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:1/624: getdents d2/d9/d23/d3b/d93/dbb 0 2024-09-21T04:15:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:1/625: creat d2/d9/d26/fce x:0 0 0 2024-09-21T04:15:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:1/626: fsync d2/d9/d26/d3d/fc5 0 2024-09-21T04:15:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:1/627: fsync d2/db7/d86/f95 0 2024-09-21T04:15:14.117 INFO:tasks.workunit.client.0.smithi110.stdout:6/312: dread d4/d12/d17/d33/d3e/f53 [0,4194304] 0 2024-09-21T04:15:14.120 INFO:tasks.workunit.client.1.smithi148.stdout:7/356: dwrite d5/dc/f19 [0,4194304] 0 2024-09-21T04:15:14.121 INFO:tasks.workunit.client.0.smithi110.stdout:6/313: dread d4/d12/d16/d48/d4b/f51 [0,4194304] 0 2024-09-21T04:15:14.123 INFO:tasks.workunit.client.1.smithi148.stdout:0/326: mkdir d5/d6/d7/d11/d48/d66 0 2024-09-21T04:15:14.123 INFO:tasks.workunit.client.1.smithi148.stdout:0/327: chown d5/d6/d2a/d2b/f39 210602 1 2024-09-21T04:15:14.124 INFO:tasks.workunit.client.1.smithi148.stdout:4/325: write d4/d12/f1b [2720407,98889] 0 2024-09-21T04:15:14.124 INFO:tasks.workunit.client.0.smithi110.stdout:5/396: mkdir d1/d7/d18/d8f 0 2024-09-21T04:15:14.125 INFO:tasks.workunit.client.1.smithi148.stdout:1/312: dwrite db/d46/f47 [0,4194304] 0 2024-09-21T04:15:14.125 INFO:tasks.workunit.client.1.smithi148.stdout:9/286: dwrite d5/d1e/f49 [0,4194304] 0 2024-09-21T04:15:14.125 INFO:tasks.workunit.client.1.smithi148.stdout:1/313: write db/d46/f57 [5642609,107702] 0 2024-09-21T04:15:14.126 INFO:tasks.workunit.client.1.smithi148.stdout:6/271: dwrite d0/d8/f10 [0,4194304] 0 2024-09-21T04:15:14.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/326: dwrite f9 [0,4194304] 0 2024-09-21T04:15:14.128 INFO:tasks.workunit.client.0.smithi110.stdout:7/352: dwrite dd/d1c/d39/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:14.128 INFO:tasks.workunit.client.1.smithi148.stdout:2/410: dwrite d0/d2b/f7d [0,4194304] 0 2024-09-21T04:15:14.128 INFO:tasks.workunit.client.1.smithi148.stdout:2/411: truncate d0/d2b/f76 4670937 0 2024-09-21T04:15:14.129 INFO:tasks.workunit.client.0.smithi110.stdout:9/270: fsync d7/d26/f5b 0 2024-09-21T04:15:14.129 INFO:tasks.workunit.client.1.smithi148.stdout:0/328: truncate d5/d6/f63 2706135 0 2024-09-21T04:15:14.131 INFO:tasks.workunit.client.0.smithi110.stdout:1/628: creat d2/d88/fcf x:0 0 0 2024-09-21T04:15:14.132 INFO:tasks.workunit.client.1.smithi148.stdout:3/296: dread d3/d1f/f3e [0,4194304] 0 2024-09-21T04:15:14.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/357: write d5/d14/f3c [1889175,2469] 0 2024-09-21T04:15:14.134 INFO:tasks.workunit.client.0.smithi110.stdout:5/397: write d1/d22/f3c [3997942,52509] 0 2024-09-21T04:15:14.135 INFO:tasks.workunit.client.0.smithi110.stdout:6/314: truncate d4/d12/d17/d2e/d40/f2f 5619881 0 2024-09-21T04:15:14.142 INFO:tasks.workunit.client.0.smithi110.stdout:7/353: mknod dd/c6b 0 2024-09-21T04:15:14.143 INFO:tasks.workunit.client.1.smithi148.stdout:9/287: symlink d5/d60/d28/l70 0 2024-09-21T04:15:14.144 INFO:tasks.workunit.client.0.smithi110.stdout:6/315: write d4/d12/d16/f5b [2093395,82333] 0 2024-09-21T04:15:14.144 INFO:tasks.workunit.client.0.smithi110.stdout:6/316: creat d4/d12/d17/d33/f64 x:0 0 0 2024-09-21T04:15:14.144 INFO:tasks.workunit.client.0.smithi110.stdout:9/271: mkdir d7/d10/d24/d3c/d14/d5c 0 2024-09-21T04:15:14.145 INFO:tasks.workunit.client.1.smithi148.stdout:1/314: creat db/f64 x:0 0 0 2024-09-21T04:15:14.146 INFO:tasks.workunit.client.0.smithi110.stdout:1/629: creat d2/fd0 x:0 0 0 2024-09-21T04:15:14.149 INFO:tasks.workunit.client.1.smithi148.stdout:6/272: rmdir d0/d34/d3c 0 2024-09-21T04:15:14.152 INFO:tasks.workunit.client.0.smithi110.stdout:0/630: dwrite d4/f57 [0,4194304] 0 2024-09-21T04:15:14.152 INFO:tasks.workunit.client.0.smithi110.stdout:7/354: mknod dd/d1c/d39/d4a/c6c 0 2024-09-21T04:15:14.152 INFO:tasks.workunit.client.0.smithi110.stdout:7/355: chown dd/f65 58329 1 2024-09-21T04:15:14.152 INFO:tasks.workunit.client.0.smithi110.stdout:7/356: truncate dd/d1c/f34 1899385 0 2024-09-21T04:15:14.152 INFO:tasks.workunit.client.0.smithi110.stdout:8/422: dwrite d0/d7/d13/d4b/f4e [0,4194304] 0 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/329: dwrite d5/d6/d7/f4f [0,4194304] 0 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/327: dwrite dc/f68 [0,4194304] 0 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/328: chown dc/f21 14490325 1 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/329: dread - dc/d43/f54 zero size 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/330: stat dc/d2a/f2e 0 2024-09-21T04:15:14.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/331: fsync f9 0 2024-09-21T04:15:14.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/272: truncate d7/d10/d24/d3c/d1f/f23 518319 0 2024-09-21T04:15:14.158 INFO:tasks.workunit.client.1.smithi148.stdout:3/297: creat d3/d5/d6/d12/da/d23/d28/d46/f5b x:0 0 0 2024-09-21T04:15:14.160 INFO:tasks.workunit.client.0.smithi110.stdout:5/398: symlink d1/d7/l90 0 2024-09-21T04:15:14.160 INFO:tasks.workunit.client.0.smithi110.stdout:5/399: readlink d1/dc/l3a 0 2024-09-21T04:15:14.167 INFO:tasks.workunit.client.0.smithi110.stdout:8/423: dread d0/f11 [4194304,4194304] 0 2024-09-21T04:15:14.167 INFO:tasks.workunit.client.0.smithi110.stdout:8/424: dread - d0/d7/d13/f57 zero size 2024-09-21T04:15:14.168 INFO:tasks.workunit.client.0.smithi110.stdout:8/425: stat d0/d7/d13/d4b/c71 0 2024-09-21T04:15:14.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/358: rename d5/d14/d25/d40/c47 to d5/d8/c71 0 2024-09-21T04:15:14.171 INFO:tasks.workunit.client.0.smithi110.stdout:5/400: write d1/f15 [6050118,87581] 0 2024-09-21T04:15:14.175 INFO:tasks.workunit.client.0.smithi110.stdout:0/631: symlink d4/d1c/d62/d69/lbf 0 2024-09-21T04:15:14.175 INFO:tasks.workunit.client.1.smithi148.stdout:0/330: dwrite d5/d6/f3c [0,4194304] 0 2024-09-21T04:15:14.176 INFO:tasks.workunit.client.0.smithi110.stdout:7/357: mkdir dd/d1c/d39/d6d 0 2024-09-21T04:15:14.177 INFO:tasks.workunit.client.0.smithi110.stdout:7/358: write dd/d1c/f34 [656774,102271] 0 2024-09-21T04:15:14.181 INFO:tasks.workunit.client.0.smithi110.stdout:9/273: stat d7/d10/d24/d3c/d1f/d43/l47 0 2024-09-21T04:15:14.182 INFO:tasks.workunit.client.0.smithi110.stdout:8/426: creat d0/d7/d6f/f93 x:0 0 0 2024-09-21T04:15:14.182 INFO:tasks.workunit.client.0.smithi110.stdout:8/427: write d0/d30/d34/d3e/d32/f7c [564488,5378] 0 2024-09-21T04:15:14.184 INFO:tasks.workunit.client.1.smithi148.stdout:4/326: getdents d4/d12/d2c 0 2024-09-21T04:15:14.184 INFO:tasks.workunit.client.0.smithi110.stdout:5/401: symlink d1/l91 0 2024-09-21T04:15:14.185 INFO:tasks.workunit.client.1.smithi148.stdout:7/359: write d5/dc/d28/f49 [4067024,105254] 0 2024-09-21T04:15:14.185 INFO:tasks.workunit.client.1.smithi148.stdout:9/288: mknod d5/d2b/d4e/c71 0 2024-09-21T04:15:14.186 INFO:tasks.workunit.client.0.smithi110.stdout:0/632: rename d4/dd/l7a to d4/d47/d2c/d3a/d84/lc0 0 2024-09-21T04:15:14.186 INFO:tasks.workunit.client.1.smithi148.stdout:0/331: write d5/d6/d7/d22/f46 [2601569,61634] 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/360: dread d5/d8/f62 [0,4194304] 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/361: chown d5/d8 235 1 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.1.smithi148.stdout:7/362: stat d5/d8 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.0.smithi110.stdout:7/359: unlink dd/f65 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.0.smithi110.stdout:7/360: fsync dd/d48/f4b 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.0.smithi110.stdout:6/317: dwrite d4/d12/d17/d33/f64 [0,4194304] 0 2024-09-21T04:15:14.189 INFO:tasks.workunit.client.0.smithi110.stdout:1/630: dwrite d2/d9/f1a [0,4194304] 0 2024-09-21T04:15:14.190 INFO:tasks.workunit.client.1.smithi148.stdout:6/273: unlink d0/d1a/d23/f37 0 2024-09-21T04:15:14.192 INFO:tasks.workunit.client.0.smithi110.stdout:9/274: link d7/d10/d24/d3c/d4f/l50 d7/d10/d24/d3c/d11/l5d 0 2024-09-21T04:15:14.192 INFO:tasks.workunit.client.0.smithi110.stdout:9/275: dread - d7/d10/d24/d3c/d11/f4e zero size 2024-09-21T04:15:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:6/274: write d0/d8/f24 [314110,107128] 0 2024-09-21T04:15:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:8/332: dwrite dc/f17 [0,4194304] 0 2024-09-21T04:15:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:8/333: truncate dc/d43/f44 456291 0 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.0.smithi110.stdout:8/428: link d0/d7/d13/f1e d0/d7/d13/f94 0 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.0.smithi110.stdout:8/429: chown d0/l15 4858 1 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/332: dread d5/d6/d7/d11/f17 [4194304,4194304] 0 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/333: truncate d5/d6/d2a/d2b/f5b 661742 0 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/334: fdatasync d5/d6/d7/f4f 0 2024-09-21T04:15:14.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/335: write d5/d6/d3e/f3a [265720,42085] 0 2024-09-21T04:15:14.196 INFO:tasks.workunit.client.0.smithi110.stdout:8/430: dread d0/d7/d13/d46/f68 [0,4194304] 0 2024-09-21T04:15:14.197 INFO:tasks.workunit.client.0.smithi110.stdout:5/402: stat d1/d49/c53 0 2024-09-21T04:15:14.198 INFO:tasks.workunit.client.0.smithi110.stdout:0/633: creat d4/dd/d53/d66/fc1 x:0 0 0 2024-09-21T04:15:14.200 INFO:tasks.workunit.client.0.smithi110.stdout:5/403: dread d1/f3e [0,4194304] 0 2024-09-21T04:15:14.200 INFO:tasks.workunit.client.0.smithi110.stdout:7/361: mknod dd/d1c/d39/d5d/c6e 0 2024-09-21T04:15:14.203 INFO:tasks.workunit.client.1.smithi148.stdout:8/334: dread dc/d13/d28/f2b [0,4194304] 0 2024-09-21T04:15:14.203 INFO:tasks.workunit.client.1.smithi148.stdout:4/327: dwrite d4/d12/d2c/d2d/d44/f4b [0,4194304] 0 2024-09-21T04:15:14.205 INFO:tasks.workunit.client.1.smithi148.stdout:2/412: rmdir d0/d9/d16/d55/d6f 39 2024-09-21T04:15:14.205 INFO:tasks.workunit.client.1.smithi148.stdout:2/413: readlink d0/d9/dc/d13/d25/l75 0 2024-09-21T04:15:14.206 INFO:tasks.workunit.client.0.smithi110.stdout:6/318: stat d4/d12/d16/c1c 0 2024-09-21T04:15:14.216 INFO:tasks.workunit.client.0.smithi110.stdout:7/362: dread dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:14.221 INFO:tasks.workunit.client.1.smithi148.stdout:1/315: rmdir db/d46 39 2024-09-21T04:15:14.224 INFO:tasks.workunit.client.1.smithi148.stdout:7/363: mknod d5/d14/c72 0 2024-09-21T04:15:14.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/631: creat d2/d9/d49/d4f/d87/fd1 x:0 0 0 2024-09-21T04:15:14.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/632: chown d2/db7/f5f 333 1 2024-09-21T04:15:14.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/633: write d2/d9/d26/d3d/f6f [4057484,55490] 0 2024-09-21T04:15:14.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/336: mknod d5/c67 0 2024-09-21T04:15:14.228 INFO:tasks.workunit.client.0.smithi110.stdout:9/276: rename d7/d26/f2c to d7/d10/d24/d3c/d35/f5e 0 2024-09-21T04:15:14.228 INFO:tasks.workunit.client.1.smithi148.stdout:8/335: rmdir dc/d2a/d50 39 2024-09-21T04:15:14.230 INFO:tasks.workunit.client.1.smithi148.stdout:4/328: dwrite d4/d12/f5b [0,4194304] 0 2024-09-21T04:15:14.230 INFO:tasks.workunit.client.1.smithi148.stdout:4/329: stat d4/d12/l1a 0 2024-09-21T04:15:14.231 INFO:tasks.workunit.client.0.smithi110.stdout:0/634: mknod d4/dd/d10/d29/cc2 0 2024-09-21T04:15:14.234 INFO:tasks.workunit.client.0.smithi110.stdout:5/404: symlink d1/d22/d51/l92 0 2024-09-21T04:15:14.234 INFO:tasks.workunit.client.0.smithi110.stdout:5/405: fdatasync d1/d7/d18/f41 0 2024-09-21T04:15:14.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/414: dwrite d0/d9/dc/d13/d25/f45 [0,4194304] 0 2024-09-21T04:15:14.235 INFO:tasks.workunit.client.0.smithi110.stdout:6/319: mkdir d4/d12/d17/d2e/d40/d65 0 2024-09-21T04:15:14.236 INFO:tasks.workunit.client.0.smithi110.stdout:6/320: chown d4/de 44 1 2024-09-21T04:15:14.237 INFO:tasks.workunit.client.1.smithi148.stdout:9/289: getdents d5/d1e/d37/d46 0 2024-09-21T04:15:14.239 INFO:tasks.workunit.client.0.smithi110.stdout:7/363: mknod dd/d1c/d39/d6d/c6f 0 2024-09-21T04:15:14.241 INFO:tasks.workunit.client.1.smithi148.stdout:0/337: rename d5/d6/d2a/d2b/l58 to d5/d6/d2a/d2b/l68 0 2024-09-21T04:15:14.241 INFO:tasks.workunit.client.1.smithi148.stdout:0/338: write d5/d6/d2a/d2b/f55 [2128,84758] 0 2024-09-21T04:15:14.242 INFO:tasks.workunit.client.0.smithi110.stdout:1/634: rmdir d2/db7/d86 39 2024-09-21T04:15:14.243 INFO:tasks.workunit.client.0.smithi110.stdout:9/277: dread d7/d10/d24/d3c/f5a [0,4194304] 0 2024-09-21T04:15:14.244 INFO:tasks.workunit.client.0.smithi110.stdout:0/635: getdents d4/dd/d60 0 2024-09-21T04:15:14.246 INFO:tasks.workunit.client.0.smithi110.stdout:6/321: mkdir d4/d12/d16/d66 0 2024-09-21T04:15:14.247 INFO:tasks.workunit.client.1.smithi148.stdout:3/298: rename d3/d48 to d3/d5/d5c 0 2024-09-21T04:15:14.249 INFO:tasks.workunit.client.0.smithi110.stdout:7/364: link dd/d1c/l47 dd/d48/d55/l70 0 2024-09-21T04:15:14.249 INFO:tasks.workunit.client.0.smithi110.stdout:7/365: dread - dd/d14/d17/f5b zero size 2024-09-21T04:15:14.250 INFO:tasks.workunit.client.0.smithi110.stdout:7/366: creat dd/d4d/f71 x:0 0 0 2024-09-21T04:15:14.252 INFO:tasks.workunit.client.0.smithi110.stdout:0/636: rmdir d4/dd/d10/daf 0 2024-09-21T04:15:14.252 INFO:tasks.workunit.client.0.smithi110.stdout:0/637: chown d4 1117664120 1 2024-09-21T04:15:14.254 INFO:tasks.workunit.client.0.smithi110.stdout:6/322: rename d4/d12/d16/d48/d54/f62 to d4/d12/d16/d66/f67 0 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/323: fsync d4/de/f44 0 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/324: chown d4/l2d 0 1 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/325: write d4/d12/d17/d33/d3e/f42 [4785628,70386] 0 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/326: read f3 [286754,119498] 0 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.1.smithi148.stdout:2/415: symlink d0/d9/dc/d13/d25/l7f 0 2024-09-21T04:15:14.255 INFO:tasks.workunit.client.1.smithi148.stdout:2/416: fsync d0/d2f/f4a 0 2024-09-21T04:15:14.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/635: dread d2/f94 [0,4194304] 0 2024-09-21T04:15:14.258 INFO:tasks.workunit.client.1.smithi148.stdout:0/339: dread d5/d6/d2a/d2b/f36 [0,4194304] 0 2024-09-21T04:15:14.259 INFO:tasks.workunit.client.0.smithi110.stdout:0/638: mkdir d4/dd/d10/d29/d5f/dc3 0 2024-09-21T04:15:14.259 INFO:tasks.workunit.client.0.smithi110.stdout:0/639: truncate d4/dd/d53/d66/fc1 994307 0 2024-09-21T04:15:14.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/327: dread d4/fc [0,4194304] 0 2024-09-21T04:15:14.261 INFO:tasks.workunit.client.1.smithi148.stdout:5/327: sync 2024-09-21T04:15:14.262 INFO:tasks.workunit.client.0.smithi110.stdout:9/278: dread d7/d10/d24/d3c/d35/f3a [0,4194304] 0 2024-09-21T04:15:14.262 INFO:tasks.workunit.client.1.smithi148.stdout:4/330: dwrite d4/f6 [4194304,4194304] 0 2024-09-21T04:15:14.262 INFO:tasks.workunit.client.1.smithi148.stdout:1/316: dwrite db/d46/f48 [0,4194304] 0 2024-09-21T04:15:14.263 INFO:tasks.workunit.client.0.smithi110.stdout:5/406: dwrite d1/d49/f71 [0,4194304] 0 2024-09-21T04:15:14.263 INFO:tasks.workunit.client.0.smithi110.stdout:5/407: truncate d1/d49/d4d/f46 1420765 0 2024-09-21T04:15:14.267 INFO:tasks.workunit.client.0.smithi110.stdout:1/636: creat d2/d9/d23/fd2 x:0 0 0 2024-09-21T04:15:14.268 INFO:tasks.workunit.client.1.smithi148.stdout:0/340: write f4 [265831,35991] 0 2024-09-21T04:15:14.271 INFO:tasks.workunit.client.0.smithi110.stdout:6/328: mknod d4/d12/d16/d48/d4b/c68 0 2024-09-21T04:15:14.271 INFO:tasks.workunit.client.0.smithi110.stdout:3/354: sync 2024-09-21T04:15:14.271 INFO:tasks.workunit.client.0.smithi110.stdout:4/532: sync 2024-09-21T04:15:14.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/204: sync 2024-09-21T04:15:14.271 INFO:tasks.workunit.client.0.smithi110.stdout:6/329: write d4/d12/f34 [2176106,33649] 0 2024-09-21T04:15:14.278 INFO:tasks.workunit.client.1.smithi148.stdout:6/275: rename d0/d1a/d23/dc/d33 to d0/d8/d18/d4e 0 2024-09-21T04:15:14.281 INFO:tasks.workunit.client.0.smithi110.stdout:4/533: dread d17/d1e/f79 [0,4194304] 0 2024-09-21T04:15:14.281 INFO:tasks.workunit.client.1.smithi148.stdout:3/299: link d3/d5/d6/d2b/l56 d3/d1f/l5d 0 2024-09-21T04:15:14.283 INFO:tasks.workunit.client.0.smithi110.stdout:0/640: dwrite d4/dd/f8e [0,4194304] 0 2024-09-21T04:15:14.283 INFO:tasks.workunit.client.0.smithi110.stdout:0/641: truncate d4/dd/d10/d29/d75/fa9 252569 0 2024-09-21T04:15:14.283 INFO:tasks.workunit.client.0.smithi110.stdout:0/642: truncate d4/f85 1812349 0 2024-09-21T04:15:14.283 INFO:tasks.workunit.client.0.smithi110.stdout:0/643: fsync f2 0 2024-09-21T04:15:14.283 INFO:tasks.workunit.client.1.smithi148.stdout:2/417: unlink d0/d2b/d48/f7c 0 2024-09-21T04:15:14.286 INFO:tasks.workunit.client.1.smithi148.stdout:5/328: creat d3/d14/d1f/d26/f77 x:0 0 0 2024-09-21T04:15:14.286 INFO:tasks.workunit.client.1.smithi148.stdout:5/329: chown d3/d14/d1f/f42 16439450 1 2024-09-21T04:15:14.288 INFO:tasks.workunit.client.1.smithi148.stdout:6/276: dread d0/f20 [0,4194304] 0 2024-09-21T04:15:14.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/355: mkdir d0/d12/d2a/d80 0 2024-09-21T04:15:14.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/356: dread - d0/d12/d16/d1a/d5c/f60 zero size 2024-09-21T04:15:14.290 INFO:tasks.workunit.client.0.smithi110.stdout:6/330: mknod d4/d12/d17/d2e/c69 0 2024-09-21T04:15:14.291 INFO:tasks.workunit.client.0.smithi110.stdout:6/331: write d4/f7 [4221796,43268] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/279: dwrite d7/d10/f37 [0,4194304] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/280: write d7/d10/d24/d3c/d14/d3e/f4d [936519,109288] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/281: read d7/d10/d24/d3c/f20 [888262,43795] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/282: readlink d7/d10/d24/d3c/ld 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/283: chown d7/d10/f12 0 1 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/408: dwrite d1/d49/d4d/f46 [0,4194304] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/284: write f2 [1940487,60215] 0 2024-09-21T04:15:14.292 INFO:tasks.workunit.client.0.smithi110.stdout:9/285: chown d7/d10/d24/d3c/d11/l5d 7564 1 2024-09-21T04:15:14.293 INFO:tasks.workunit.client.0.smithi110.stdout:5/409: creat d1/d7/d18/d3f/f93 x:0 0 0 2024-09-21T04:15:14.293 INFO:tasks.workunit.client.1.smithi148.stdout:3/300: dread d3/d5/d6/d12/da/d23/d28/f42 [0,4194304] 0 2024-09-21T04:15:14.293 INFO:tasks.workunit.client.0.smithi110.stdout:4/534: creat d17/d1e/d7b/fb8 x:0 0 0 2024-09-21T04:15:14.294 INFO:tasks.workunit.client.0.smithi110.stdout:1/637: dwrite d2/d9/d23/d4e/fad [0,4194304] 0 2024-09-21T04:15:14.294 INFO:tasks.workunit.client.0.smithi110.stdout:1/638: write d2/db7/f5f [656810,41409] 0 2024-09-21T04:15:14.302 INFO:tasks.workunit.client.0.smithi110.stdout:1/639: dread d2/d9/d26/d3d/d8f/fbc [0,4194304] 0 2024-09-21T04:15:14.302 INFO:tasks.workunit.client.0.smithi110.stdout:1/640: write d2/d9/d49/fbe [2938848,12286] 0 2024-09-21T04:15:14.302 INFO:tasks.workunit.client.0.smithi110.stdout:0/644: dwrite d4/dd/d10/d29/f5d [0,4194304] 0 2024-09-21T04:15:14.302 INFO:tasks.workunit.client.0.smithi110.stdout:0/645: chown d4/dd/d10/d29/l95 185953 1 2024-09-21T04:15:14.303 INFO:tasks.workunit.client.0.smithi110.stdout:5/410: write d1/f35 [114359,100119] 0 2024-09-21T04:15:14.303 INFO:tasks.workunit.client.1.smithi148.stdout:3/301: dread d3/d5/d6/d12/da/d23/f20 [0,4194304] 0 2024-09-21T04:15:14.303 INFO:tasks.workunit.client.1.smithi148.stdout:0/341: symlink d5/d6/d7/d22/l69 0 2024-09-21T04:15:14.304 INFO:tasks.workunit.client.0.smithi110.stdout:3/357: mkdir d0/d12/d3c/d5a/d70/d81 0 2024-09-21T04:15:14.306 INFO:tasks.workunit.client.1.smithi148.stdout:2/418: rmdir d0/d9/dc/d13/d25/d2d 39 2024-09-21T04:15:14.307 INFO:tasks.workunit.client.0.smithi110.stdout:5/411: dread d1/d22/f3c [0,4194304] 0 2024-09-21T04:15:14.308 INFO:tasks.workunit.client.0.smithi110.stdout:6/332: unlink d4/d12/d17/d33/l39 0 2024-09-21T04:15:14.310 INFO:tasks.workunit.client.0.smithi110.stdout:6/333: read d4/f7 [3465720,55015] 0 2024-09-21T04:15:14.311 INFO:tasks.workunit.client.1.smithi148.stdout:5/330: truncate d3/d14/d1f/d26/d2e/d56/f3f 764990 0 2024-09-21T04:15:14.314 INFO:tasks.workunit.client.1.smithi148.stdout:6/277: mkdir d0/d1a/d23/d16/d4f 0 2024-09-21T04:15:14.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/334: dread d4/d12/d17/d2e/d40/f2f [0,4194304] 0 2024-09-21T04:15:14.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/335: creat d4/f6a x:0 0 0 2024-09-21T04:15:14.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/336: write d4/d12/d17/f2a [4306461,119339] 0 2024-09-21T04:15:14.316 INFO:tasks.workunit.client.1.smithi148.stdout:1/317: getdents db/d24/d2e 0 2024-09-21T04:15:14.316 INFO:tasks.workunit.client.1.smithi148.stdout:1/318: chown db/dd/c38 2662880 1 2024-09-21T04:15:14.319 INFO:tasks.workunit.client.1.smithi148.stdout:3/302: symlink d3/d5/d6/d12/da/d23/d28/l5e 0 2024-09-21T04:15:14.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/342: dwrite d5/d6/f62 [0,4194304] 0 2024-09-21T04:15:14.322 INFO:tasks.workunit.client.1.smithi148.stdout:7/364: rename d5/d14/l5e to d5/dc/l73 0 2024-09-21T04:15:14.323 INFO:tasks.workunit.client.0.smithi110.stdout:9/286: rename d7/d10/d24/d3c/d35/c51 to d7/d10/d24/d3c/d14/d3e/d59/c5f 0 2024-09-21T04:15:14.324 INFO:tasks.workunit.client.1.smithi148.stdout:1/319: truncate db/f1b 3430049 0 2024-09-21T04:15:14.324 INFO:tasks.workunit.client.1.smithi148.stdout:1/320: readlink db/l49 0 2024-09-21T04:15:14.324 INFO:tasks.workunit.client.0.smithi110.stdout:4/535: getdents d17/d1e/d77 0 2024-09-21T04:15:14.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/343: symlink d5/d6/d7/d18/l6a 0 2024-09-21T04:15:14.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/344: chown d5/c54 10 1 2024-09-21T04:15:14.331 INFO:tasks.workunit.client.0.smithi110.stdout:1/641: truncate d2/d9/d49/f5b 1112308 0 2024-09-21T04:15:14.333 INFO:tasks.workunit.client.1.smithi148.stdout:7/365: symlink d5/l74 0 2024-09-21T04:15:14.335 INFO:tasks.workunit.client.1.smithi148.stdout:1/321: mknod db/d24/d2e/c65 0 2024-09-21T04:15:14.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/331: dwrite d3/f36 [0,4194304] 0 2024-09-21T04:15:14.336 INFO:tasks.workunit.client.0.smithi110.stdout:0/646: symlink d4/d8b/lc4 0 2024-09-21T04:15:14.336 INFO:tasks.workunit.client.0.smithi110.stdout:0/647: readlink d4/d1c/d62/l68 0 2024-09-21T04:15:14.340 INFO:tasks.workunit.client.0.smithi110.stdout:3/358: creat d0/d12/f82 x:0 0 0 2024-09-21T04:15:14.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/648: dread d4/d47/f40 [0,4194304] 0 2024-09-21T04:15:14.343 INFO:tasks.workunit.client.0.smithi110.stdout:5/412: unlink d1/d49/d69/l7f 0 2024-09-21T04:15:14.346 INFO:tasks.workunit.client.1.smithi148.stdout:3/303: rmdir d3/d5 39 2024-09-21T04:15:14.348 INFO:tasks.workunit.client.0.smithi110.stdout:6/337: mknod d4/d12/d17/d2e/d40/d65/c6b 0 2024-09-21T04:15:14.348 INFO:tasks.workunit.client.0.smithi110.stdout:6/338: chown d4/f3d 2 1 2024-09-21T04:15:14.348 INFO:tasks.workunit.client.0.smithi110.stdout:6/339: fdatasync d4/d12/d16/d66/f67 0 2024-09-21T04:15:14.350 INFO:tasks.workunit.client.0.smithi110.stdout:9/287: creat d7/d10/d24/d3c/d1f/d43/f60 x:0 0 0 2024-09-21T04:15:14.350 INFO:tasks.workunit.client.0.smithi110.stdout:9/288: read f2 [560927,26115] 0 2024-09-21T04:15:14.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/366: mknod d5/d8/d64/c75 0 2024-09-21T04:15:14.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/367: chown d5/d14/d25/c43 135029895 1 2024-09-21T04:15:14.353 INFO:tasks.workunit.client.0.smithi110.stdout:4/536: mkdir d17/d34/d3d/d57/db9 0 2024-09-21T04:15:14.356 INFO:tasks.workunit.client.1.smithi148.stdout:1/322: rmdir db/d46 39 2024-09-21T04:15:14.357 INFO:tasks.workunit.client.0.smithi110.stdout:6/340: dread d4/d12/d17/d33/f64 [0,4194304] 0 2024-09-21T04:15:14.357 INFO:tasks.workunit.client.0.smithi110.stdout:6/341: creat d4/f6c x:0 0 0 2024-09-21T04:15:14.358 INFO:tasks.workunit.client.0.smithi110.stdout:1/642: truncate d2/d9/d23/f38 1356722 0 2024-09-21T04:15:14.358 INFO:tasks.workunit.client.0.smithi110.stdout:1/643: creat d2/d9/d49/d7a/da8/fd3 x:0 0 0 2024-09-21T04:15:14.360 INFO:tasks.workunit.client.0.smithi110.stdout:5/413: mknod d1/d39/c94 0 2024-09-21T04:15:14.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/332: mkdir d3/d34/d78 0 2024-09-21T04:15:14.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/333: stat d3/f7 0 2024-09-21T04:15:14.366 INFO:tasks.workunit.client.0.smithi110.stdout:5/414: dread d1/d22/f27 [0,4194304] 0 2024-09-21T04:15:14.366 INFO:tasks.workunit.client.0.smithi110.stdout:5/415: dread - d1/d49/d4d/d1e/f81 zero size 2024-09-21T04:15:14.368 INFO:tasks.workunit.client.1.smithi148.stdout:0/345: getdents d5/d6/d2a/d3f 0 2024-09-21T04:15:14.368 INFO:tasks.workunit.client.1.smithi148.stdout:0/346: write d5/d6/d7/d18/f32 [5035969,73933] 0 2024-09-21T04:15:14.368 INFO:tasks.workunit.client.0.smithi110.stdout:9/289: read d7/d10/d24/d3c/d14/f54 [3921512,70746] 0 2024-09-21T04:15:14.368 INFO:tasks.workunit.client.0.smithi110.stdout:1/644: dread d2/d9/d26/f8b [0,4194304] 0 2024-09-21T04:15:14.371 INFO:tasks.workunit.client.0.smithi110.stdout:6/342: rmdir d4/d12/d17/d2e/d38 39 2024-09-21T04:15:14.373 INFO:tasks.workunit.client.0.smithi110.stdout:6/343: dread f3 [0,4194304] 0 2024-09-21T04:15:14.373 INFO:tasks.workunit.client.1.smithi148.stdout:3/304: unlink d3/d5/d6/d2b/l40 0 2024-09-21T04:15:14.376 INFO:tasks.workunit.client.0.smithi110.stdout:9/290: rename d7/d10/c2f to d7/d10/d24/d3c/c61 0 2024-09-21T04:15:14.376 INFO:tasks.workunit.client.0.smithi110.stdout:3/359: dwrite d0/d12/d16/d1a/f3b [0,4194304] 0 2024-09-21T04:15:14.376 INFO:tasks.workunit.client.0.smithi110.stdout:3/360: read d0/d12/d2a/f4d [18848,58976] 0 2024-09-21T04:15:14.376 INFO:tasks.workunit.client.0.smithi110.stdout:0/649: dwrite d4/d47/d2c/d3a/f9f [0,4194304] 0 2024-09-21T04:15:14.377 INFO:tasks.workunit.client.0.smithi110.stdout:0/650: creat d4/dd/d53/fc5 x:0 0 0 2024-09-21T04:15:14.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/331: rename d4/d12/d2c/d2d/d44 to d4/d12/d16/d29/d61 0 2024-09-21T04:15:14.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/332: stat d4/d12/d2c/d2d/l48 0 2024-09-21T04:15:14.378 INFO:tasks.workunit.client.0.smithi110.stdout:4/537: dwrite d17/d34/d3d/d20/d43/d51/fa1 [4194304,4194304] 0 2024-09-21T04:15:14.379 INFO:tasks.workunit.client.0.smithi110.stdout:4/538: read d17/f27 [4925522,49245] 0 2024-09-21T04:15:14.379 INFO:tasks.workunit.client.0.smithi110.stdout:1/645: creat d2/d9/d49/d4f/d53/fd4 x:0 0 0 2024-09-21T04:15:14.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/368: mknod d5/dc/d28/d4a/c76 0 2024-09-21T04:15:14.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/323: creat db/d46/f66 x:0 0 0 2024-09-21T04:15:14.382 INFO:tasks.workunit.client.0.smithi110.stdout:0/651: dread d4/dd/d10/d29/f6b [0,4194304] 0 2024-09-21T04:15:14.383 INFO:tasks.workunit.client.0.smithi110.stdout:6/344: mkdir d4/d12/d16/d6d 0 2024-09-21T04:15:14.383 INFO:tasks.workunit.client.0.smithi110.stdout:6/345: write d4/de/f10 [685195,22299] 0 2024-09-21T04:15:14.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/334: truncate d3/f9 4870985 0 2024-09-21T04:15:14.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/335: readlink d3/d14/d1a/l35 0 2024-09-21T04:15:14.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/336: getdents d3/d34/d78 0 2024-09-21T04:15:14.386 INFO:tasks.workunit.client.0.smithi110.stdout:9/291: rename d7/d10/d24/d3c/ld to d7/d10/d24/d3c/d1f/d43/l62 0 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/292: creat d7/d10/d24/d3c/d14/d3e/f63 x:0 0 0 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/293: chown d7/d10/d24/d3c/d1f/l48 818540 1 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/294: chown d7/d10/d24/d3c/d14/d3e/f63 1727969 1 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/295: fsync d7/d10/d24/d3c/d35/f5e 0 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/296: chown d7/d10/d24/d3c/lc 15810 1 2024-09-21T04:15:14.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/297: chown d7/d10/d24/d3c/d4f/l50 23 1 2024-09-21T04:15:14.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/416: dwrite d1/d39/f88 [0,4194304] 0 2024-09-21T04:15:14.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/417: getdents d1/d7/d58/d72 0 2024-09-21T04:15:14.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/333: write d4/d12/d16/d29/f59 [543528,48433] 0 2024-09-21T04:15:14.388 INFO:tasks.workunit.client.1.smithi148.stdout:3/305: mkdir d3/d5/d6/d12/d15/d5f 0 2024-09-21T04:15:14.390 INFO:tasks.workunit.client.1.smithi148.stdout:2/419: rename d0/f11 to d0/d9/dc/d13/f80 0 2024-09-21T04:15:14.391 INFO:tasks.workunit.client.0.smithi110.stdout:3/361: symlink d0/d12/d3c/d5a/l83 0 2024-09-21T04:15:14.395 INFO:tasks.workunit.client.1.smithi148.stdout:7/369: truncate d5/dc/d28/f49 1574403 0 2024-09-21T04:15:14.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/336: sync 2024-09-21T04:15:14.395 INFO:tasks.workunit.client.1.smithi148.stdout:9/290: sync 2024-09-21T04:15:14.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/337: mknod d3/d34/c79 0 2024-09-21T04:15:14.402 INFO:tasks.workunit.client.1.smithi148.stdout:4/334: creat d4/d12/d16/f62 x:0 0 0 2024-09-21T04:15:14.404 INFO:tasks.workunit.client.1.smithi148.stdout:3/306: rmdir d3/d5/d6/d12/da/d23 39 2024-09-21T04:15:14.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/278: rename d0/d1a/d23/dc/f3e to d0/d34/d45/f50 0 2024-09-21T04:15:14.407 INFO:tasks.workunit.client.1.smithi148.stdout:6/279: stat d0/d1a/d23/d16/f2c 0 2024-09-21T04:15:14.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/420: truncate d0/d9/d38/f4f 298393 0 2024-09-21T04:15:14.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/421: stat d0/d9 0 2024-09-21T04:15:14.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/422: chown d0/d9/dc/f12 30276671 1 2024-09-21T04:15:14.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/423: read d0/d9/f17 [2462209,14291] 0 2024-09-21T04:15:14.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/424: fdatasync d0/f14 0 2024-09-21T04:15:14.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/370: mknod d5/d8/d64/c77 0 2024-09-21T04:15:14.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/539: symlink d17/d34/d3d/d20/d43/d51/d7a/d8d/lba 0 2024-09-21T04:15:14.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/540: fsync d17/d34/d3d/f41 0 2024-09-21T04:15:14.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/337: mkdir dc/d13/d6a 0 2024-09-21T04:15:14.416 INFO:tasks.workunit.client.1.smithi148.stdout:2/425: write d0/f23 [2804422,41743] 0 2024-09-21T04:15:14.417 INFO:tasks.workunit.client.1.smithi148.stdout:1/324: rmdir db/d24/d2a 39 2024-09-21T04:15:14.418 INFO:tasks.workunit.client.0.smithi110.stdout:1/646: getdents d2/d9/d23/d4e/d57 0 2024-09-21T04:15:14.418 INFO:tasks.workunit.client.0.smithi110.stdout:1/647: fdatasync d2/d9/d26/f31 0 2024-09-21T04:15:14.420 INFO:tasks.workunit.client.1.smithi148.stdout:5/338: mknod d3/d14/d1f/d72/c7a 0 2024-09-21T04:15:14.422 INFO:tasks.workunit.client.1.smithi148.stdout:4/335: creat d4/d12/f63 x:0 0 0 2024-09-21T04:15:14.424 INFO:tasks.workunit.client.1.smithi148.stdout:9/291: dwrite d5/d39/f6f [4194304,4194304] 0 2024-09-21T04:15:14.425 INFO:tasks.workunit.client.0.smithi110.stdout:0/652: mknod d4/d47/cc6 0 2024-09-21T04:15:14.425 INFO:tasks.workunit.client.0.smithi110.stdout:0/653: chown d4/dd/d10/d29/f6b 16676 1 2024-09-21T04:15:14.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/307: dwrite d3/d5/f35 [0,4194304] 0 2024-09-21T04:15:14.428 INFO:tasks.workunit.client.1.smithi148.stdout:6/280: dwrite d0/d1a/d23/f19 [0,4194304] 0 2024-09-21T04:15:14.429 INFO:tasks.workunit.client.1.smithi148.stdout:8/338: rename dc/d3e/l45 to dc/d13/l6b 0 2024-09-21T04:15:14.431 INFO:tasks.workunit.client.0.smithi110.stdout:6/346: mknod d4/d12/d16/d48/d63/c6e 0 2024-09-21T04:15:14.432 INFO:tasks.workunit.client.1.smithi148.stdout:2/426: symlink d0/d9/d16/l81 0 2024-09-21T04:15:14.432 INFO:tasks.workunit.client.1.smithi148.stdout:2/427: chown d0/d9/dc/d13/d25/f2a 892 1 2024-09-21T04:15:14.435 INFO:tasks.workunit.client.1.smithi148.stdout:5/339: rename d3/d14/d1f/d26/d2e/d56/ce to d3/d14/d1f/d26/d2e/d56/c7b 0 2024-09-21T04:15:14.436 INFO:tasks.workunit.client.0.smithi110.stdout:9/298: creat d7/d10/d24/d3c/d4f/f64 x:0 0 0 2024-09-21T04:15:14.436 INFO:tasks.workunit.client.0.smithi110.stdout:9/299: truncate d7/d10/d24/d3c/d4f/f64 554234 0 2024-09-21T04:15:14.438 INFO:tasks.workunit.client.1.smithi148.stdout:3/308: dread d3/d5/d6/d12/da/f1d [0,4194304] 0 2024-09-21T04:15:14.438 INFO:tasks.workunit.client.1.smithi148.stdout:3/309: dread - d3/f49 zero size 2024-09-21T04:15:14.438 INFO:tasks.workunit.client.1.smithi148.stdout:4/336: symlink d4/d23/d2a/l64 0 2024-09-21T04:15:14.439 INFO:tasks.workunit.client.1.smithi148.stdout:1/325: dwrite db/d46/f57 [0,4194304] 0 2024-09-21T04:15:14.440 INFO:tasks.workunit.client.1.smithi148.stdout:7/371: getdents d5/d8 0 2024-09-21T04:15:14.440 INFO:tasks.workunit.client.1.smithi148.stdout:7/372: dread - d5/dc/d28/f70 zero size 2024-09-21T04:15:14.444 INFO:tasks.workunit.client.0.smithi110.stdout:5/418: creat d1/d49/d4d/d1e/d5a/d64/d7b/f95 x:0 0 0 2024-09-21T04:15:14.444 INFO:tasks.workunit.client.0.smithi110.stdout:5/419: dread - d1/d49/d4d/d1e/d4a/f83 zero size 2024-09-21T04:15:14.446 INFO:tasks.workunit.client.0.smithi110.stdout:3/362: creat d0/d12/d3c/d5a/d70/f84 x:0 0 0 2024-09-21T04:15:14.446 INFO:tasks.workunit.client.0.smithi110.stdout:3/363: readlink d0/l57 0 2024-09-21T04:15:14.446 INFO:tasks.workunit.client.0.smithi110.stdout:3/364: creat d0/d12/d16/d1a/d5c/f85 x:0 0 0 2024-09-21T04:15:14.449 INFO:tasks.workunit.client.0.smithi110.stdout:4/541: truncate d17/d34/d3d/d20/d43/f92 489227 0 2024-09-21T04:15:14.451 INFO:tasks.workunit.client.0.smithi110.stdout:1/648: unlink d2/d9/d23/d3b/l79 0 2024-09-21T04:15:14.453 INFO:tasks.workunit.client.1.smithi148.stdout:6/281: unlink d0/d1a/d23/dc/c1d 0 2024-09-21T04:15:14.453 INFO:tasks.workunit.client.1.smithi148.stdout:6/282: chown d0/d8/d18/d4e/f4a 4 1 2024-09-21T04:15:14.453 INFO:tasks.workunit.client.0.smithi110.stdout:1/649: dread d2/d9/d26/d3d/d8f/f90 [0,4194304] 0 2024-09-21T04:15:14.454 INFO:tasks.workunit.client.1.smithi148.stdout:8/339: symlink dc/d43/l6c 0 2024-09-21T04:15:14.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/340: read dc/f31 [635897,52690] 0 2024-09-21T04:15:14.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/654: mknod d4/d47/d2c/cc7 0 2024-09-21T04:15:14.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/655: readlink d4/dd/d10/d29/l95 0 2024-09-21T04:15:14.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/656: truncate d4/dd/d53/d96/fb0 849445 0 2024-09-21T04:15:14.459 INFO:tasks.workunit.client.0.smithi110.stdout:6/347: symlink d4/d12/l6f 0 2024-09-21T04:15:14.460 INFO:tasks.workunit.client.1.smithi148.stdout:2/428: mkdir d0/d9/dc/d13/d25/d4e/d82 0 2024-09-21T04:15:14.460 INFO:tasks.workunit.client.1.smithi148.stdout:2/429: chown d0/d9/dc/d13/c63 943 1 2024-09-21T04:15:14.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/340: mknod d3/d34/d78/c7c 0 2024-09-21T04:15:14.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/341: chown d3/d14/d1f/d26/l38 411024591 1 2024-09-21T04:15:14.465 INFO:tasks.workunit.client.0.smithi110.stdout:9/300: symlink d7/d10/d24/d3c/d14/d3e/d59/l65 0 2024-09-21T04:15:14.465 INFO:tasks.workunit.client.0.smithi110.stdout:9/301: read - d7/d10/d24/f38 zero size 2024-09-21T04:15:14.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/310: mkdir d3/d5/d5c/d60 0 2024-09-21T04:15:14.468 INFO:tasks.workunit.client.1.smithi148.stdout:4/337: truncate d4/d23/f2b 1220994 0 2024-09-21T04:15:14.469 INFO:tasks.workunit.client.1.smithi148.stdout:1/326: symlink db/dd/d21/l67 0 2024-09-21T04:15:14.469 INFO:tasks.workunit.client.0.smithi110.stdout:4/542: unlink d17/d34/l78 0 2024-09-21T04:15:14.471 INFO:tasks.workunit.client.0.smithi110.stdout:5/420: dwrite d1/d49/d4d/d1e/f5f [0,4194304] 0 2024-09-21T04:15:14.471 INFO:tasks.workunit.client.0.smithi110.stdout:5/421: chown d1/d39/f79 11810085 1 2024-09-21T04:15:14.471 INFO:tasks.workunit.client.0.smithi110.stdout:5/422: write d1/d39/f50 [3768876,47238] 0 2024-09-21T04:15:14.472 INFO:tasks.workunit.client.1.smithi148.stdout:7/373: creat d5/d14/f78 x:0 0 0 2024-09-21T04:15:14.472 INFO:tasks.workunit.client.1.smithi148.stdout:7/374: chown d5/dc/d24/c30 198064 1 2024-09-21T04:15:14.472 INFO:tasks.workunit.client.0.smithi110.stdout:9/302: symlink d7/d10/d24/d3c/d14/d31/l66 0 2024-09-21T04:15:14.473 INFO:tasks.workunit.client.0.smithi110.stdout:4/543: symlink d17/d34/d3d/d20/d43/d51/d7a/d8d/lbb 0 2024-09-21T04:15:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/338: dread d4/fe [0,4194304] 0 2024-09-21T04:15:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/339: chown d4/f51 7045915 1 2024-09-21T04:15:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/340: chown d4/d3a/f53 45784 1 2024-09-21T04:15:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/341: chown d4/d12/f41 665903448 1 2024-09-21T04:15:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/342: fdatasync d4/d12/d2c/d2d/f52 0 2024-09-21T04:15:14.478 INFO:tasks.workunit.client.1.smithi148.stdout:6/283: creat d0/d34/d45/f51 x:0 0 0 2024-09-21T04:15:14.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/348: dread d4/f13 [0,4194304] 0 2024-09-21T04:15:14.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/349: readlink d4/d12/d17/d2e/l58 0 2024-09-21T04:15:14.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/350: truncate d4/d12/d16/f56 216130 0 2024-09-21T04:15:14.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/351: write d4/d12/d16/d66/f67 [466813,26354] 0 2024-09-21T04:15:14.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/352: fsync d4/d12/f34 0 2024-09-21T04:15:14.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/431: sync 2024-09-21T04:15:14.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/432: dread - d0/d30/d34/d3e/d32/f73 zero size 2024-09-21T04:15:14.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/341: rename dc/d13/d28/f2b to dc/d2a/f6d 0 2024-09-21T04:15:14.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/423: mknod d1/d49/d4d/c96 0 2024-09-21T04:15:14.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/424: creat d1/d22/f97 x:0 0 0 2024-09-21T04:15:14.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/425: creat d1/d22/d51/f98 x:0 0 0 2024-09-21T04:15:14.484 INFO:tasks.workunit.client.1.smithi148.stdout:2/430: symlink d0/d9/dc/d13/d25/d2d/l83 0 2024-09-21T04:15:14.484 INFO:tasks.workunit.client.1.smithi148.stdout:2/431: write d0/ff [1590080,27574] 0 2024-09-21T04:15:14.484 INFO:tasks.workunit.client.1.smithi148.stdout:2/432: creat d0/d2f/f84 x:0 0 0 2024-09-21T04:15:14.485 INFO:tasks.workunit.client.0.smithi110.stdout:9/303: symlink d7/d10/d24/d3c/d14/l67 0 2024-09-21T04:15:14.486 INFO:tasks.workunit.client.1.smithi148.stdout:1/327: symlink db/dd/l68 0 2024-09-21T04:15:14.486 INFO:tasks.workunit.client.0.smithi110.stdout:1/650: dwrite d2/d9/d26/d3d/f6f [0,4194304] 0 2024-09-21T04:15:14.486 INFO:tasks.workunit.client.0.smithi110.stdout:1/651: chown d2/d9/d23/f40 40943 1 2024-09-21T04:15:14.487 INFO:tasks.workunit.client.0.smithi110.stdout:4/544: mknod d17/d1e/d7b/cbc 0 2024-09-21T04:15:14.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/367: sync 2024-09-21T04:15:14.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/368: fdatasync dd/f57 0 2024-09-21T04:15:14.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/369: chown dd/d14/f46 215943473 1 2024-09-21T04:15:14.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/375: mkdir d5/d14/d25/d79 0 2024-09-21T04:15:14.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/376: fsync d5/dc/f3e 0 2024-09-21T04:15:14.489 INFO:tasks.workunit.client.0.smithi110.stdout:2/205: sync 2024-09-21T04:15:14.491 INFO:tasks.workunit.client.0.smithi110.stdout:8/433: creat d0/d7/d13/d4b/d91/f95 x:0 0 0 2024-09-21T04:15:14.492 INFO:tasks.workunit.client.0.smithi110.stdout:2/206: dread d2/d1a/d1f/d29/d19/d36/f2b [0,4194304] 0 2024-09-21T04:15:14.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/657: dwrite d4/d47/f23 [0,4194304] 0 2024-09-21T04:15:14.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/658: stat d4/dd/f32 0 2024-09-21T04:15:14.496 INFO:tasks.workunit.client.1.smithi148.stdout:5/342: dwrite d3/d14/d1f/d26/d2e/d3d/f67 [0,4194304] 0 2024-09-21T04:15:14.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/207: dread d2/f16 [0,4194304] 0 2024-09-21T04:15:14.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/208: truncate d2/d1a/d1f/d29/d19/d36/f3a 234050 0 2024-09-21T04:15:14.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/209: creat d2/f3b x:0 0 0 2024-09-21T04:15:14.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/210: chown d2/d1a/d1f/d29/d19/d36/f24 0 1 2024-09-21T04:15:14.499 INFO:tasks.workunit.client.0.smithi110.stdout:5/426: mknod d1/d49/d4d/d1e/d5a/c99 0 2024-09-21T04:15:14.503 INFO:tasks.workunit.client.1.smithi148.stdout:1/328: dread db/d46/f57 [0,4194304] 0 2024-09-21T04:15:14.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/329: write db/d46/f53 [312180,64120] 0 2024-09-21T04:15:14.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/330: read - db/d24/d2a/f42 zero size 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.0.smithi110.stdout:9/304: mkdir d7/d10/d24/d3c/d1f/d68 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.0.smithi110.stdout:9/305: fdatasync d7/d10/d24/d3c/d11/f4e 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/370: write dd/f2e [3940011,25183] 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/371: read dd/d1c/f38 [125471,20181] 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.0.smithi110.stdout:4/545: truncate d17/d34/d3d/d20/f80 521714 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/284: creat d0/d1a/d23/d16/f52 x:0 0 0 2024-09-21T04:15:14.507 INFO:tasks.workunit.client.1.smithi148.stdout:8/342: mkdir dc/d13/d39/d6e 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/343: chown dc/c11 5853005 1 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/344: chown dc/f46 1019763649 1 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/345: write dc/d13/d39/f66 [672229,112337] 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/311: dwrite d3/d5/d6/d12/da/d23/d28/d46/f5b [0,4194304] 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/433: mkdir d0/d9/d16/d85 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/434: truncate d0/d9/dc/d13/f21 370803 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/377: mknod d5/d8/d64/c7a 0 2024-09-21T04:15:14.508 INFO:tasks.workunit.client.0.smithi110.stdout:0/659: dread d4/dd/d53/fa2 [0,4194304] 0 2024-09-21T04:15:14.509 INFO:tasks.workunit.client.0.smithi110.stdout:8/434: mknod d0/d30/d34/c96 0 2024-09-21T04:15:14.509 INFO:tasks.workunit.client.1.smithi148.stdout:2/435: dread d0/d9/dc/d13/f1a [0,4194304] 0 2024-09-21T04:15:14.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/343: link d4/fe d4/d12/f65 0 2024-09-21T04:15:14.514 INFO:tasks.workunit.client.0.smithi110.stdout:5/427: unlink d1/dc/cd 0 2024-09-21T04:15:14.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/306: rename d7/d10/c56 to d7/d10/d24/d3c/d14/d3e/c69 0 2024-09-21T04:15:14.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/307: write d7/d10/d24/f2a [5087780,19732] 0 2024-09-21T04:15:14.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/308: creat d7/d10/d24/d3c/d14/d31/f6a x:0 0 0 2024-09-21T04:15:14.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/309: read - d7/d10/d24/d3c/d14/d3e/f63 zero size 2024-09-21T04:15:14.520 INFO:tasks.workunit.client.0.smithi110.stdout:5/428: write d1/dc/f1a [2196733,402] 0 2024-09-21T04:15:14.521 INFO:tasks.workunit.client.1.smithi148.stdout:5/343: link d3/d14/d1f/l65 d3/d14/d1f/d26/d2e/l7d 0 2024-09-21T04:15:14.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/435: dread d0/d30/d54/d6b/f75 [0,4194304] 0 2024-09-21T04:15:14.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/436: fdatasync d0/d30/f31 0 2024-09-21T04:15:14.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/437: truncate d0/d7/d13/d4b/d91/f95 62393 0 2024-09-21T04:15:14.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/438: stat d0/d30/d34/d3e/d32 0 2024-09-21T04:15:14.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/285: rename d0/d1a/d4b to d0/d1a/d23/d53 0 2024-09-21T04:15:14.525 INFO:tasks.workunit.client.0.smithi110.stdout:4/546: mkdir d17/d3e/dbd 0 2024-09-21T04:15:14.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/344: read d4/d12/f41 [929367,82932] 0 2024-09-21T04:15:14.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/345: stat d4 0 2024-09-21T04:15:14.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/344: dread d3/d14/d1f/d26/f66 [0,4194304] 0 2024-09-21T04:15:14.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/345: creat d3/d14/d1f/f7e x:0 0 0 2024-09-21T04:15:14.527 INFO:tasks.workunit.client.0.smithi110.stdout:7/372: rename dd/d14/d17/c30 to dd/d1c/d39/d6d/c72 0 2024-09-21T04:15:14.530 INFO:tasks.workunit.client.0.smithi110.stdout:6/353: dwrite d4/d12/d16/d66/f67 [0,4194304] 0 2024-09-21T04:15:14.530 INFO:tasks.workunit.client.0.smithi110.stdout:6/354: chown d4/d12/d17/d2e 14 1 2024-09-21T04:15:14.531 INFO:tasks.workunit.client.0.smithi110.stdout:1/652: dwrite d2/d9/d49/f5b [0,4194304] 0 2024-09-21T04:15:14.531 INFO:tasks.workunit.client.0.smithi110.stdout:1/653: fsync d2/d9/d26/f31 0 2024-09-21T04:15:14.531 INFO:tasks.workunit.client.1.smithi148.stdout:1/331: dwrite db/f58 [0,4194304] 0 2024-09-21T04:15:14.532 INFO:tasks.workunit.client.0.smithi110.stdout:5/429: symlink d1/d49/d4d/d1e/l9a 0 2024-09-21T04:15:14.532 INFO:tasks.workunit.client.0.smithi110.stdout:5/430: fsync d1/d7/d18/d3f/f93 0 2024-09-21T04:15:14.532 INFO:tasks.workunit.client.0.smithi110.stdout:5/431: truncate d1/d7/f2b 2027524 0 2024-09-21T04:15:14.533 INFO:tasks.workunit.client.0.smithi110.stdout:8/439: link d0/d30/d34/d3e/d5d/f62 d0/d7/d13/f97 0 2024-09-21T04:15:14.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/346: creat dc/d13/d39/d6e/f6f x:0 0 0 2024-09-21T04:15:14.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/347: chown dc/c26 260463905 1 2024-09-21T04:15:14.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/348: fsync dc/d13/d28/f48 0 2024-09-21T04:15:14.536 INFO:tasks.workunit.client.0.smithi110.stdout:4/547: creat d17/d1e/d21/fbe x:0 0 0 2024-09-21T04:15:14.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/312: mknod d3/d5/c61 0 2024-09-21T04:15:14.538 INFO:tasks.workunit.client.0.smithi110.stdout:9/310: rename f6 to d7/d10/d24/d3c/d1f/d68/f6b 0 2024-09-21T04:15:14.539 INFO:tasks.workunit.client.1.smithi148.stdout:2/436: symlink d0/d9/d16/d55/l86 0 2024-09-21T04:15:14.540 INFO:tasks.workunit.client.0.smithi110.stdout:7/373: creat dd/d14/f73 x:0 0 0 2024-09-21T04:15:14.543 INFO:tasks.workunit.client.1.smithi148.stdout:7/378: rename d5/dc/d24 to d5/d14/d25/d40/d7b 0 2024-09-21T04:15:14.545 INFO:tasks.workunit.client.0.smithi110.stdout:6/355: mknod d4/d12/d16/c70 0 2024-09-21T04:15:14.546 INFO:tasks.workunit.client.1.smithi148.stdout:6/286: truncate d0/d8/d18/d4e/f4a 4700289 0 2024-09-21T04:15:14.546 INFO:tasks.workunit.client.1.smithi148.stdout:6/287: readlink d0/d1a/d23/l2b 0 2024-09-21T04:15:14.547 INFO:tasks.workunit.client.1.smithi148.stdout:6/288: write d0/d1a/d23/f3d [958284,47817] 0 2024-09-21T04:15:14.547 INFO:tasks.workunit.client.1.smithi148.stdout:2/437: dread d0/d9/dc/fd [0,4194304] 0 2024-09-21T04:15:14.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/346: unlink d4/d23/f37 0 2024-09-21T04:15:14.550 INFO:tasks.workunit.client.0.smithi110.stdout:5/432: mknod d1/d49/d6f/c9b 0 2024-09-21T04:15:14.550 INFO:tasks.workunit.client.0.smithi110.stdout:5/433: read - d1/d49/d4d/d1e/d5a/f7a zero size 2024-09-21T04:15:14.550 INFO:tasks.workunit.client.0.smithi110.stdout:5/434: write d1/d7/d58/f60 [267198,15901] 0 2024-09-21T04:15:14.551 INFO:tasks.workunit.client.0.smithi110.stdout:5/435: write d1/f4c [497135,589] 0 2024-09-21T04:15:14.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/346: truncate d3/f36 160723 0 2024-09-21T04:15:14.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/347: truncate d3/d14/d1f/f50 727231 0 2024-09-21T04:15:14.560 INFO:tasks.workunit.client.0.smithi110.stdout:7/374: dwrite dd/f2e [4194304,4194304] 0 2024-09-21T04:15:14.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/349: creat dc/d3e/f70 x:0 0 0 2024-09-21T04:15:14.561 INFO:tasks.workunit.client.0.smithi110.stdout:7/375: dread dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:14.564 INFO:tasks.workunit.client.1.smithi148.stdout:3/313: rmdir d3/d5/d6/d12/d15/d5f 0 2024-09-21T04:15:14.564 INFO:tasks.workunit.client.1.smithi148.stdout:3/314: dread - d3/f49 zero size 2024-09-21T04:15:14.569 INFO:tasks.workunit.client.1.smithi148.stdout:7/379: rmdir d5/d8 39 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:6/289: creat d0/d1a/d23/d16/d4f/f54 x:0 0 0 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:6/290: fdatasync d0/f20 0 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:6/291: write d0/d8/f29 [4684077,116901] 0 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:0/347: sync 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/292: sync 2024-09-21T04:15:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/293: fdatasync d5/d2b/d32/f5a 0 2024-09-21T04:15:14.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/294: dread d5/f1b [0,4194304] 0 2024-09-21T04:15:14.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/295: chown d5/d60/d28/d2f 2 1 2024-09-21T04:15:14.573 INFO:tasks.workunit.client.1.smithi148.stdout:2/438: truncate d0/d9/f39 2023749 0 2024-09-21T04:15:14.574 INFO:tasks.workunit.client.0.smithi110.stdout:5/436: dwrite d1/d49/d4d/d1e/d5a/d64/f80 [0,4194304] 0 2024-09-21T04:15:14.575 INFO:tasks.workunit.client.0.smithi110.stdout:8/440: creat d0/d30/d54/f98 x:0 0 0 2024-09-21T04:15:14.576 INFO:tasks.workunit.client.0.smithi110.stdout:5/437: dread d1/f4c [0,4194304] 0 2024-09-21T04:15:14.576 INFO:tasks.workunit.client.1.smithi148.stdout:4/347: rmdir d4/d12/d2c 39 2024-09-21T04:15:14.577 INFO:tasks.workunit.client.0.smithi110.stdout:4/548: creat d17/d1e/fbf x:0 0 0 2024-09-21T04:15:14.580 INFO:tasks.workunit.client.1.smithi148.stdout:5/348: rename d3/d14/d1f/d26/f66 to d3/d37/f7f 0 2024-09-21T04:15:14.580 INFO:tasks.workunit.client.1.smithi148.stdout:2/439: write d0/d9/dc/f24 [1542874,109133] 0 2024-09-21T04:15:14.582 INFO:tasks.workunit.client.1.smithi148.stdout:3/315: link d3/d5/d6/d12/da/d52/l2a d3/d5/d6/d12/d43/l62 0 2024-09-21T04:15:14.582 INFO:tasks.workunit.client.1.smithi148.stdout:3/316: chown d3/d5 27 1 2024-09-21T04:15:14.582 INFO:tasks.workunit.client.1.smithi148.stdout:3/317: write d3/d5/d6/d12/da/d52/f4a [4846409,42059] 0 2024-09-21T04:15:14.583 INFO:tasks.workunit.client.0.smithi110.stdout:1/654: rename d2/d9/d23/d4e/d57/c73 to d2/d9/d23/d3b/cd5 0 2024-09-21T04:15:14.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/348: rmdir d5/d6/d2a 39 2024-09-21T04:15:14.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/349: read d5/d6/d3e/f26 [1420320,62976] 0 2024-09-21T04:15:14.584 INFO:tasks.workunit.client.0.smithi110.stdout:5/438: dread d1/d49/d4d/f1b [0,4194304] 0 2024-09-21T04:15:14.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/311: unlink d7/d10/d24/d3c/d1f/d25/c1a 0 2024-09-21T04:15:14.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/312: creat d7/d10/d24/d3c/d1f/d43/f6c x:0 0 0 2024-09-21T04:15:14.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/296: creat d5/d1e/d37/d46/f72 x:0 0 0 2024-09-21T04:15:14.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/297: chown d5/d1e/l68 6 1 2024-09-21T04:15:14.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/298: truncate d5/d1f/f4b 625902 0 2024-09-21T04:15:14.591 INFO:tasks.workunit.client.1.smithi148.stdout:4/348: truncate d4/f11 2433946 0 2024-09-21T04:15:14.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/350: rename dc/d13/d3c/f61 to dc/d2a/d42/f71 0 2024-09-21T04:15:14.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/351: chown dc/l4d 37497 1 2024-09-21T04:15:14.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/352: stat dc/d13/d3c/f4b 0 2024-09-21T04:15:14.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/353: creat dc/d13/d3c/f72 x:0 0 0 2024-09-21T04:15:14.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/354: fsync dc/f21 0 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:3/318: dread d3/d1f/f47 [0,4194304] 0 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/299: mkdir d5/d60/d28/d73 0 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/300: chown d5/f23 98670 1 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:3/319: fsync d3/d5/d5c/f58 0 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:5/349: write d3/d14/d1f/d26/d2e/d56/f54 [2142950,126931] 0 2024-09-21T04:15:14.599 INFO:tasks.workunit.client.1.smithi148.stdout:5/350: creat d3/d14/d1f/f80 x:0 0 0 2024-09-21T04:15:14.603 INFO:tasks.workunit.client.0.smithi110.stdout:4/549: link d17/d3e/l69 d17/lc0 0 2024-09-21T04:15:14.604 INFO:tasks.workunit.client.1.smithi148.stdout:7/380: rename d5/d14/d25/d40/d7b/l39 to d5/d8/d6f/l7c 0 2024-09-21T04:15:14.605 INFO:tasks.workunit.client.1.smithi148.stdout:6/292: dwrite d0/d1a/d23/d16/d4f/f54 [0,4194304] 0 2024-09-21T04:15:14.606 INFO:tasks.workunit.client.0.smithi110.stdout:6/356: rename d4/de/f10 to d4/d12/d16/d48/d4b/f71 0 2024-09-21T04:15:14.610 INFO:tasks.workunit.client.1.smithi148.stdout:9/301: symlink d5/d2b/d32/d67/l74 0 2024-09-21T04:15:14.610 INFO:tasks.workunit.client.1.smithi148.stdout:2/440: dwrite d0/d9/dc/d13/d25/f79 [0,4194304] 0 2024-09-21T04:15:14.610 INFO:tasks.workunit.client.1.smithi148.stdout:9/302: truncate f0 2800494 0 2024-09-21T04:15:14.610 INFO:tasks.workunit.client.0.smithi110.stdout:4/550: symlink d17/d34/d3d/d20/d43/d51/d52/d98/lc1 0 2024-09-21T04:15:14.611 INFO:tasks.workunit.client.0.smithi110.stdout:4/551: chown d17/d34/d3d/d57/f5a 389 1 2024-09-21T04:15:14.613 INFO:tasks.workunit.client.1.smithi148.stdout:3/320: creat d3/d5/d6/d12/da/d21/f63 x:0 0 0 2024-09-21T04:15:14.615 INFO:tasks.workunit.client.0.smithi110.stdout:1/655: rename d2/d9/d49/l65 to d2/d9/d49/d4f/d87/ld6 0 2024-09-21T04:15:14.615 INFO:tasks.workunit.client.0.smithi110.stdout:8/441: read d0/d7/d13/d4b/f28 [3154923,92056] 0 2024-09-21T04:15:14.616 INFO:tasks.workunit.client.0.smithi110.stdout:6/357: getdents d4/d12/d17/d2e/d40/d65 0 2024-09-21T04:15:14.618 INFO:tasks.workunit.client.0.smithi110.stdout:4/552: symlink d17/d3e/d6b/lc2 0 2024-09-21T04:15:14.620 INFO:tasks.workunit.client.0.smithi110.stdout:1/656: symlink d2/d9/d26/ld7 0 2024-09-21T04:15:14.620 INFO:tasks.workunit.client.0.smithi110.stdout:1/657: fdatasync d2/d9/d49/d4f/f67 0 2024-09-21T04:15:14.621 INFO:tasks.workunit.client.0.smithi110.stdout:8/442: chown d0/d7/d13/d4b/f5b 1 1 2024-09-21T04:15:14.621 INFO:tasks.workunit.client.0.smithi110.stdout:8/443: truncate d0/d7/d13/f5a 383421 0 2024-09-21T04:15:14.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/303: write d5/f16 [4675519,118393] 0 2024-09-21T04:15:14.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/304: fsync d5/d1e/d37/f4d 0 2024-09-21T04:15:14.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/305: write d5/d60/f45 [1015735,116734] 0 2024-09-21T04:15:14.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/306: creat d5/d60/d28/f75 x:0 0 0 2024-09-21T04:15:14.623 INFO:tasks.workunit.client.0.smithi110.stdout:4/553: creat d17/d34/d3d/d20/d43/d51/d7a/fc3 x:0 0 0 2024-09-21T04:15:14.625 INFO:tasks.workunit.client.1.smithi148.stdout:5/351: mknod d3/d34/c81 0 2024-09-21T04:15:14.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/349: getdents d4/d12 0 2024-09-21T04:15:14.626 INFO:tasks.workunit.client.0.smithi110.stdout:1/658: mknod d2/d9/d23/d4e/d8e/cd8 0 2024-09-21T04:15:14.626 INFO:tasks.workunit.client.0.smithi110.stdout:1/659: creat d2/d9/d26/d3d/fd9 x:0 0 0 2024-09-21T04:15:14.626 INFO:tasks.workunit.client.0.smithi110.stdout:1/660: truncate d2/d9/d26/d3d/f43 1622335 0 2024-09-21T04:15:14.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/444: creat d0/d7/d89/f99 x:0 0 0 2024-09-21T04:15:14.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/445: write d0/d30/d34/d3e/d32/f7c [1409143,102651] 0 2024-09-21T04:15:14.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/446: truncate d0/d7/d13/d46/f68 311197 0 2024-09-21T04:15:14.630 INFO:tasks.workunit.client.1.smithi148.stdout:4/350: dread d4/d12/d16/d26/f3e [0,4194304] 0 2024-09-21T04:15:14.632 INFO:tasks.workunit.client.0.smithi110.stdout:6/358: dread d4/f3d [0,4194304] 0 2024-09-21T04:15:14.633 INFO:tasks.workunit.client.0.smithi110.stdout:4/554: rmdir d17/d1e/d64 39 2024-09-21T04:15:14.635 INFO:tasks.workunit.client.1.smithi148.stdout:0/350: dwrite d5/d6/d7/d22/f46 [4194304,4194304] 0 2024-09-21T04:15:14.636 INFO:tasks.workunit.client.0.smithi110.stdout:1/661: dread d2/d9/d23/fa2 [0,4194304] 0 2024-09-21T04:15:14.636 INFO:tasks.workunit.client.0.smithi110.stdout:1/662: dread - d2/d9/d49/d7a/da8/fd3 zero size 2024-09-21T04:15:14.637 INFO:tasks.workunit.client.1.smithi148.stdout:8/355: dwrite dc/d2a/d42/f53 [0,4194304] 0 2024-09-21T04:15:14.637 INFO:tasks.workunit.client.1.smithi148.stdout:8/356: dread - dc/d13/d3c/f72 zero size 2024-09-21T04:15:14.637 INFO:tasks.workunit.client.1.smithi148.stdout:8/357: write dc/d13/f24 [466699,4087] 0 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/439: dwrite d1/d7/d18/f41 [0,4194304] 0 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/440: chown d1/f3e 208155 1 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/441: creat d1/d49/d4d/f9c x:0 0 0 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/442: truncate d1/d49/d4d/d1e/d5a/d64/d7b/f95 599755 0 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/443: write d1/f3e [1430328,120966] 0 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/444: chown d1/d39/f50 556 1 2024-09-21T04:15:14.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/445: write d1/f35 [88036,81882] 0 2024-09-21T04:15:14.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/313: dwrite d7/d10/f13 [4194304,4194304] 0 2024-09-21T04:15:14.647 INFO:tasks.workunit.client.1.smithi148.stdout:7/381: dwrite d5/d14/d25/d40/d7b/f48 [0,4194304] 0 2024-09-21T04:15:14.647 INFO:tasks.workunit.client.1.smithi148.stdout:2/441: symlink d0/d9/dc/d13/d25/d4e/l87 0 2024-09-21T04:15:14.650 INFO:tasks.workunit.client.0.smithi110.stdout:4/555: getdents d17/d34/d3d/d20/d43/d51/d7a/d86 0 2024-09-21T04:15:14.650 INFO:tasks.workunit.client.0.smithi110.stdout:4/556: creat d17/d1e/d7b/fc4 x:0 0 0 2024-09-21T04:15:14.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/352: unlink d3/d14/d1f/d26/d2e/d56/f3f 0 2024-09-21T04:15:14.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/353: write d3/d14/d1f/d26/d2e/d3d/f61 [952969,64935] 0 2024-09-21T04:15:14.655 INFO:tasks.workunit.client.1.smithi148.stdout:4/351: mkdir d4/d66 0 2024-09-21T04:15:14.655 INFO:tasks.workunit.client.1.smithi148.stdout:4/352: chown d4/d23/l28 3222 1 2024-09-21T04:15:14.664 INFO:tasks.workunit.client.1.smithi148.stdout:4/353: dread d4/d12/d2c/f32 [4194304,4194304] 0 2024-09-21T04:15:14.666 INFO:tasks.workunit.client.0.smithi110.stdout:9/314: rename d7/d10/d24/d3c/c1d to d7/c6d 0 2024-09-21T04:15:14.666 INFO:tasks.workunit.client.0.smithi110.stdout:9/315: creat d7/d10/d24/d3c/d1f/d43/f6e x:0 0 0 2024-09-21T04:15:14.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/358: symlink dc/d13/d39/d6e/l73 0 2024-09-21T04:15:14.668 INFO:tasks.workunit.client.1.smithi148.stdout:8/359: truncate dc/d13/d3c/f49 454865 0 2024-09-21T04:15:14.671 INFO:tasks.workunit.client.0.smithi110.stdout:6/359: getdents d4/d12/d16 0 2024-09-21T04:15:14.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/354: symlink d3/d34/d3c/d5d/d75/l82 0 2024-09-21T04:15:14.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/355: write d3/d14/d1f/f50 [1284582,74815] 0 2024-09-21T04:15:14.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/307: dwrite d5/d1e/d37/d46/f6c [0,4194304] 0 2024-09-21T04:15:14.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/308: write d5/f1b [126482,33035] 0 2024-09-21T04:15:14.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/309: stat d5/d60/d28/l70 0 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:6/293: rename d0/d34/d45 to d0/d1a/d23/d16/d4f/d55 0 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:6/294: chown d0/d8/d18/d4e/f4a 1 1 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/354: dread d4/d12/f1b [0,4194304] 0 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/355: getdents d4/d12/d2c 0 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/356: stat d4/d12/f41 0 2024-09-21T04:15:14.677 INFO:tasks.workunit.client.0.smithi110.stdout:4/557: mkdir d17/d1e/d7b/dc5 0 2024-09-21T04:15:14.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/360: symlink dc/d2a/d5d/l74 0 2024-09-21T04:15:14.680 INFO:tasks.workunit.client.1.smithi148.stdout:5/356: truncate d3/f9 3129474 0 2024-09-21T04:15:14.680 INFO:tasks.workunit.client.0.smithi110.stdout:1/663: dwrite d2/d9/d26/d3d/d8f/f90 [0,4194304] 0 2024-09-21T04:15:14.682 INFO:tasks.workunit.client.0.smithi110.stdout:5/446: dwrite d1/d49/d4d/d1e/d5a/d64/f73 [0,4194304] 0 2024-09-21T04:15:14.682 INFO:tasks.workunit.client.0.smithi110.stdout:5/447: fsync d1/d49/d4d/d1e/d4a/f83 0 2024-09-21T04:15:14.682 INFO:tasks.workunit.client.0.smithi110.stdout:8/447: dwrite d0/d30/d34/d3e/f88 [0,4194304] 0 2024-09-21T04:15:14.682 INFO:tasks.workunit.client.0.smithi110.stdout:5/448: fsync d1/d7/f13 0 2024-09-21T04:15:14.683 INFO:tasks.workunit.client.0.smithi110.stdout:5/449: dread d1/d7/d58/f77 [0,4194304] 0 2024-09-21T04:15:14.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/382: dwrite f4 [4194304,4194304] 0 2024-09-21T04:15:14.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/383: chown d5/d14/f78 1842896 1 2024-09-21T04:15:14.686 INFO:tasks.workunit.client.0.smithi110.stdout:4/558: read d17/d34/d3d/d20/d43/d51/d7a/d86/f89 [22113,56455] 0 2024-09-21T04:15:14.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/559: write d17/d1e/fbf [464758,109617] 0 2024-09-21T04:15:14.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/560: chown d17/d1e/d64/f8f 10892 1 2024-09-21T04:15:14.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/561: readlink d17/d1e/d7b/l83 0 2024-09-21T04:15:14.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/562: chown d17/d34/d3d/d20/d43/d51/d52/c75 921 1 2024-09-21T04:15:14.687 INFO:tasks.workunit.client.1.smithi148.stdout:0/351: dwrite d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:14 smithi148 ceph-mon[34957]: Deploying cephadm binary to smithi148 2024-09-21T04:15:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:14 smithi148 ceph-mon[34957]: Deploying cephadm binary to smithi110 2024-09-21T04:15:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:14 smithi148 ceph-mon[34957]: pgmap v4: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:14 smithi148 ceph-mon[34957]: mgrmap e23: smithi148.zjrahi(active, since 2s) 2024-09-21T04:15:14.692 INFO:tasks.workunit.client.0.smithi110.stdout:5/450: mknod d1/dc/c9d 0 2024-09-21T04:15:14.694 INFO:tasks.workunit.client.0.smithi110.stdout:4/563: rename d17/d34/d3d/d20/d43/f92 to d17/d3e/dbd/fc6 0 2024-09-21T04:15:14.695 INFO:tasks.workunit.client.0.smithi110.stdout:8/448: dread d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:14.695 INFO:tasks.workunit.client.0.smithi110.stdout:8/449: chown d0/d7/d13/d4b/d91 169990 1 2024-09-21T04:15:14.699 INFO:tasks.workunit.client.0.smithi110.stdout:5/451: mkdir d1/d7/d58/d9e 0 2024-09-21T04:15:14.700 INFO:tasks.workunit.client.0.smithi110.stdout:8/450: write d0/d30/d54/d6b/f75 [2553760,79496] 0 2024-09-21T04:15:14.700 INFO:tasks.workunit.client.0.smithi110.stdout:8/451: stat d0/d7/c4f 0 2024-09-21T04:15:14.700 INFO:tasks.workunit.client.0.smithi110.stdout:8/452: readlink d0/l15 0 2024-09-21T04:15:14.703 INFO:tasks.workunit.client.0.smithi110.stdout:9/316: dwrite d7/d10/d24/d3c/d14/f54 [0,4194304] 0 2024-09-21T04:15:14.703 INFO:tasks.workunit.client.0.smithi110.stdout:9/317: dread - d7/d10/d24/f52 zero size 2024-09-21T04:15:14.704 INFO:tasks.workunit.client.1.smithi148.stdout:4/357: dwrite d4/d12/d2c/f57 [0,4194304] 0 2024-09-21T04:15:14.704 INFO:tasks.workunit.client.0.smithi110.stdout:5/452: creat d1/d49/d4d/d1e/d4a/f9f x:0 0 0 2024-09-21T04:15:14.704 INFO:tasks.workunit.client.0.smithi110.stdout:5/453: readlink d1/d39/l42 0 2024-09-21T04:15:14.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/360: dwrite d4/d12/d17/d2e/d40/f5d [0,4194304] 0 2024-09-21T04:15:14.710 INFO:tasks.workunit.client.1.smithi148.stdout:6/295: symlink d0/d1a/d23/dc/l56 0 2024-09-21T04:15:14.710 INFO:tasks.workunit.client.1.smithi148.stdout:6/296: fsync d0/d1a/d23/d27/f2d 0 2024-09-21T04:15:14.711 INFO:tasks.workunit.client.1.smithi148.stdout:6/297: write d0/d8/f24 [1443573,13776] 0 2024-09-21T04:15:14.711 INFO:tasks.workunit.client.0.smithi110.stdout:4/564: dwrite d17/d1e/d7b/fb8 [0,4194304] 0 2024-09-21T04:15:14.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/361: dread d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:14.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/362: creat d4/d12/d16/d48/d54/f72 x:0 0 0 2024-09-21T04:15:14.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/321: rename d3/d5/d6/d12/da/d23/d28/l5e to d3/d5/d6/d2b/l64 0 2024-09-21T04:15:14.714 INFO:tasks.workunit.client.1.smithi148.stdout:2/442: rename d0/d9/dc to d0/d9/dc/d13/d25/d2d/d88 22 2024-09-21T04:15:14.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/322: readlink d3/d5/d6/d12/da/d52/l2a 0 2024-09-21T04:15:14.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/323: dread - d3/f55 zero size 2024-09-21T04:15:14.717 INFO:tasks.workunit.client.1.smithi148.stdout:5/357: creat d3/f83 x:0 0 0 2024-09-21T04:15:14.718 INFO:tasks.workunit.client.1.smithi148.stdout:5/358: chown d3/d14/d1f/d26/d2e/d56/f15 127423 1 2024-09-21T04:15:14.723 INFO:tasks.workunit.client.0.smithi110.stdout:5/454: dwrite d1/d7/d58/f84 [0,4194304] 0 2024-09-21T04:15:14.725 INFO:tasks.workunit.client.1.smithi148.stdout:0/352: creat d5/d6/d7/d11/d48/f6b x:0 0 0 2024-09-21T04:15:14.726 INFO:tasks.workunit.client.1.smithi148.stdout:6/298: write d0/d1a/d23/d16/d4f/f54 [499566,36510] 0 2024-09-21T04:15:14.729 INFO:tasks.workunit.client.1.smithi148.stdout:4/358: mknod d4/d3a/c67 0 2024-09-21T04:15:14.730 INFO:tasks.workunit.client.1.smithi148.stdout:4/359: creat d4/d12/d16/d26/f68 x:0 0 0 2024-09-21T04:15:14.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/310: mknod d5/c76 0 2024-09-21T04:15:14.735 INFO:tasks.workunit.client.0.smithi110.stdout:9/318: rmdir d7/d10/d24/d3c/d14/d3e 39 2024-09-21T04:15:14.735 INFO:tasks.workunit.client.0.smithi110.stdout:9/319: fdatasync d7/d10/d24/d3c/f20 0 2024-09-21T04:15:14.736 INFO:tasks.workunit.client.1.smithi148.stdout:8/361: rmdir dc/d13/d3c 39 2024-09-21T04:15:14.737 INFO:tasks.workunit.client.1.smithi148.stdout:8/362: dread dc/d13/d3c/f5f [0,4194304] 0 2024-09-21T04:15:14.738 INFO:tasks.workunit.client.1.smithi148.stdout:3/324: dwrite d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:14.739 INFO:tasks.workunit.client.0.smithi110.stdout:4/565: mkdir d17/d3e/d6b/dc7 0 2024-09-21T04:15:14.741 INFO:tasks.workunit.client.1.smithi148.stdout:4/360: symlink d4/d12/d2c/d2d/l69 0 2024-09-21T04:15:14.744 INFO:tasks.workunit.client.0.smithi110.stdout:6/363: rename d4/d12/d16/c1e to d4/d12/d16/d52/c73 0 2024-09-21T04:15:14.744 INFO:tasks.workunit.client.0.smithi110.stdout:6/364: chown d4/c1b 3928 1 2024-09-21T04:15:14.745 INFO:tasks.workunit.client.0.smithi110.stdout:6/365: write d4/d12/d16/d48/d4b/f71 [647146,95297] 0 2024-09-21T04:15:14.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/363: dread f7 [0,4194304] 0 2024-09-21T04:15:14.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/364: readlink dc/d13/d28/l36 0 2024-09-21T04:15:14.749 INFO:tasks.workunit.client.0.smithi110.stdout:5/455: getdents d1/d49/d4d/d1e 0 2024-09-21T04:15:14.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/332: sync 2024-09-21T04:15:14.750 INFO:tasks.workunit.client.1.smithi148.stdout:0/353: dwrite d5/d6/d7/d11/f29 [0,4194304] 0 2024-09-21T04:15:14.752 INFO:tasks.workunit.client.1.smithi148.stdout:8/365: dread dc/d2a/f5b [0,4194304] 0 2024-09-21T04:15:14.752 INFO:tasks.workunit.client.1.smithi148.stdout:8/366: write dc/d2a/f5b [1202662,74761] 0 2024-09-21T04:15:14.754 INFO:tasks.workunit.client.0.smithi110.stdout:4/566: dread d17/d34/f37 [0,4194304] 0 2024-09-21T04:15:14.755 INFO:tasks.workunit.client.1.smithi148.stdout:3/325: truncate d3/d5/d6/d12/da/d23/d28/d46/f5b 1666890 0 2024-09-21T04:15:14.756 INFO:tasks.workunit.client.0.smithi110.stdout:6/366: rename d4/d12/d16/c1c to d4/d12/d17/d33/c74 0 2024-09-21T04:15:14.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/361: mknod d4/d12/d16/d29/c6a 0 2024-09-21T04:15:14.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/362: write d4/f6 [8808252,97469] 0 2024-09-21T04:15:14.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/363: chown d4/d12/d16/d26/f3e 57536941 1 2024-09-21T04:15:14.760 INFO:tasks.workunit.client.1.smithi148.stdout:9/311: getdents d5/d1e/d37/d46 0 2024-09-21T04:15:14.760 INFO:tasks.workunit.client.0.smithi110.stdout:9/320: getdents d7/d10/d24/d3c/d1f/d25 0 2024-09-21T04:15:14.762 INFO:tasks.workunit.client.1.smithi148.stdout:1/333: mknod db/d24/d4f/c69 0 2024-09-21T04:15:14.762 INFO:tasks.workunit.client.0.smithi110.stdout:6/367: mknod d4/d12/d17/d2e/d38/c75 0 2024-09-21T04:15:14.764 INFO:tasks.workunit.client.0.smithi110.stdout:5/456: getdents d1/d22 0 2024-09-21T04:15:14.764 INFO:tasks.workunit.client.0.smithi110.stdout:5/457: chown d1/d49/d4d/d1e/d5a/d64/c7e 492901 1 2024-09-21T04:15:14.766 INFO:tasks.workunit.client.1.smithi148.stdout:7/384: rename d5/dc/d28/d4a to d5/d8/d6f/d7d 0 2024-09-21T04:15:14.766 INFO:tasks.workunit.client.0.smithi110.stdout:9/321: mkdir d7/d10/d24/d3c/d14/d5c/d6f 0 2024-09-21T04:15:14.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/367: getdents dc/d43/d4c 0 2024-09-21T04:15:14.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/368: chown dc/d2a/d42 213 1 2024-09-21T04:15:14.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/369: readlink dc/l25 0 2024-09-21T04:15:14.768 INFO:tasks.workunit.client.1.smithi148.stdout:1/334: dread db/d24/f26 [0,4194304] 0 2024-09-21T04:15:14.769 INFO:tasks.workunit.client.1.smithi148.stdout:3/326: mkdir d3/d5/d6/d12/da/d23/d28/d65 0 2024-09-21T04:15:14.770 INFO:tasks.workunit.client.0.smithi110.stdout:5/458: symlink d1/d22/d51/la0 0 2024-09-21T04:15:14.772 INFO:tasks.workunit.client.1.smithi148.stdout:4/364: unlink d4/d12/d2c/d2d/f52 0 2024-09-21T04:15:14.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/459: creat d1/d7/fa1 x:0 0 0 2024-09-21T04:15:14.773 INFO:tasks.workunit.client.0.smithi110.stdout:5/460: fdatasync d1/d49/d4d/d1e/f2f 0 2024-09-21T04:15:14.778 INFO:tasks.workunit.client.1.smithi148.stdout:7/385: creat d5/d8/d6f/d7d/f7e x:0 0 0 2024-09-21T04:15:14.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/443: rename d0/d9/f47 to d0/d9/dc/d13/d25/f89 0 2024-09-21T04:15:14.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/444: creat d0/d9/dc/d13/d4c/f8a x:0 0 0 2024-09-21T04:15:14.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/445: creat d0/d9/dc/d13/f8b x:0 0 0 2024-09-21T04:15:14.779 INFO:tasks.workunit.client.1.smithi148.stdout:8/370: symlink dc/d13/d39/d6e/l75 0 2024-09-21T04:15:14.780 INFO:tasks.workunit.client.1.smithi148.stdout:0/354: dwrite d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:14.781 INFO:tasks.workunit.client.1.smithi148.stdout:4/365: truncate d4/d23/f2b 30642 0 2024-09-21T04:15:14.781 INFO:tasks.workunit.client.1.smithi148.stdout:4/366: read d4/fb [2846042,71457] 0 2024-09-21T04:15:14.781 INFO:tasks.workunit.client.1.smithi148.stdout:4/367: getdents d4/d12/d2c 0 2024-09-21T04:15:14.782 INFO:tasks.workunit.client.0.smithi110.stdout:5/461: write d1/d22/d51/f70 [3882078,91391] 0 2024-09-21T04:15:14.782 INFO:tasks.workunit.client.0.smithi110.stdout:6/368: rename d4/d12/d16/d52 to d4/d12/d17/d33/d76 0 2024-09-21T04:15:14.784 INFO:tasks.workunit.client.0.smithi110.stdout:5/462: mkdir d1/d39/da2 0 2024-09-21T04:15:14.784 INFO:tasks.workunit.client.0.smithi110.stdout:5/463: write d1/d7/d18/f8b [328041,109613] 0 2024-09-21T04:15:14.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/446: unlink d0/d9/dc/d13/f3b 0 2024-09-21T04:15:14.787 INFO:tasks.workunit.client.0.smithi110.stdout:9/322: write d7/d10/d24/d3c/d35/f3a [3405995,121681] 0 2024-09-21T04:15:14.787 INFO:tasks.workunit.client.0.smithi110.stdout:6/369: symlink d4/d12/d17/d33/d3e/l77 0 2024-09-21T04:15:14.790 INFO:tasks.workunit.client.1.smithi148.stdout:4/368: dread d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:14.792 INFO:tasks.workunit.client.0.smithi110.stdout:5/464: symlink d1/d49/d4d/d1e/la3 0 2024-09-21T04:15:14.792 INFO:tasks.workunit.client.0.smithi110.stdout:4/567: dwrite d17/d34/d3d/f41 [0,4194304] 0 2024-09-21T04:15:14.798 INFO:tasks.workunit.client.0.smithi110.stdout:9/323: getdents d7/d10/d24/d3c/d14/d31 0 2024-09-21T04:15:14.798 INFO:tasks.workunit.client.0.smithi110.stdout:9/324: write d7/d10/f13 [588784,72057] 0 2024-09-21T04:15:14.798 INFO:tasks.workunit.client.0.smithi110.stdout:9/325: chown d7/d10/d24/d3c/d14/d31/f53 433431 1 2024-09-21T04:15:14.800 INFO:tasks.workunit.client.1.smithi148.stdout:8/371: rename dc/d2a to dc/d13/d76 0 2024-09-21T04:15:14.800 INFO:tasks.workunit.client.1.smithi148.stdout:8/372: readlink dc/l5e 0 2024-09-21T04:15:14.801 INFO:tasks.workunit.client.1.smithi148.stdout:8/373: chown dc/d3e/f70 8103670 1 2024-09-21T04:15:14.803 INFO:tasks.workunit.client.0.smithi110.stdout:9/326: symlink d7/d10/d24/d3c/d14/d5c/d6f/l70 0 2024-09-21T04:15:14.805 INFO:tasks.workunit.client.0.smithi110.stdout:9/327: mkdir d7/d10/d24/d3c/d35/d71 0 2024-09-21T04:15:14.805 INFO:tasks.workunit.client.0.smithi110.stdout:9/328: write d7/d26/f5b [25824,46416] 0 2024-09-21T04:15:14.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/312: dwrite d5/d60/d28/d38/f52 [0,4194304] 0 2024-09-21T04:15:14.807 INFO:tasks.workunit.client.1.smithi148.stdout:8/374: dread dc/f31 [0,4194304] 0 2024-09-21T04:15:14.811 INFO:tasks.workunit.client.1.smithi148.stdout:9/313: creat d5/d39/f77 x:0 0 0 2024-09-21T04:15:14.814 INFO:tasks.workunit.client.0.smithi110.stdout:4/568: dread d17/f32 [0,4194304] 0 2024-09-21T04:15:14.814 INFO:tasks.workunit.client.1.smithi148.stdout:8/375: read f7 [1656373,8575] 0 2024-09-21T04:15:14.814 INFO:tasks.workunit.client.1.smithi148.stdout:1/335: dwrite db/d24/d2a/f42 [0,4194304] 0 2024-09-21T04:15:14.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/336: creat db/d46/f6a x:0 0 0 2024-09-21T04:15:14.815 INFO:tasks.workunit.client.1.smithi148.stdout:3/327: dwrite d3/d5/d6/d12/da/d52/f59 [0,4194304] 0 2024-09-21T04:15:14.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/314: mknod d5/d2b/d32/d67/c78 0 2024-09-21T04:15:14.820 INFO:tasks.workunit.client.1.smithi148.stdout:8/376: link dc/d13/l20 dc/d13/d76/d42/l77 0 2024-09-21T04:15:14.822 INFO:tasks.workunit.client.1.smithi148.stdout:8/377: dread dc/f31 [0,4194304] 0 2024-09-21T04:15:14.825 INFO:tasks.workunit.client.1.smithi148.stdout:1/337: creat db/d46/d4c/f6b x:0 0 0 2024-09-21T04:15:14.825 INFO:tasks.workunit.client.1.smithi148.stdout:7/386: dwrite d5/dc/d28/f57 [0,4194304] 0 2024-09-21T04:15:14.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/369: dwrite d4/d12/d16/d29/f33 [0,4194304] 0 2024-09-21T04:15:14.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/370: fdatasync d4/d12/d2c/f32 0 2024-09-21T04:15:14.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/371: chown d4/d23/c60 42 1 2024-09-21T04:15:14.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/372: truncate d4/d12/d2c/f5c 623240 0 2024-09-21T04:15:14.827 INFO:tasks.workunit.client.1.smithi148.stdout:2/447: dwrite d0/d2b/d48/f57 [4194304,4194304] 0 2024-09-21T04:15:14.829 INFO:tasks.workunit.client.0.smithi110.stdout:5/465: dwrite d1/d7/f4e [0,4194304] 0 2024-09-21T04:15:14.829 INFO:tasks.workunit.client.0.smithi110.stdout:5/466: readlink d1/d39/l4b 0 2024-09-21T04:15:14.829 INFO:tasks.workunit.client.0.smithi110.stdout:5/467: creat d1/d39/fa4 x:0 0 0 2024-09-21T04:15:14.831 INFO:tasks.workunit.client.0.smithi110.stdout:4/569: write d17/d1e/f48 [2439884,114109] 0 2024-09-21T04:15:14.832 INFO:tasks.workunit.client.1.smithi148.stdout:3/328: creat d3/d5/d6/d12/f66 x:0 0 0 2024-09-21T04:15:14.835 INFO:tasks.workunit.client.1.smithi148.stdout:9/315: chown d5/c11 107416340 1 2024-09-21T04:15:14.835 INFO:tasks.workunit.client.1.smithi148.stdout:9/316: fdatasync d5/d1e/d37/d46/f6c 0 2024-09-21T04:15:14.835 INFO:tasks.workunit.client.1.smithi148.stdout:0/355: dwrite d5/d6/d3e/f3a [0,4194304] 0 2024-09-21T04:15:14.835 INFO:tasks.workunit.client.0.smithi110.stdout:9/329: dwrite d7/d10/d24/d3c/d11/f2e [0,4194304] 0 2024-09-21T04:15:14.836 INFO:tasks.workunit.client.0.smithi110.stdout:9/330: dread d7/d10/d24/d3c/d1f/f41 [0,4194304] 0 2024-09-21T04:15:14.837 INFO:tasks.workunit.client.1.smithi148.stdout:1/338: mknod db/d46/c6c 0 2024-09-21T04:15:14.842 INFO:tasks.workunit.client.0.smithi110.stdout:4/570: creat d17/d34/d3d/d57/db9/fc8 x:0 0 0 2024-09-21T04:15:14.842 INFO:tasks.workunit.client.0.smithi110.stdout:4/571: write d17/d34/d3d/d20/d43/d51/d52/d98/f99 [1240876,51898] 0 2024-09-21T04:15:14.844 INFO:tasks.workunit.client.1.smithi148.stdout:7/387: rename d5/dc/f3e to d5/d8/d6f/d7d/f7f 0 2024-09-21T04:15:14.846 INFO:tasks.workunit.client.1.smithi148.stdout:0/356: write d5/d6/f15 [2502336,29738] 0 2024-09-21T04:15:14.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/373: creat d4/d12/d2c/d2d/f6b x:0 0 0 2024-09-21T04:15:14.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/374: fsync d4/d3a/f5f 0 2024-09-21T04:15:14.852 INFO:tasks.workunit.client.0.smithi110.stdout:5/468: dwrite d1/d22/f4f [0,4194304] 0 2024-09-21T04:15:14.852 INFO:tasks.workunit.client.0.smithi110.stdout:5/469: write d1/d22/d51/f98 [883766,644] 0 2024-09-21T04:15:14.853 INFO:tasks.workunit.client.0.smithi110.stdout:9/331: rename d7/d10/d24/d3c/d14/d3e to d7/d10/d24/d3c/d1f/d72 0 2024-09-21T04:15:14.853 INFO:tasks.workunit.client.0.smithi110.stdout:9/332: chown d7/d10/d24/l2b 2032339731 1 2024-09-21T04:15:14.854 INFO:tasks.workunit.client.1.smithi148.stdout:2/448: getdents d0/d9/dc/d13/d25/d2d 0 2024-09-21T04:15:14.855 INFO:tasks.workunit.client.1.smithi148.stdout:4/375: dread d4/d12/d16/d29/f5a [0,4194304] 0 2024-09-21T04:15:14.856 INFO:tasks.workunit.client.0.smithi110.stdout:4/572: unlink d17/d34/f37 0 2024-09-21T04:15:14.857 INFO:tasks.workunit.client.1.smithi148.stdout:3/329: getdents d3/d1f/d4d 0 2024-09-21T04:15:14.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/330: getdents d3/d5/d5c/d60 0 2024-09-21T04:15:14.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/357: dwrite d5/d6/d2a/d2b/f36 [0,4194304] 0 2024-09-21T04:15:14.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/358: chown d5/d6/d2a/d2b/c53 0 1 2024-09-21T04:15:14.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/359: dread - d5/d6/d7/d11/d48/f6b zero size 2024-09-21T04:15:14.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/360: creat d5/d6/d7/d22/f6c x:0 0 0 2024-09-21T04:15:14.863 INFO:tasks.workunit.client.1.smithi148.stdout:0/361: fdatasync d5/d6/d7/d11/f1d 0 2024-09-21T04:15:14.864 INFO:tasks.workunit.client.0.smithi110.stdout:4/573: dread d17/f3c [0,4194304] 0 2024-09-21T04:15:14.864 INFO:tasks.workunit.client.0.smithi110.stdout:4/574: chown d17/d1e 58205820 1 2024-09-21T04:15:14.864 INFO:tasks.workunit.client.0.smithi110.stdout:4/575: write d17/d34/d3d/d20/f95 [2145012,56533] 0 2024-09-21T04:15:14.864 INFO:tasks.workunit.client.0.smithi110.stdout:4/576: truncate fa 1863015 0 2024-09-21T04:15:14.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/449: dread d0/d9/dc/d13/d25/f45 [0,4194304] 0 2024-09-21T04:15:14.865 INFO:tasks.workunit.client.0.smithi110.stdout:5/470: getdents d1/d49 0 2024-09-21T04:15:14.865 INFO:tasks.workunit.client.0.smithi110.stdout:9/333: dread d7/d10/d24/d3c/f20 [0,4194304] 0 2024-09-21T04:15:14.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/317: stat d5/d1e/l2a 0 2024-09-21T04:15:14.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/318: chown d5/d60/d28/c5c 1952414 1 2024-09-21T04:15:14.871 INFO:tasks.workunit.client.0.smithi110.stdout:4/577: rename d17/f27 to d17/d34/d3d/d20/d43/d51/d7a/d86/fc9 0 2024-09-21T04:15:14.874 INFO:tasks.workunit.client.0.smithi110.stdout:9/334: getdents d7/d10/d24/d3c/d14/d5c/d6f 0 2024-09-21T04:15:14.874 INFO:tasks.workunit.client.0.smithi110.stdout:5/471: write d1/d39/f79 [721192,66696] 0 2024-09-21T04:15:14.875 INFO:tasks.workunit.client.0.smithi110.stdout:5/472: write d1/d49/d4d/d1e/f3b [1246364,62913] 0 2024-09-21T04:15:14.876 INFO:tasks.workunit.client.1.smithi148.stdout:4/376: dwrite d4/d23/d2a/f36 [0,4194304] 0 2024-09-21T04:15:14.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/473: unlink d1/d39/l2c 0 2024-09-21T04:15:14.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/474: fdatasync d1/d22/d51/f65 0 2024-09-21T04:15:14.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/475: truncate d1/d49/d4d/f48 4230408 0 2024-09-21T04:15:14.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/476: creat d1/d22/d51/fa5 x:0 0 0 2024-09-21T04:15:14.886 INFO:tasks.workunit.client.1.smithi148.stdout:1/339: mkdir db/dd/d21/d6d 0 2024-09-21T04:15:14.888 INFO:tasks.workunit.client.1.smithi148.stdout:3/331: link d3/d1f/f47 d3/d5/d6/d12/da/d23/d28/d65/f67 0 2024-09-21T04:15:14.891 INFO:tasks.workunit.client.1.smithi148.stdout:0/362: mkdir d5/d6/d2a/d3f/d6d 0 2024-09-21T04:15:14.892 INFO:tasks.workunit.client.1.smithi148.stdout:3/332: read d3/d1f/f47 [2809978,10965] 0 2024-09-21T04:15:14.893 INFO:tasks.workunit.client.1.smithi148.stdout:2/450: symlink d0/d9/dc/l8c 0 2024-09-21T04:15:14.893 INFO:tasks.workunit.client.1.smithi148.stdout:5/359: sync 2024-09-21T04:15:14.894 INFO:tasks.workunit.client.1.smithi148.stdout:9/319: symlink d5/d1e/d37/d46/l79 0 2024-09-21T04:15:14.894 INFO:tasks.workunit.client.1.smithi148.stdout:9/320: chown d5/c14 4 1 2024-09-21T04:15:14.895 INFO:tasks.workunit.client.1.smithi148.stdout:4/377: mkdir d4/d12/d16/d29/d61/d6c 0 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:1/340: link db/l18 db/d24/d2e/d59/l6e 0 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:1/341: write f6 [1375362,126378] 0 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:1/342: chown db/d24/d2a 536 1 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:1/343: fdatasync db/dd/d21/f43 0 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/388: mknod d5/dc/c80 0 2024-09-21T04:15:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/389: chown f4 109 1 2024-09-21T04:15:14.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/451: truncate d0/d2b/f53 2021273 0 2024-09-21T04:15:14.903 INFO:tasks.workunit.client.1.smithi148.stdout:9/321: link d5/l7 d5/d60/d4c/l7a 0 2024-09-21T04:15:14.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/378: dread d4/d12/d16/d29/f33 [0,4194304] 0 2024-09-21T04:15:14.904 INFO:tasks.workunit.client.1.smithi148.stdout:0/363: rmdir d5/d6/d2a/d45 39 2024-09-21T04:15:14.904 INFO:tasks.workunit.client.1.smithi148.stdout:6/299: sync 2024-09-21T04:15:14.904 INFO:tasks.workunit.client.1.smithi148.stdout:0/364: write d5/d6/d2a/d2b/f39 [1581858,112120] 0 2024-09-21T04:15:14.905 INFO:tasks.workunit.client.0.smithi110.stdout:4/578: dwrite d17/d34/d3d/d20/d43/d51/d7a/fa7 [0,4194304] 0 2024-09-21T04:15:14.906 INFO:tasks.workunit.client.0.smithi110.stdout:9/335: dwrite d7/d10/f37 [0,4194304] 0 2024-09-21T04:15:14.908 INFO:tasks.workunit.client.1.smithi148.stdout:1/344: dread db/d24/d4f/f55 [0,4194304] 0 2024-09-21T04:15:14.908 INFO:tasks.workunit.client.0.smithi110.stdout:4/579: mkdir d17/d34/d3d/d20/dca 0 2024-09-21T04:15:14.911 INFO:tasks.workunit.client.0.smithi110.stdout:9/336: creat d7/d10/d24/d3c/d35/f73 x:0 0 0 2024-09-21T04:15:14.911 INFO:tasks.workunit.client.0.smithi110.stdout:9/337: write d7/d10/d24/f38 [143567,13830] 0 2024-09-21T04:15:14.912 INFO:tasks.workunit.client.1.smithi148.stdout:5/360: rmdir d3/d14/d1f/d26 39 2024-09-21T04:15:14.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/580: symlink d17/d34/d3d/d20/lcb 0 2024-09-21T04:15:14.915 INFO:tasks.workunit.client.1.smithi148.stdout:9/322: mkdir d5/d1e/d37/d7b 0 2024-09-21T04:15:14.918 INFO:tasks.workunit.client.0.smithi110.stdout:9/338: symlink d7/d10/d24/d3c/d1f/d25/l74 0 2024-09-21T04:15:14.918 INFO:tasks.workunit.client.0.smithi110.stdout:9/339: chown d7/d10/d24/d3c/d35/f5e 125782145 1 2024-09-21T04:15:14.918 INFO:tasks.workunit.client.0.smithi110.stdout:9/340: dread - d7/d10/d24/d3c/d1f/d43/f6e zero size 2024-09-21T04:15:14.918 INFO:tasks.workunit.client.0.smithi110.stdout:9/341: creat d7/d10/d24/d3c/d35/f75 x:0 0 0 2024-09-21T04:15:14.918 INFO:tasks.workunit.client.0.smithi110.stdout:9/342: creat d7/d10/d24/d3c/d35/f76 x:0 0 0 2024-09-21T04:15:14.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:14 smithi110 ceph-mon[27723]: Deploying cephadm binary to smithi148 2024-09-21T04:15:14.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:14 smithi110 ceph-mon[27723]: Deploying cephadm binary to smithi110 2024-09-21T04:15:14.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:14 smithi110 ceph-mon[27723]: pgmap v4: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:14.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:14 smithi110 ceph-mon[27723]: mgrmap e23: smithi148.zjrahi(active, since 2s) 2024-09-21T04:15:14.919 INFO:tasks.workunit.client.1.smithi148.stdout:7/390: symlink d5/dc/l81 0 2024-09-21T04:15:14.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/300: rmdir d0/d1a/d23/d27/d2a 0 2024-09-21T04:15:14.926 INFO:tasks.workunit.client.1.smithi148.stdout:0/365: creat d5/d6/d2a/d45/f6e x:0 0 0 2024-09-21T04:15:14.929 INFO:tasks.workunit.client.1.smithi148.stdout:3/333: rmdir d3/d5/d5c/d60 0 2024-09-21T04:15:14.931 INFO:tasks.workunit.client.1.smithi148.stdout:2/452: mknod d0/d9/c8d 0 2024-09-21T04:15:14.931 INFO:tasks.workunit.client.1.smithi148.stdout:2/453: fsync d0/d9/dc/f46 0 2024-09-21T04:15:14.933 INFO:tasks.workunit.client.1.smithi148.stdout:1/345: symlink db/d24/d4f/l6f 0 2024-09-21T04:15:14.935 INFO:tasks.workunit.client.1.smithi148.stdout:5/361: creat d3/d14/f84 x:0 0 0 2024-09-21T04:15:14.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/366: dread d5/d6/d7/f4f [0,4194304] 0 2024-09-21T04:15:14.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/323: link d5/d1e/d37/d46/l79 d5/d1e/d37/l7c 0 2024-09-21T04:15:14.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/391: creat d5/d14/d25/d40/d7b/f82 x:0 0 0 2024-09-21T04:15:14.941 INFO:tasks.workunit.client.1.smithi148.stdout:4/379: getdents d4/d12/d16 0 2024-09-21T04:15:14.941 INFO:tasks.workunit.client.1.smithi148.stdout:4/380: truncate d4/d23/f2b 258653 0 2024-09-21T04:15:14.944 INFO:tasks.workunit.client.1.smithi148.stdout:9/324: read d5/f16 [234423,124011] 0 2024-09-21T04:15:14.944 INFO:tasks.workunit.client.1.smithi148.stdout:9/325: creat d5/d1f/f7d x:0 0 0 2024-09-21T04:15:14.945 INFO:tasks.workunit.client.1.smithi148.stdout:6/301: mkdir d0/d34/d38/d57 0 2024-09-21T04:15:14.947 INFO:tasks.workunit.client.1.smithi148.stdout:3/334: link d3/l45 d3/d1f/d4d/l68 0 2024-09-21T04:15:14.947 INFO:tasks.workunit.client.1.smithi148.stdout:3/335: stat d3/d5/d6/d12/da/d21/f63 0 2024-09-21T04:15:14.948 INFO:tasks.workunit.client.0.smithi110.stdout:4/581: rmdir d17/d34/d3d/d20/d43/d51/d7a 39 2024-09-21T04:15:14.948 INFO:tasks.workunit.client.0.smithi110.stdout:4/582: chown d17/d1e/d7b/c6f 6 1 2024-09-21T04:15:14.949 INFO:tasks.workunit.client.1.smithi148.stdout:2/454: mknod d0/c8e 0 2024-09-21T04:15:14.950 INFO:tasks.workunit.client.0.smithi110.stdout:9/343: getdents d7/d10/d24/d3c/d14 0 2024-09-21T04:15:14.950 INFO:tasks.workunit.client.0.smithi110.stdout:9/344: write d7/d10/f13 [2241321,31560] 0 2024-09-21T04:15:14.952 INFO:tasks.workunit.client.1.smithi148.stdout:1/346: unlink db/d24/d2e/l35 0 2024-09-21T04:15:14.952 INFO:tasks.workunit.client.1.smithi148.stdout:9/326: dread f1 [0,4194304] 0 2024-09-21T04:15:14.953 INFO:tasks.workunit.client.1.smithi148.stdout:9/327: fsync d5/d60/d28/d38/f52 0 2024-09-21T04:15:14.954 INFO:tasks.workunit.client.0.smithi110.stdout:4/583: creat d17/d34/d3d/d20/d43/d51/d7a/d96/fcc x:0 0 0 2024-09-21T04:15:14.954 INFO:tasks.workunit.client.0.smithi110.stdout:4/584: write d17/d34/d3d/d20/d43/d51/d52/fb1 [3660834,100014] 0 2024-09-21T04:15:14.954 INFO:tasks.workunit.client.0.smithi110.stdout:4/585: dread - d17/d34/d3d/fa6 zero size 2024-09-21T04:15:14.955 INFO:tasks.workunit.client.1.smithi148.stdout:5/362: symlink d3/d34/d3c/l85 0 2024-09-21T04:15:14.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/392: link d5/d14/l65 d5/d14/l83 0 2024-09-21T04:15:14.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/393: dread - d5/d14/f78 zero size 2024-09-21T04:15:14.959 INFO:tasks.workunit.client.1.smithi148.stdout:4/381: mkdir d4/d12/d16/d29/d61/d6d 0 2024-09-21T04:15:14.968 INFO:tasks.workunit.client.1.smithi148.stdout:6/302: mkdir d0/d34/d38/d57/d58 0 2024-09-21T04:15:14.969 INFO:tasks.workunit.client.1.smithi148.stdout:7/394: dread d5/dc/f5d [0,4194304] 0 2024-09-21T04:15:14.969 INFO:tasks.workunit.client.1.smithi148.stdout:3/336: rename d3/d5/d6/d12/d15/l27 to d3/d5/l69 0 2024-09-21T04:15:14.970 INFO:tasks.workunit.client.1.smithi148.stdout:2/455: truncate d0/d9/dc/f12 517314 0 2024-09-21T04:15:14.971 INFO:tasks.workunit.client.1.smithi148.stdout:1/347: stat db/d24/d2e/d59/l5d 0 2024-09-21T04:15:14.972 INFO:tasks.workunit.client.1.smithi148.stdout:5/363: symlink d3/d14/d1f/d26/l86 0 2024-09-21T04:15:14.974 INFO:tasks.workunit.client.1.smithi148.stdout:0/367: getdents d5/d6 0 2024-09-21T04:15:14.974 INFO:tasks.workunit.client.1.smithi148.stdout:0/368: truncate d5/d6/d2a/d2b/f5b 833476 0 2024-09-21T04:15:14.976 INFO:tasks.workunit.client.1.smithi148.stdout:7/395: rename d5/d14/d25/d40/d7b/f48 to d5/d14/d25/d40/d7b/f84 0 2024-09-21T04:15:14.978 INFO:tasks.workunit.client.1.smithi148.stdout:3/337: creat d3/d5/d6/d12/d43/d54/f6a x:0 0 0 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 -- 172.21.15.110:0/3743920918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0fea80 msgr2=0x7f8e8c0fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/3743920918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c0fee80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009880 tx=0x7f8e7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 -- 172.21.15.110:0/3743920918 shutdown_connections 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/3743920918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0ffc80 0x7f8e8c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/3743920918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c0fee80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 -- 172.21.15.110:0/3743920918 >> 172.21.15.110:0/3743920918 conn(0x7f8e8c0fa210 msgr2=0x7f8e8c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 -- 172.21.15.110:0/3743920918 shutdown_connections 2024-09-21T04:15:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.974+0000 7f8e90ce4640 1 -- 172.21.15.110:0/3743920918 wait complete. 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.975+0000 7f8e90ce4640 1 Processor -- start 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.975+0000 7f8e90ce4640 1 -- start start 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e90ce4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e90ce4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0ffc80 0x7f8e8c105080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e90ce4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c1055c0 con 0x7f8e8c0ffc80 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e90ce4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c105730 con 0x7f8e8c0fea80 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0ffc80 0x7f8e8c105080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:58478/0 (socket says 172.21.15.110:58478) 2024-09-21T04:15:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.976+0000 7f8e8a59c640 1 -- 172.21.15.110:0/1016343885 learned_addr learned my addr 172.21.15.110:0/1016343885 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:14.981 INFO:tasks.workunit.client.1.smithi148.stdout:9/328: dwrite d5/d60/d28/d38/f47 [0,4194304] 0 2024-09-21T04:15:14.981 INFO:tasks.workunit.client.1.smithi148.stdout:2/456: mkdir d0/d9/dc/d13/d25/d4e/d8f 0 2024-09-21T04:15:14.982 INFO:tasks.workunit.client.1.smithi148.stdout:2/457: dread - d0/d9/d16/d55/d6f/f7e zero size 2024-09-21T04:15:14.982 INFO:tasks.workunit.client.1.smithi148.stdout:2/458: stat d0/d9/d16/d55/d6f/l49 0 2024-09-21T04:15:14.982 INFO:tasks.workunit.client.1.smithi148.stdout:6/303: write d0/f20 [3244504,730] 0 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.977+0000 7f8e8a59c640 1 -- 172.21.15.110:0/1016343885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0ffc80 msgr2=0x7f8e8c105080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.977+0000 7f8e8a59c640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0ffc80 0x7f8e8c105080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.977+0000 7f8e8a59c640 1 -- 172.21.15.110:0/1016343885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009560 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.977+0000 7f8e8a59c640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004650 tx=0x7f8e7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.978+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e7c0051f0 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.978+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c1059b0 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.978+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c19d650 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.978+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e7c004bb0 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.979+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e7c0039e0 con 0x7f8e8c0fea80 2024-09-21T04:15:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.980+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c005280 con 0x7f8e8c0fea80 2024-09-21T04:15:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.980+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 23) v1 ==== 51738+0+0 (secure 0 0 0) 0x7f8e7c031630 con 0x7f8e8c0fea80 2024-09-21T04:15:14.984 INFO:tasks.workunit.client.1.smithi148.stdout:4/382: dwrite d4/f8 [0,4194304] 0 2024-09-21T04:15:14.985 INFO:tasks.workunit.client.1.smithi148.stdout:4/383: dread - d4/f51 zero size 2024-09-21T04:15:14.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/348: link db/d46/f5f db/d24/f70 0 2024-09-21T04:15:14.985 INFO:tasks.workunit.client.1.smithi148.stdout:8/378: sync 2024-09-21T04:15:14.985 INFO:tasks.workunit.client.1.smithi148.stdout:5/364: rename d3/d34/d78/c7c to d3/d34/d78/c87 0 2024-09-21T04:15:14.985 INFO:tasks.workunit.client.1.smithi148.stdout:5/365: stat d3/d14/d1f/d26/d2e/d56/f19 0 2024-09-21T04:15:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.981+0000 7f8e7b7fe640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 0x7f8e74041d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.981+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8e7c076060 con 0x7f8e8c0fea80 2024-09-21T04:15:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.983+0000 7f8e89d9b640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 0x7f8e74041d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:14.986 INFO:tasks.workunit.client.1.smithi148.stdout:3/338: read d3/d5/d6/d12/da/f1d [769371,79494] 0 2024-09-21T04:15:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:2/459: mknod d0/c90 0 2024-09-21T04:15:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:2/460: chown d0/d9/d16/f60 3565413 1 2024-09-21T04:15:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:2/461: chown d0/d9/dc/d13/d25/l7f 485188886 1 2024-09-21T04:15:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:6/304: mkdir d0/d59 0 2024-09-21T04:15:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:0/369: symlink d5/d6/d7/d22/l6f 0 2024-09-21T04:15:14.990 INFO:tasks.workunit.client.1.smithi148.stdout:9/329: write d5/fc [1320027,32904] 0 2024-09-21T04:15:14.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.990+0000 7f8e89d9b640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 0x7f8e74041d40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c1067b0 tx=0x7f8e80005f90 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:14.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:14.992+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f8e7c003b40 con 0x7f8e8c0fea80 2024-09-21T04:15:14.997 INFO:tasks.workunit.client.1.smithi148.stdout:8/379: write f5 [6819269,65848] 0 2024-09-21T04:15:14.998 INFO:tasks.workunit.client.1.smithi148.stdout:5/366: write d3/d14/d1a/f57 [2306665,116079] 0 2024-09-21T04:15:15.004 INFO:tasks.workunit.client.1.smithi148.stdout:7/396: dwrite d5/dc/d28/f53 [0,4194304] 0 2024-09-21T04:15:15.005 INFO:tasks.workunit.client.1.smithi148.stdout:4/384: dwrite d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:15.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/385: write d4/d12/d16/d29/f33 [752008,11317] 0 2024-09-21T04:15:15.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/386: readlink d4/lf 0 2024-09-21T04:15:15.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/387: read - d4/d12/d16/f62 zero size 2024-09-21T04:15:15.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/388: read - d4/d3a/f5f zero size 2024-09-21T04:15:15.011 INFO:tasks.workunit.client.1.smithi148.stdout:4/389: truncate d4/d3a/f53 557217 0 2024-09-21T04:15:15.056 INFO:tasks.workunit.client.1.smithi148.stdout:1/349: stat db/c52 0 2024-09-21T04:15:15.067 INFO:tasks.workunit.client.1.smithi148.stdout:6/305: mknod d0/d1a/d23/d32/c5a 0 2024-09-21T04:15:15.070 INFO:tasks.workunit.client.1.smithi148.stdout:0/370: rmdir d5/d6/d7/d11/d48/d66 0 2024-09-21T04:15:15.076 INFO:tasks.workunit.client.1.smithi148.stdout:9/330: truncate d5/d60/d28/f41 3441244 0 2024-09-21T04:15:15.076 INFO:tasks.workunit.client.1.smithi148.stdout:9/331: truncate d5/d2b/d32/f5a 1039399 0 2024-09-21T04:15:15.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/380: mkdir dc/d13/d76/d5d/d78 0 2024-09-21T04:15:15.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/381: write dc/d3e/f70 [633271,123619] 0 2024-09-21T04:15:15.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/382: write dc/f41 [79560,79976] 0 2024-09-21T04:15:15.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/306: dwrite d0/f21 [0,4194304] 0 2024-09-21T04:15:15.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/307: stat d0/d8/f24 0 2024-09-21T04:15:15.094 INFO:tasks.workunit.client.0.smithi110.stdout:0/660: sync 2024-09-21T04:15:15.095 INFO:tasks.workunit.client.0.smithi110.stdout:1/664: sync 2024-09-21T04:15:15.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/211: sync 2024-09-21T04:15:15.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/212: chown d2/l23 5338 1 2024-09-21T04:15:15.095 INFO:tasks.workunit.client.0.smithi110.stdout:3/365: sync 2024-09-21T04:15:15.095 INFO:tasks.workunit.client.0.smithi110.stdout:7/376: sync 2024-09-21T04:15:15.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/390: symlink d4/d12/d16/d26/l6e 0 2024-09-21T04:15:15.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/391: fsync d4/f8 0 2024-09-21T04:15:15.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/392: stat d4/d12/f65 0 2024-09-21T04:15:15.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/393: fdatasync d4/d12/f63 0 2024-09-21T04:15:15.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/394: chown d4/d23/f4a 38 1 2024-09-21T04:15:15.123 INFO:tasks.workunit.client.0.smithi110.stdout:1/665: dwrite d2/d9/d49/f99 [0,4194304] 0 2024-09-21T04:15:15.126 INFO:tasks.workunit.client.0.smithi110.stdout:0/661: dwrite d4/dd/d53/d66/f74 [0,4194304] 0 2024-09-21T04:15:15.126 INFO:tasks.workunit.client.0.smithi110.stdout:0/662: write d4/dd/d53/fb9 [117541,39357] 0 2024-09-21T04:15:15.128 INFO:tasks.workunit.client.0.smithi110.stdout:0/663: dread d4/dd/f32 [0,4194304] 0 2024-09-21T04:15:15.138 INFO:tasks.workunit.client.1.smithi148.stdout:4/395: dwrite d4/d12/d16/d29/f5a [0,4194304] 0 2024-09-21T04:15:15.143 INFO:tasks.workunit.client.0.smithi110.stdout:1/666: dwrite d2/d9/f6c [0,4194304] 0 2024-09-21T04:15:15.178 INFO:tasks.workunit.client.1.smithi148.stdout:0/371: mkdir d5/d6/d2a/d45/d70 0 2024-09-21T04:15:15.182 INFO:tasks.workunit.client.1.smithi148.stdout:9/332: creat d5/d1e/d37/d46/f7e x:0 0 0 2024-09-21T04:15:15.186 INFO:tasks.workunit.client.1.smithi148.stdout:8/383: rename dc/d13/d76/d42/f5c to dc/d43/d4c/d4f/f79 0 2024-09-21T04:15:15.189 INFO:tasks.workunit.client.1.smithi148.stdout:6/308: truncate d0/d1a/d23/f19 112926 0 2024-09-21T04:15:15.194 INFO:tasks.workunit.client.0.smithi110.stdout:2/213: rmdir d2 39 2024-09-21T04:15:15.196 INFO:tasks.workunit.client.1.smithi148.stdout:6/309: write d0/d8/f10 [3818859,62071] 0 2024-09-21T04:15:15.196 INFO:tasks.workunit.client.1.smithi148.stdout:6/310: truncate d0/d1a/d23/fb 5472153 0 2024-09-21T04:15:15.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/311: read d0/d1a/d23/d16/f2c [871628,11329] 0 2024-09-21T04:15:15.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/312: readlink d0/l2f 0 2024-09-21T04:15:15.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/313: fdatasync d0/d1a/d23/d16/d4f/f54 0 2024-09-21T04:15:15.206 INFO:tasks.workunit.client.0.smithi110.stdout:0/664: symlink d4/d1c/d62/d69/lc8 0 2024-09-21T04:15:15.207 INFO:tasks.workunit.client.1.smithi148.stdout:6/314: write d0/d1a/d23/d16/f30 [3632885,125325] 0 2024-09-21T04:15:15.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/396: mknod d4/d12/d2c/d2d/c6f 0 2024-09-21T04:15:15.212 INFO:tasks.workunit.client.1.smithi148.stdout:9/333: mknod d5/d1e/d37/d46/c7f 0 2024-09-21T04:15:15.212 INFO:tasks.workunit.client.1.smithi148.stdout:9/334: chown d5/d60/d28/d38/c40 39 1 2024-09-21T04:15:15.212 INFO:tasks.workunit.client.1.smithi148.stdout:8/384: symlink dc/d13/d28/l7a 0 2024-09-21T04:15:15.213 INFO:tasks.workunit.client.0.smithi110.stdout:2/214: dwrite d2/d1a/d1f/d2a/f21 [4194304,4194304] 0 2024-09-21T04:15:15.215 INFO:tasks.workunit.client.1.smithi148.stdout:4/397: write d4/f11 [1233951,99959] 0 2024-09-21T04:15:15.218 INFO:tasks.workunit.client.1.smithi148.stdout:6/315: rename d0/d8/d18/d4e to d0/d1a/d23/d16/d4f/d55/d5b 0 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/398: dread d4/d12/d16/d26/f3e [0,4194304] 0 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:6/316: write d0/d1a/d23/d16/f39 [354207,42709] 0 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/399: read d4/d12/f1b [1761361,20646] 0 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:6/317: chown d0/d59 1633935 1 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:9/335: mkdir d5/d2b/d32/d67/d80 0 2024-09-21T04:15:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/400: read d4/d3a/f53 [541471,96188] 0 2024-09-21T04:15:15.221 INFO:tasks.workunit.client.1.smithi148.stdout:6/318: write d0/d8/d18/f25 [359471,14329] 0 2024-09-21T04:15:15.221 INFO:tasks.workunit.client.1.smithi148.stdout:6/319: truncate d0/d1a/d23/d16/d4f/d55/f50 77957 0 2024-09-21T04:15:15.222 INFO:tasks.workunit.client.1.smithi148.stdout:8/385: dread dc/fe [0,4194304] 0 2024-09-21T04:15:15.222 INFO:tasks.workunit.client.1.smithi148.stdout:9/336: mkdir d5/d1f/d81 0 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/401: unlink d4/d12/d16/f62 0 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/402: write d4/d12/d16/f4f [145977,91688] 0 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/403: read - d4/d3a/f5f zero size 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/404: readlink d4/d12/d16/l1d 0 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/405: readlink d4/d12/d2c/d2d/l48 0 2024-09-21T04:15:15.227 INFO:tasks.workunit.client.1.smithi148.stdout:4/406: creat d4/d12/d16/d26/f70 x:0 0 0 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:9/337: write d5/d60/d28/d38/f52 [622303,85675] 0 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:9/338: chown d5/d60/d28/d2f/l5f 1 1 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:9/339: write d5/d1e/d37/d46/f7e [486915,14055] 0 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:9/340: creat d5/d2b/d32/f82 x:0 0 0 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:6/320: creat d0/d34/d38/d57/d58/f5c x:0 0 0 2024-09-21T04:15:15.230 INFO:tasks.workunit.client.1.smithi148.stdout:6/321: creat d0/d1a/d23/d16/f5d x:0 0 0 2024-09-21T04:15:15.231 INFO:tasks.workunit.client.1.smithi148.stdout:8/386: symlink dc/d13/d6a/l7b 0 2024-09-21T04:15:15.233 INFO:tasks.workunit.client.1.smithi148.stdout:4/407: symlink d4/d12/d16/l71 0 2024-09-21T04:15:15.237 INFO:tasks.workunit.client.1.smithi148.stdout:9/341: unlink d5/d60/d28/d38/c40 0 2024-09-21T04:15:15.238 INFO:tasks.workunit.client.1.smithi148.stdout:4/408: read d4/d12/d16/f1f [2116400,63607] 0 2024-09-21T04:15:15.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/322: getdents d0/d1a/d23/d32 0 2024-09-21T04:15:15.244 INFO:tasks.workunit.client.1.smithi148.stdout:4/409: dread d4/d23/f4a [0,4194304] 0 2024-09-21T04:15:15.250 INFO:tasks.workunit.client.1.smithi148.stdout:9/342: dwrite d5/fc [4194304,4194304] 0 2024-09-21T04:15:15.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/343: mkdir d5/d60/d4c/d83 0 2024-09-21T04:15:15.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/344: fdatasync d5/d60/f3f 0 2024-09-21T04:15:15.256 INFO:tasks.workunit.client.1.smithi148.stdout:9/345: dread d5/fe [0,4194304] 0 2024-09-21T04:15:15.263 INFO:tasks.workunit.client.1.smithi148.stdout:9/346: dread d5/d2b/f4f [0,4194304] 0 2024-09-21T04:15:15.266 INFO:tasks.workunit.client.1.smithi148.stdout:7/397: rmdir d5/d14 39 2024-09-21T04:15:15.272 INFO:tasks.workunit.client.0.smithi110.stdout:7/377: mknod dd/d14/c74 0 2024-09-21T04:15:15.272 INFO:tasks.workunit.client.0.smithi110.stdout:7/378: creat dd/d1c/f75 x:0 0 0 2024-09-21T04:15:15.272 INFO:tasks.workunit.client.0.smithi110.stdout:7/379: read - dd/d14/d17/f54 zero size 2024-09-21T04:15:15.280 INFO:tasks.workunit.client.1.smithi148.stdout:5/367: mkdir d3/d88 0 2024-09-21T04:15:15.283 INFO:tasks.workunit.client.1.smithi148.stdout:5/368: write d3/d14/d1f/d26/d2e/d56/f19 [2244161,25762] 0 2024-09-21T04:15:15.283 INFO:tasks.workunit.client.1.smithi148.stdout:5/369: write d3/d14/d41/f49 [761122,62908] 0 2024-09-21T04:15:15.284 INFO:tasks.workunit.client.1.smithi148.stdout:9/347: dwrite d5/d1e/d37/d46/f7e [0,4194304] 0 2024-09-21T04:15:15.286 INFO:tasks.workunit.client.0.smithi110.stdout:7/380: read dd/d1c/d39/f3c [3052546,7238] 0 2024-09-21T04:15:15.287 INFO:tasks.workunit.client.1.smithi148.stdout:5/370: unlink d3/f7 0 2024-09-21T04:15:15.289 INFO:tasks.workunit.client.1.smithi148.stdout:7/398: dwrite d5/dc/d28/f49 [0,4194304] 0 2024-09-21T04:15:15.290 INFO:tasks.workunit.client.0.smithi110.stdout:7/381: getdents dd/d1c/d39 0 2024-09-21T04:15:15.291 INFO:tasks.workunit.client.0.smithi110.stdout:7/382: dread - dd/d4d/f68 zero size 2024-09-21T04:15:15.291 INFO:tasks.workunit.client.0.smithi110.stdout:7/383: write dd/f3e [658677,63645] 0 2024-09-21T04:15:15.291 INFO:tasks.workunit.client.0.smithi110.stdout:7/384: write dd/d1c/d39/f61 [2081297,93448] 0 2024-09-21T04:15:15.291 INFO:tasks.workunit.client.0.smithi110.stdout:7/385: chown dd/d14/c42 151 1 2024-09-21T04:15:15.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/348: mknod d5/d39/c84 0 2024-09-21T04:15:15.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/349: dread - d5/d60/d4c/f6b zero size 2024-09-21T04:15:15.295 INFO:tasks.workunit.client.1.smithi148.stdout:5/371: unlink d3/f5b 0 2024-09-21T04:15:15.295 INFO:tasks.workunit.client.1.smithi148.stdout:5/372: readlink d3/d34/d3c/l63 0 2024-09-21T04:15:15.295 INFO:tasks.workunit.client.1.smithi148.stdout:3/339: rmdir d3/d5/d6/d12/d43 39 2024-09-21T04:15:15.296 INFO:tasks.workunit.client.0.smithi110.stdout:7/386: getdents dd/d1c/d39/d4a 0 2024-09-21T04:15:15.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/373: creat d3/d14/d1f/d26/d2e/d3d/f89 x:0 0 0 2024-09-21T04:15:15.300 INFO:tasks.workunit.client.1.smithi148.stdout:2/462: mknod d0/d9/dc/d13/d25/c91 0 2024-09-21T04:15:15.301 INFO:tasks.workunit.client.0.smithi110.stdout:7/387: creat dd/d1c/d39/f76 x:0 0 0 2024-09-21T04:15:15.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/340: symlink d3/d5/d6/d12/da/d23/l6b 0 2024-09-21T04:15:15.304 INFO:tasks.workunit.client.0.smithi110.stdout:7/388: getdents dd/d14/d17 0 2024-09-21T04:15:15.304 INFO:tasks.workunit.client.0.smithi110.stdout:7/389: getdents dd/d14/d17 0 2024-09-21T04:15:15.308 INFO:tasks.workunit.client.1.smithi148.stdout:3/341: creat d3/d5/d6/d12/d15/f6c x:0 0 0 2024-09-21T04:15:15.308 INFO:tasks.workunit.client.1.smithi148.stdout:3/342: chown d3/ce 54789729 1 2024-09-21T04:15:15.310 INFO:tasks.workunit.client.0.smithi110.stdout:7/390: creat dd/d1c/d39/d4a/f77 x:0 0 0 2024-09-21T04:15:15.311 INFO:tasks.workunit.client.0.smithi110.stdout:7/391: symlink dd/d1c/d39/d56/l78 0 2024-09-21T04:15:15.316 INFO:tasks.workunit.client.0.smithi110.stdout:7/392: dread dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:15.316 INFO:tasks.workunit.client.0.smithi110.stdout:7/393: chown dd/d14/d17/c5c 30083614 1 2024-09-21T04:15:15.317 INFO:tasks.workunit.client.1.smithi148.stdout:3/343: dread d3/d5/d6/d12/da/d52/f4a [0,4194304] 0 2024-09-21T04:15:15.319 INFO:tasks.workunit.client.1.smithi148.stdout:3/344: mknod d3/d5/d6/d12/d43/c6d 0 2024-09-21T04:15:15.321 INFO:tasks.workunit.client.1.smithi148.stdout:9/350: dwrite d5/fc [0,4194304] 0 2024-09-21T04:15:15.321 INFO:tasks.workunit.client.1.smithi148.stdout:5/374: dwrite d3/d14/d1f/d26/d2e/d3d/f89 [0,4194304] 0 2024-09-21T04:15:15.322 INFO:tasks.workunit.client.0.smithi110.stdout:7/394: fsync dd/f3e 0 2024-09-21T04:15:15.322 INFO:tasks.workunit.client.0.smithi110.stdout:7/395: fdatasync dd/d1c/d39/f76 0 2024-09-21T04:15:15.322 INFO:tasks.workunit.client.0.smithi110.stdout:7/396: chown dd/d4d/f71 2572957 1 2024-09-21T04:15:15.325 INFO:tasks.workunit.client.0.smithi110.stdout:7/397: symlink dd/d1c/d39/d4a/l79 0 2024-09-21T04:15:15.335 INFO:tasks.workunit.client.1.smithi148.stdout:8/387: getdents dc/d13/d28 0 2024-09-21T04:15:15.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/388: write dc/fe [3245542,78040] 0 2024-09-21T04:15:15.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/389: chown dc/d3e/c63 554379962 1 2024-09-21T04:15:15.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/390: getdents dc/d13/d76/d5d/d78 0 2024-09-21T04:15:15.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/375: dwrite d3/d14/d1f/f7e [0,4194304] 0 2024-09-21T04:15:15.347 INFO:tasks.workunit.client.1.smithi148.stdout:0/372: link d5/d6/d7/d18/f32 d5/d6/d7/d11/f71 0 2024-09-21T04:15:15.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/391: dread dc/f68 [0,4194304] 0 2024-09-21T04:15:15.351 INFO:tasks.workunit.client.1.smithi148.stdout:4/410: rmdir d4/d12/d16 39 2024-09-21T04:15:15.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/392: mkdir dc/d13/d76/d7c 0 2024-09-21T04:15:15.353 INFO:tasks.workunit.client.1.smithi148.stdout:8/393: link dc/f14 dc/f7d 0 2024-09-21T04:15:15.354 INFO:tasks.workunit.client.1.smithi148.stdout:8/394: stat dc/d13/d76/f5b 0 2024-09-21T04:15:15.354 INFO:tasks.workunit.client.1.smithi148.stdout:5/376: dwrite d3/d14/d1f/f50 [0,4194304] 0 2024-09-21T04:15:15.357 INFO:tasks.workunit.client.1.smithi148.stdout:0/373: dread d5/d6/d7/d11/f71 [0,4194304] 0 2024-09-21T04:15:15.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/377: creat d3/d14/d1f/d26/d2e/d56/f8a x:0 0 0 2024-09-21T04:15:15.359 INFO:tasks.workunit.client.0.smithi110.stdout:3/366: rmdir d0/d12 39 2024-09-21T04:15:15.361 INFO:tasks.workunit.client.1.smithi148.stdout:0/374: symlink d5/d6/d2a/d3f/d6d/l72 0 2024-09-21T04:15:15.362 INFO:tasks.workunit.client.1.smithi148.stdout:4/411: dread d4/d12/d2c/f32 [4194304,4194304] 0 2024-09-21T04:15:15.362 INFO:tasks.workunit.client.1.smithi148.stdout:4/412: chown d4/d23/d2a 60 1 2024-09-21T04:15:15.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/367: unlink d0/d12/d16/l6b 0 2024-09-21T04:15:15.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/368: write d0/d51/f67 [115557,19386] 0 2024-09-21T04:15:15.368 INFO:tasks.workunit.client.0.smithi110.stdout:0/665: rename d4/dd/d53/d66/f81 to d4/d47/fc9 0 2024-09-21T04:15:15.368 INFO:tasks.workunit.client.0.smithi110.stdout:2/215: rename d2/d1a/d1f to d2/d1a/d1f/d29/d19/d2e/d3c 22 2024-09-21T04:15:15.368 INFO:tasks.workunit.client.0.smithi110.stdout:2/216: truncate d2/d1a/d1f/d29/d19/d36/f2b 1440921 0 2024-09-21T04:15:15.369 INFO:tasks.workunit.client.0.smithi110.stdout:2/217: write d2/d1a/d1f/d29/d19/d36/f3a [984649,92812] 0 2024-09-21T04:15:15.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/666: rename d4/f31 to d4/dd/d10/d29/d45/fca 0 2024-09-21T04:15:15.369 INFO:tasks.workunit.client.0.smithi110.stdout:0/667: readlink d4/dd/l7d 0 2024-09-21T04:15:15.371 INFO:tasks.workunit.client.0.smithi110.stdout:2/218: link d2/d1a/c34 d2/d1a/d1f/d29/c3d 0 2024-09-21T04:15:15.376 INFO:tasks.workunit.client.0.smithi110.stdout:2/219: unlink d2/d1a/d1f/d2a/f22 0 2024-09-21T04:15:15.376 INFO:tasks.workunit.client.0.smithi110.stdout:2/220: write d2/d1a/d1f/d29/fd [4428639,90051] 0 2024-09-21T04:15:15.377 INFO:tasks.workunit.client.0.smithi110.stdout:2/221: chown d2/d1a/d1f/l10 1210145674 1 2024-09-21T04:15:15.378 INFO:tasks.workunit.client.1.smithi148.stdout:9/351: mknod d5/c85 0 2024-09-21T04:15:15.378 INFO:tasks.workunit.client.1.smithi148.stdout:9/352: fdatasync f0 0 2024-09-21T04:15:15.379 INFO:tasks.workunit.client.0.smithi110.stdout:2/222: mknod d2/d1a/d1f/d29/c3e 0 2024-09-21T04:15:15.381 INFO:tasks.workunit.client.0.smithi110.stdout:7/398: rename dd/d1c/d39/d5d to dd/d48/d55/d7a 0 2024-09-21T04:15:15.382 INFO:tasks.workunit.client.0.smithi110.stdout:7/399: write dd/d1c/f34 [703812,98903] 0 2024-09-21T04:15:15.382 INFO:tasks.workunit.client.0.smithi110.stdout:7/400: chown dd/d1c/d39/d56/d66 447148172 1 2024-09-21T04:15:15.382 INFO:tasks.workunit.client.0.smithi110.stdout:7/401: write dd/d1c/d39/f3b [1573740,3032] 0 2024-09-21T04:15:15.385 INFO:tasks.workunit.client.1.smithi148.stdout:9/353: rmdir d5/d60/d4c/d83 0 2024-09-21T04:15:15.386 INFO:tasks.workunit.client.1.smithi148.stdout:4/413: dwrite d4/d12/d2c/f57 [0,4194304] 0 2024-09-21T04:15:15.386 INFO:tasks.workunit.client.0.smithi110.stdout:3/369: dwrite d0/d12/d16/d1a/f3b [4194304,4194304] 0 2024-09-21T04:15:15.387 INFO:tasks.workunit.client.0.smithi110.stdout:7/402: unlink dd/c44 0 2024-09-21T04:15:15.388 INFO:tasks.workunit.client.1.smithi148.stdout:9/354: mknod d5/d2b/d32/d67/c86 0 2024-09-21T04:15:15.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/414: mknod d4/d12/d16/c72 0 2024-09-21T04:15:15.393 INFO:tasks.workunit.client.0.smithi110.stdout:2/223: dread d2/d1a/d1f/d29/d19/d36/f24 [0,4194304] 0 2024-09-21T04:15:15.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/415: dread d4/d12/d2c/f5c [0,4194304] 0 2024-09-21T04:15:15.398 INFO:tasks.workunit.client.1.smithi148.stdout:6/323: rename d0/d1a/d23/fb to d0/d34/d38/d43/f5e 0 2024-09-21T04:15:15.400 INFO:tasks.workunit.client.0.smithi110.stdout:7/403: link dd/d1c/d39/d4a/d52/c69 dd/c7b 0 2024-09-21T04:15:15.402 INFO:tasks.workunit.client.1.smithi148.stdout:7/399: rename d5/d14/f3c to d5/d14/d25/d79/f85 0 2024-09-21T04:15:15.404 INFO:tasks.workunit.client.0.smithi110.stdout:3/370: dwrite d0/d12/d16/d1a/f39 [4194304,4194304] 0 2024-09-21T04:15:15.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/324: creat d0/d1a/d23/d53/f5f x:0 0 0 2024-09-21T04:15:15.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/325: fdatasync d0/d8/f49 0 2024-09-21T04:15:15.407 INFO:tasks.workunit.client.0.smithi110.stdout:7/404: truncate dd/d4d/f59 2672921 0 2024-09-21T04:15:15.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/375: rename d5/d6/d2a/d2b/f36 to d5/d6/d7/d22/f73 0 2024-09-21T04:15:15.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/376: write d5/d6/d7/d22/f59 [385212,40121] 0 2024-09-21T04:15:15.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.410+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6c000bf0 con 0x7f8e7403f880 2024-09-21T04:15:15.414 INFO:tasks.workunit.client.1.smithi148.stdout:7/400: mkdir d5/d14/d25/d79/d86 0 2024-09-21T04:15:15.416 INFO:tasks.workunit.client.0.smithi110.stdout:7/405: creat dd/d1c/d39/d6d/f7c x:0 0 0 2024-09-21T04:15:15.416 INFO:tasks.workunit.client.1.smithi148.stdout:0/377: dread d5/d6/d3e/f26 [0,4194304] 0 2024-09-21T04:15:15.417 INFO:tasks.workunit.client.1.smithi148.stdout:6/326: getdents d0/d1a/d23/d27 0 2024-09-21T04:15:15.417 INFO:tasks.workunit.client.1.smithi148.stdout:6/327: creat d0/d8/d18/f60 x:0 0 0 2024-09-21T04:15:15.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/224: rename d2/d1a/d1f/d29/d19/d36 to d2/d1a/d1f/d3f 0 2024-09-21T04:15:15.419 INFO:tasks.workunit.client.1.smithi148.stdout:9/355: rename d5/d60/d28/f41 to d5/d60/d28/d73/f87 0 2024-09-21T04:15:15.420 INFO:tasks.workunit.client.1.smithi148.stdout:9/356: creat d5/d1e/d37/f88 x:0 0 0 2024-09-21T04:15:15.421 INFO:tasks.workunit.client.1.smithi148.stdout:0/378: link d5/c67 d5/d6/d2a/d3f/c74 0 2024-09-21T04:15:15.421 INFO:tasks.workunit.client.1.smithi148.stdout:9/357: dread d5/d2b/d32/f5a [0,4194304] 0 2024-09-21T04:15:15.421 INFO:tasks.workunit.client.0.smithi110.stdout:3/371: rename d0/d51/f7e to d0/d12/d16/d1a/f86 0 2024-09-21T04:15:15.424 INFO:tasks.workunit.client.0.smithi110.stdout:2/225: symlink d2/d1a/d1f/d29/d19/l40 0 2024-09-21T04:15:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/328: creat d0/d1a/d23/f61 x:0 0 0 2024-09-21T04:15:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/329: write d0/d1a/d23/f12 [3555356,22341] 0 2024-09-21T04:15:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/330: chown d0/d8/c9 1478460 1 2024-09-21T04:15:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/331: fdatasync d0/d1a/d23/ff 0 2024-09-21T04:15:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/332: readlink d0/l47 0 2024-09-21T04:15:15.427 INFO:tasks.workunit.client.0.smithi110.stdout:3/372: mkdir d0/d22/d29/d87 0 2024-09-21T04:15:15.427 INFO:tasks.workunit.client.0.smithi110.stdout:3/373: write d0/d12/d3c/d5a/d70/f84 [94394,44981] 0 2024-09-21T04:15:15.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/226: getdents d2/d1a/d1f/d3f 0 2024-09-21T04:15:15.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/227: write d2/f16 [2005586,49523] 0 2024-09-21T04:15:15.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/228: dread - d2/f35 zero size 2024-09-21T04:15:15.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/229: write d2/d1a/d1f/d2a/f1b [1623242,18407] 0 2024-09-21T04:15:15.428 INFO:tasks.workunit.client.1.smithi148.stdout:7/401: dwrite d5/d14/d25/f42 [0,4194304] 0 2024-09-21T04:15:15.429 INFO:tasks.workunit.client.1.smithi148.stdout:4/416: dwrite d4/d23/f2e [0,4194304] 0 2024-09-21T04:15:15.429 INFO:tasks.workunit.client.1.smithi148.stdout:7/402: write d5/d14/d25/d79/f85 [1302997,113580] 0 2024-09-21T04:15:15.431 INFO:tasks.workunit.client.0.smithi110.stdout:3/374: unlink d0/l26 0 2024-09-21T04:15:15.432 INFO:tasks.workunit.client.0.smithi110.stdout:0/668: unlink d4/d1c/f24 0 2024-09-21T04:15:15.434 INFO:tasks.workunit.client.1.smithi148.stdout:0/379: creat d5/d6/d2a/d45/d70/f75 x:0 0 0 2024-09-21T04:15:15.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.433+0000 7f8e7b7fe640 1 -- 172.21.15.110:0/1016343885 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f8e6c000bf0 con 0x7f8e7403f880 2024-09-21T04:15:15.437 INFO:tasks.workunit.client.0.smithi110.stdout:2/230: rename d2/d1a/d1f/d29/d19/d2e to d2/d1a/d1f/d2a/d41 0 2024-09-21T04:15:15.437 INFO:tasks.workunit.client.0.smithi110.stdout:2/231: stat d2/d1a/d1f/d3f/f1d 0 2024-09-21T04:15:15.438 INFO:tasks.workunit.client.1.smithi148.stdout:9/358: symlink d5/d60/d28/l89 0 2024-09-21T04:15:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.436+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 msgr2=0x7f8e74041d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.436+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 0x7f8e74041d40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c1067b0 tx=0x7f8e80005f90 comp rx=0 tx=0).stop 2024-09-21T04:15:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.436+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 msgr2=0x7f8e8c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.436+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004650 tx=0x7f8e7c0046c0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 shutdown_connections 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f8e7403f880 0x7f8e74041d40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8c0ffc80 0x7f8e8c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 --2- 172.21.15.110:0/1016343885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e8c0fea80 0x7f8e8c106a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 >> 172.21.15.110:0/1016343885 conn(0x7f8e8c0fa210 msgr2=0x7f8e8c102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 shutdown_connections 2024-09-21T04:15:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.437+0000 7f8e90ce4640 1 -- 172.21.15.110:0/1016343885 wait complete. 2024-09-21T04:15:15.442 INFO:tasks.workunit.client.1.smithi148.stdout:6/333: mkdir d0/d1a/d23/d27/d62 0 2024-09-21T04:15:15.451 INFO:tasks.workunit.client.1.smithi148.stdout:0/380: link d5/c54 d5/d6/c76 0 2024-09-21T04:15:15.451 INFO:tasks.workunit.client.1.smithi148.stdout:9/359: creat d5/f8a x:0 0 0 2024-09-21T04:15:15.454 INFO:tasks.workunit.client.1.smithi148.stdout:6/334: mkdir d0/d34/d38/d43/d63 0 2024-09-21T04:15:15.454 INFO:tasks.workunit.client.1.smithi148.stdout:6/335: rename d0/d1a/d23 to d0/d1a/d23/d32/d64 22 2024-09-21T04:15:15.454 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:15:15.455 INFO:tasks.workunit.client.1.smithi148.stdout:7/403: getdents d5/dc 0 2024-09-21T04:15:15.455 INFO:tasks.workunit.client.1.smithi148.stdout:7/404: creat d5/dc/d28/f87 x:0 0 0 2024-09-21T04:15:15.455 INFO:tasks.workunit.client.1.smithi148.stdout:7/405: fdatasync d5/d8/f1b 0 2024-09-21T04:15:15.457 INFO:tasks.workunit.client.1.smithi148.stdout:0/381: mknod d5/d6/d7/d18/c77 0 2024-09-21T04:15:15.458 INFO:tasks.workunit.client.1.smithi148.stdout:9/360: mknod d5/d1e/d37/c8b 0 2024-09-21T04:15:15.459 INFO:tasks.workunit.client.1.smithi148.stdout:6/336: truncate d0/d34/d38/f3a 164030 0 2024-09-21T04:15:15.464 INFO:tasks.workunit.client.1.smithi148.stdout:7/406: mknod d5/d8/d6f/c88 0 2024-09-21T04:15:15.465 INFO:tasks.workunit.client.1.smithi148.stdout:7/407: write d5/d8/f62 [589948,130448] 0 2024-09-21T04:15:15.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/417: dwrite d4/d23/f2b [0,4194304] 0 2024-09-21T04:15:15.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/418: readlink d4/d12/d2c/d2d/l69 0 2024-09-21T04:15:15.465 INFO:tasks.workunit.client.0.smithi110.stdout:0/669: dwrite d4/dd/d10/f99 [0,4194304] 0 2024-09-21T04:15:15.466 INFO:tasks.workunit.client.1.smithi148.stdout:1/350: sync 2024-09-21T04:15:15.466 INFO:tasks.workunit.client.1.smithi148.stdout:1/351: chown db/dd/l3f 37039 1 2024-09-21T04:15:15.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/361: creat d5/d1f/d81/f8c x:0 0 0 2024-09-21T04:15:15.467 INFO:tasks.workunit.client.0.smithi110.stdout:3/375: getdents d0/d12/d16 0 2024-09-21T04:15:15.467 INFO:tasks.workunit.client.0.smithi110.stdout:2/232: dwrite d2/d1a/d1f/d29/fd [0,4194304] 0 2024-09-21T04:15:15.467 INFO:tasks.workunit.client.1.smithi148.stdout:4/419: dread d4/d3a/f53 [0,4194304] 0 2024-09-21T04:15:15.468 INFO:tasks.workunit.client.0.smithi110.stdout:6/370: sync 2024-09-21T04:15:15.469 INFO:tasks.workunit.client.0.smithi110.stdout:8/453: sync 2024-09-21T04:15:15.479 INFO:tasks.workunit.client.1.smithi148.stdout:9/362: dread d5/ff [4194304,4194304] 0 2024-09-21T04:15:15.482 INFO:tasks.workunit.client.1.smithi148.stdout:9/363: write d5/ff [7179640,45735] 0 2024-09-21T04:15:15.482 INFO:tasks.workunit.client.1.smithi148.stdout:9/364: creat d5/d1e/d37/f8d x:0 0 0 2024-09-21T04:15:15.482 INFO:tasks.workunit.client.1.smithi148.stdout:9/365: chown d5/d1e/d37/f4d 34248 1 2024-09-21T04:15:15.483 INFO:tasks.workunit.client.1.smithi148.stdout:6/337: dwrite d0/d34/d38/d57/d58/f5c [0,4194304] 0 2024-09-21T04:15:15.483 INFO:tasks.workunit.client.1.smithi148.stdout:6/338: dread - d0/d1a/d23/d16/f5d zero size 2024-09-21T04:15:15.486 INFO:tasks.workunit.client.1.smithi148.stdout:4/420: rename d4/d12/f1b to d4/d23/d2a/f73 0 2024-09-21T04:15:15.486 INFO:tasks.workunit.client.1.smithi148.stdout:4/421: read - d4/d12/f63 zero size 2024-09-21T04:15:15.488 INFO:tasks.workunit.client.0.smithi110.stdout:0/670: dwrite d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:15.488 INFO:tasks.workunit.client.0.smithi110.stdout:0/671: readlink d4/d47/l87 0 2024-09-21T04:15:15.489 INFO:tasks.workunit.client.0.smithi110.stdout:3/376: creat d0/f88 x:0 0 0 2024-09-21T04:15:15.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/408: dwrite d5/d8/d6f/d7d/f68 [0,4194304] 0 2024-09-21T04:15:15.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/409: fdatasync d5/dc/d28/f29 0 2024-09-21T04:15:15.493 INFO:tasks.workunit.client.1.smithi148.stdout:9/366: mkdir d5/d60/d28/d2f/d8e 0 2024-09-21T04:15:15.493 INFO:tasks.workunit.client.0.smithi110.stdout:6/371: mkdir d4/d78 0 2024-09-21T04:15:15.494 INFO:tasks.workunit.client.0.smithi110.stdout:6/372: dread - d4/f6c zero size 2024-09-21T04:15:15.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/233: dwrite d2/d1a/d1f/f38 [0,4194304] 0 2024-09-21T04:15:15.497 INFO:tasks.workunit.client.0.smithi110.stdout:0/672: creat d4/dd/fcb x:0 0 0 2024-09-21T04:15:15.499 INFO:tasks.workunit.client.0.smithi110.stdout:3/377: symlink d0/d12/d2a/d80/l89 0 2024-09-21T04:15:15.502 INFO:tasks.workunit.client.0.smithi110.stdout:2/234: dread d2/f16 [0,4194304] 0 2024-09-21T04:15:15.502 INFO:tasks.workunit.client.0.smithi110.stdout:2/235: chown d2/d1a/d1f/d2a/d41 44 1 2024-09-21T04:15:15.505 INFO:tasks.workunit.client.0.smithi110.stdout:6/373: creat d4/f79 x:0 0 0 2024-09-21T04:15:15.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/339: symlink d0/l65 0 2024-09-21T04:15:15.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/340: fsync d0/d1a/d23/f3d 0 2024-09-21T04:15:15.508 INFO:tasks.workunit.client.1.smithi148.stdout:4/422: symlink d4/d12/d16/d29/d61/d6d/l74 0 2024-09-21T04:15:15.508 INFO:tasks.workunit.client.1.smithi148.stdout:4/423: chown d4/fb 19082 1 2024-09-21T04:15:15.509 INFO:tasks.workunit.client.1.smithi148.stdout:4/424: stat d4/d23/f2b 0 2024-09-21T04:15:15.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/367: rename d5/d60/d28/f2c to d5/d60/d28/d38/f8f 0 2024-09-21T04:15:15.515 INFO:tasks.workunit.client.1.smithi148.stdout:6/341: creat d0/d1a/d23/d16/d4f/d55/f66 x:0 0 0 2024-09-21T04:15:15.515 INFO:tasks.workunit.client.1.smithi148.stdout:6/342: dread - d0/d1a/d23/d16/f52 zero size 2024-09-21T04:15:15.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/425: mkdir d4/d12/d75 0 2024-09-21T04:15:15.518 INFO:tasks.workunit.client.1.smithi148.stdout:9/368: creat d5/d1e/f90 x:0 0 0 2024-09-21T04:15:15.518 INFO:tasks.workunit.client.1.smithi148.stdout:9/369: fdatasync d5/d1e/d37/f8d 0 2024-09-21T04:15:15.519 INFO:tasks.workunit.client.1.smithi148.stdout:6/343: rmdir d0/d1a/d23/d32 39 2024-09-21T04:15:15.526 INFO:tasks.workunit.client.0.smithi110.stdout:8/454: rmdir d0 39 2024-09-21T04:15:15.526 INFO:tasks.workunit.client.0.smithi110.stdout:8/455: fsync d0/d7/d13/d4b/f28 0 2024-09-21T04:15:15.527 INFO:tasks.workunit.client.1.smithi148.stdout:9/370: rmdir d5/d2b 39 2024-09-21T04:15:15.528 INFO:tasks.workunit.client.0.smithi110.stdout:3/378: dwrite d0/d12/d16/d1a/d5c/f60 [0,4194304] 0 2024-09-21T04:15:15.529 INFO:tasks.workunit.client.0.smithi110.stdout:0/673: unlink d4/dd/d10/d29/d75/l80 0 2024-09-21T04:15:15.531 INFO:tasks.workunit.client.0.smithi110.stdout:2/236: link d2/d1a/d1f/d29/c1e d2/d1a/c42 0 2024-09-21T04:15:15.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/426: dwrite d4/d12/d16/d29/d61/f4b [0,4194304] 0 2024-09-21T04:15:15.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/427: fsync d4/d23/d2a/f36 0 2024-09-21T04:15:15.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/428: creat d4/d12/d2c/d2d/f76 x:0 0 0 2024-09-21T04:15:15.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/429: write d4/fb [1756664,128092] 0 2024-09-21T04:15:15.537 INFO:tasks.workunit.client.1.smithi148.stdout:4/430: write d4/d12/d16/d29/f3d [981167,61102] 0 2024-09-21T04:15:15.537 INFO:tasks.workunit.client.1.smithi148.stdout:4/431: chown d4/f51 3103450 1 2024-09-21T04:15:15.538 INFO:tasks.workunit.client.0.smithi110.stdout:8/456: truncate d0/d7/d23/f33 502068 0 2024-09-21T04:15:15.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/344: dwrite d0/d34/d38/d57/d58/f5c [0,4194304] 0 2024-09-21T04:15:15.545 INFO:tasks.workunit.client.0.smithi110.stdout:3/379: symlink d0/d12/d3c/d5a/d70/l8a 0 2024-09-21T04:15:15.547 INFO:tasks.workunit.client.0.smithi110.stdout:2/237: truncate d2/d1a/d1f/d3f/f1d 3813544 0 2024-09-21T04:15:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/345: creat d0/d1a/d23/d27/d62/f67 x:0 0 0 2024-09-21T04:15:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/346: readlink d0/d1a/d23/d32/l41 0 2024-09-21T04:15:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/347: chown d0/d34/d38/d43 16209332 1 2024-09-21T04:15:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/348: chown d0/d1a/d23/d16/f1f 234112 1 2024-09-21T04:15:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/349: stat d0/d8/c9 0 2024-09-21T04:15:15.553 INFO:tasks.workunit.client.0.smithi110.stdout:6/374: rename d4/d12/d16/d48 to d4/d12/d16/d66/d7a 0 2024-09-21T04:15:15.554 INFO:tasks.workunit.client.0.smithi110.stdout:8/457: mknod d0/d7/d23/c9a 0 2024-09-21T04:15:15.555 INFO:tasks.workunit.client.0.smithi110.stdout:0/674: dwrite d4/d1c/d62/f36 [0,4194304] 0 2024-09-21T04:15:15.557 INFO:tasks.workunit.client.1.smithi148.stdout:6/350: unlink d0/d8/c11 0 2024-09-21T04:15:15.558 INFO:tasks.workunit.client.0.smithi110.stdout:2/238: creat d2/d1a/d1f/d3f/f43 x:0 0 0 2024-09-21T04:15:15.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/351: dread d0/d1a/d23/d16/f2c [0,4194304] 0 2024-09-21T04:15:15.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/352: write d0/d1a/d23/f14 [1439812,43127] 0 2024-09-21T04:15:15.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/353: write d0/d1a/d23/d16/d4f/d55/f51 [1026455,92426] 0 2024-09-21T04:15:15.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/354: fsync d0/d34/d38/f3a 0 2024-09-21T04:15:15.560 INFO:tasks.workunit.client.1.smithi148.stdout:4/432: dwrite d4/d12/f41 [0,4194304] 0 2024-09-21T04:15:15.562 INFO:tasks.workunit.client.0.smithi110.stdout:3/380: dwrite d0/d12/d3c/f48 [0,4194304] 0 2024-09-21T04:15:15.563 INFO:tasks.workunit.client.0.smithi110.stdout:8/458: mkdir d0/d7/d23/d82/d85/d9b 0 2024-09-21T04:15:15.564 INFO:tasks.workunit.client.0.smithi110.stdout:0/675: creat d4/d47/fcc x:0 0 0 2024-09-21T04:15:15.564 INFO:tasks.workunit.client.0.smithi110.stdout:0/676: write d4/dd/d10/f70 [1177256,48637] 0 2024-09-21T04:15:15.564 INFO:tasks.workunit.client.1.smithi148.stdout:2/463: sync 2024-09-21T04:15:15.566 INFO:tasks.workunit.client.0.smithi110.stdout:6/375: getdents d4/d12/d17/d33 0 2024-09-21T04:15:15.569 INFO:tasks.workunit.client.1.smithi148.stdout:4/433: dread d4/d23/d2a/f73 [0,4194304] 0 2024-09-21T04:15:15.571 INFO:tasks.workunit.client.0.smithi110.stdout:2/239: write d2/d1a/d1f/f38 [1180454,14705] 0 2024-09-21T04:15:15.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/464: mknod d0/d2b/d48/c92 0 2024-09-21T04:15:15.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/465: write d0/d2b/d48/f5c [97535,49676] 0 2024-09-21T04:15:15.578 INFO:tasks.workunit.client.1.smithi148.stdout:4/434: symlink d4/d12/d16/d29/l77 0 2024-09-21T04:15:15.580 INFO:tasks.workunit.client.0.smithi110.stdout:6/376: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:15.587 INFO:tasks.workunit.client.0.smithi110.stdout:0/677: creat d4/dd/d10/d8a/fcd x:0 0 0 2024-09-21T04:15:15.588 INFO:tasks.workunit.client.1.smithi148.stdout:2/466: read d0/d2b/f7d [3088280,117840] 0 2024-09-21T04:15:15.588 INFO:tasks.workunit.client.1.smithi148.stdout:2/467: dread - d0/d9/dc/d13/d25/f89 zero size 2024-09-21T04:15:15.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/435: write d4/d12/f5b [2835848,40077] 0 2024-09-21T04:15:15.592 INFO:tasks.workunit.client.1.smithi148.stdout:2/468: creat d0/d9/d16/d55/d71/f93 x:0 0 0 2024-09-21T04:15:15.594 INFO:tasks.workunit.client.0.smithi110.stdout:2/240: truncate d2/d1a/d1f/d29/fd 2437355 0 2024-09-21T04:15:15.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 -- 172.21.15.110:0/3039883181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c1074b0 msgr2=0x7fc02c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:15.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 --2- 172.21.15.110:0/3039883181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c1074b0 0x7fc02c1098a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc02400ac70 tx=0x7fc02402bac0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 -- 172.21.15.110:0/3039883181 shutdown_connections 2024-09-21T04:15:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 --2- 172.21.15.110:0/3039883181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c109de0 0x7fc02c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 --2- 172.21.15.110:0/3039883181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c1074b0 0x7fc02c1098a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 -- 172.21.15.110:0/3039883181 >> 172.21.15.110:0/3039883181 conn(0x7fc02c06cf90 msgr2=0x7fc02c06f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 -- 172.21.15.110:0/3039883181 shutdown_connections 2024-09-21T04:15:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.592+0000 7fc0333f3640 1 -- 172.21.15.110:0/3039883181 wait complete. 2024-09-21T04:15:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.593+0000 7fc0333f3640 1 Processor -- start 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc0333f3640 1 -- start start 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc0333f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc0333f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 0x7fc02c110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc0333f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c10d040 con 0x7fc02c1074b0 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc0333f3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c10d1b0 con 0x7fc02c109de0 2024-09-21T04:15:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc03118f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.594+0000 7fc03118f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47972/0 (socket says 172.21.15.110:47972) 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03118f640 1 -- 172.21.15.110:0/4087857242 learned_addr learned my addr 172.21.15.110:0/4087857242 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03098e640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 0x7fc02c110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03118f640 1 -- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 msgr2=0x7fc02c110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03118f640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 0x7fc02c110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03118f640 1 -- 172.21.15.110:0/4087857242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02400a950 con 0x7fc02c1074b0 2024-09-21T04:15:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03098e640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 0x7fc02c110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:15:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.595+0000 7fc03118f640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc02402c0b0 tx=0x7fc02402dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:15.599 INFO:tasks.workunit.client.0.smithi110.stdout:5/477: sync 2024-09-21T04:15:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.596+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02402e7d0 con 0x7fc02c1074b0 2024-09-21T04:15:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.596+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c10d430 con 0x7fc02c1074b0 2024-09-21T04:15:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.596+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c10d920 con 0x7fc02c1074b0 2024-09-21T04:15:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.596+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc024003e80 con 0x7fc02c1074b0 2024-09-21T04:15:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.597+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02403f630 con 0x7fc02c1074b0 2024-09-21T04:15:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.598+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) v1 ==== 51935+0+0 (secure 0 0 0) 0x7fc02403f850 con 0x7fc02c1074b0 2024-09-21T04:15:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.598+0000 7fc0227fc640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 0x7fc018041e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.599+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc024076fe0 con 0x7fc02c1074b0 2024-09-21T04:15:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.599+0000 7fc03098e640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 0x7fc018041e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.600+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010005280 con 0x7fc02c1074b0 2024-09-21T04:15:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.600+0000 7fc03098e640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 0x7fc018041e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc02c10e4d0 tx=0x7fc028009260 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:15.608 INFO:tasks.workunit.client.0.smithi110.stdout:6/377: dwrite d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:15.608 INFO:tasks.workunit.client.0.smithi110.stdout:0/678: creat d4/d47/d2c/fce x:0 0 0 2024-09-21T04:15:15.612 INFO:tasks.workunit.client.0.smithi110.stdout:2/241: stat d2/d1a/d1f/d2a/l11 0 2024-09-21T04:15:15.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:15.610+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fc02403fc40 con 0x7fc02c1074b0 2024-09-21T04:15:15.614 INFO:tasks.workunit.client.1.smithi148.stdout:4/436: dwrite d4/d12/d16/f1f [0,4194304] 0 2024-09-21T04:15:15.614 INFO:tasks.workunit.client.0.smithi110.stdout:6/378: dread d4/fb [4194304,4194304] 0 2024-09-21T04:15:15.619 INFO:tasks.workunit.client.0.smithi110.stdout:0/679: mknod d4/d1c/d62/d69/ccf 0 2024-09-21T04:15:15.619 INFO:tasks.workunit.client.0.smithi110.stdout:0/680: read - d4/d47/fcc zero size 2024-09-21T04:15:15.621 INFO:tasks.workunit.client.0.smithi110.stdout:9/345: sync 2024-09-21T04:15:15.622 INFO:tasks.workunit.client.0.smithi110.stdout:1/667: sync 2024-09-21T04:15:15.622 INFO:tasks.workunit.client.0.smithi110.stdout:4/586: sync 2024-09-21T04:15:15.627 INFO:tasks.workunit.client.0.smithi110.stdout:2/242: write d2/d1a/d1f/d2a/f21 [6278291,100064] 0 2024-09-21T04:15:15.627 INFO:tasks.workunit.client.0.smithi110.stdout:5/478: dwrite d1/d39/f88 [0,4194304] 0 2024-09-21T04:15:15.628 INFO:tasks.workunit.client.0.smithi110.stdout:9/346: chown d7/d26/l30 525 1 2024-09-21T04:15:15.628 INFO:tasks.workunit.client.0.smithi110.stdout:1/668: link d2/d9/d26/d3d/d8f/l9b d2/d9/d23/d4e/d8e/lda 0 2024-09-21T04:15:15.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/469: dwrite d0/d9/dc/d13/d25/f79 [0,4194304] 0 2024-09-21T04:15:15.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/470: fsync d0/d2f/f84 0 2024-09-21T04:15:15.634 INFO:tasks.workunit.client.0.smithi110.stdout:2/243: rename d2/d1a/d1f/d2a/f1b to d2/f44 0 2024-09-21T04:15:15.635 INFO:tasks.workunit.client.0.smithi110.stdout:4/587: truncate d17/d1e/f48 2195895 0 2024-09-21T04:15:15.635 INFO:tasks.workunit.client.0.smithi110.stdout:5/479: creat d1/d49/d4d/d1e/fa6 x:0 0 0 2024-09-21T04:15:15.635 INFO:tasks.workunit.client.0.smithi110.stdout:5/480: chown d1/d39/f79 42965 1 2024-09-21T04:15:15.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/471: truncate d0/d9/d59/f65 97657 0 2024-09-21T04:15:15.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/472: dread - d0/d9/d16/f6a zero size 2024-09-21T04:15:15.637 INFO:tasks.workunit.client.0.smithi110.stdout:9/347: creat d7/d10/d24/d3c/d14/d5c/d6f/f77 x:0 0 0 2024-09-21T04:15:15.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/345: sync 2024-09-21T04:15:15.643 INFO:tasks.workunit.client.0.smithi110.stdout:1/669: truncate d2/d9/d23/f29 548518 0 2024-09-21T04:15:15.646 INFO:tasks.workunit.client.1.smithi148.stdout:2/473: truncate d0/d9/dc/d13/d25/d2d/f2e 910570 0 2024-09-21T04:15:15.646 INFO:tasks.workunit.client.1.smithi148.stdout:2/474: read - d0/d9/d16/f5b zero size 2024-09-21T04:15:15.647 INFO:tasks.workunit.client.1.smithi148.stdout:3/346: mkdir d3/d5/d6/d12/d43/d6e 0 2024-09-21T04:15:15.649 INFO:tasks.workunit.client.0.smithi110.stdout:2/244: truncate d2/f44 460032 0 2024-09-21T04:15:15.649 INFO:tasks.workunit.client.0.smithi110.stdout:1/670: truncate d2/d9/d49/d5c/f84 2829041 0 2024-09-21T04:15:15.649 INFO:tasks.workunit.client.0.smithi110.stdout:1/671: dread - d2/d9/d26/d3d/fd9 zero size 2024-09-21T04:15:15.649 INFO:tasks.workunit.client.0.smithi110.stdout:2/245: truncate d2/f3b 631016 0 2024-09-21T04:15:15.649 INFO:tasks.workunit.client.0.smithi110.stdout:1/672: readlink d2/d9/d49/d4f/d87/la9 0 2024-09-21T04:15:15.650 INFO:tasks.workunit.client.1.smithi148.stdout:2/475: rename d0/d9/dc/d13/d25/f89 to d0/d2b/d48/f94 0 2024-09-21T04:15:15.650 INFO:tasks.workunit.client.1.smithi148.stdout:3/347: symlink d3/d5/d6/d12/da/d23/l6f 0 2024-09-21T04:15:15.650 INFO:tasks.workunit.client.1.smithi148.stdout:3/348: read - d3/f49 zero size 2024-09-21T04:15:15.651 INFO:tasks.workunit.client.0.smithi110.stdout:1/673: symlink d2/d9/d26/d3d/ldb 0 2024-09-21T04:15:15.652 INFO:tasks.workunit.client.0.smithi110.stdout:0/681: dwrite d4/d47/d2c/f88 [0,4194304] 0 2024-09-21T04:15:15.653 INFO:tasks.workunit.client.0.smithi110.stdout:0/682: fsync d4/d47/fc9 0 2024-09-21T04:15:15.653 INFO:tasks.workunit.client.0.smithi110.stdout:4/588: dread d17/d3e/f40 [0,4194304] 0 2024-09-21T04:15:15.653 INFO:tasks.workunit.client.0.smithi110.stdout:4/589: fdatasync d17/d1e/f48 0 2024-09-21T04:15:15.653 INFO:tasks.workunit.client.1.smithi148.stdout:3/349: rename d3/f4c to d3/d1f/d4d/f70 0 2024-09-21T04:15:15.653 INFO:tasks.workunit.client.1.smithi148.stdout:3/350: write d3/d5/d6/d12/da/f3d [7697605,10167] 0 2024-09-21T04:15:15.656 INFO:tasks.workunit.client.0.smithi110.stdout:0/683: mknod d4/d8b/cd0 0 2024-09-21T04:15:15.657 INFO:tasks.workunit.client.1.smithi148.stdout:2/476: dread d0/d2b/d48/f44 [0,4194304] 0 2024-09-21T04:15:15.657 INFO:tasks.workunit.client.1.smithi148.stdout:2/477: getdents d0/d9/dc/d13/d25/d4e/d82 0 2024-09-21T04:15:15.659 INFO:tasks.workunit.client.0.smithi110.stdout:4/590: mkdir d17/d3e/dcd 0 2024-09-21T04:15:15.660 INFO:tasks.workunit.client.0.smithi110.stdout:0/684: rename d4/dd/d53/l76 to d4/d1c/ld1 0 2024-09-21T04:15:15.663 INFO:tasks.workunit.client.0.smithi110.stdout:6/379: dwrite f3 [0,4194304] 0 2024-09-21T04:15:15.664 INFO:tasks.workunit.client.0.smithi110.stdout:5/481: dwrite d1/d22/d51/f98 [0,4194304] 0 2024-09-21T04:15:15.665 INFO:tasks.workunit.client.1.smithi148.stdout:2/478: dread d0/d9/dc/d13/d25/f2a [0,4194304] 0 2024-09-21T04:15:15.665 INFO:tasks.workunit.client.1.smithi148.stdout:2/479: write d0/d9/f29 [976511,31422] 0 2024-09-21T04:15:15.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/437: dwrite d4/d12/f41 [0,4194304] 0 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:15 smithi110 ceph-mon[27723]: [21/Sep/2024:04:15:14] ENGINE Bus STARTING 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:15 smithi110 ceph-mon[27723]: [21/Sep/2024:04:15:14] ENGINE Serving on https://172.21.15.148:7150 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:15 smithi110 ceph-mon[27723]: [21/Sep/2024:04:15:14] ENGINE Error in HTTPServer.serve 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:15:15.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:15 smithi110 ceph-mon[27723]: [21/Sep/2024:04:15:14] ENGINE Serving on http://172.21.15.148:8765 2024-09-21T04:15:15.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:15 smithi110 ceph-mon[27723]: [21/Sep/2024:04:15:14] ENGINE Bus STARTED 2024-09-21T04:15:15.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/480: truncate d0/d9/dc/d13/f80 3471264 0 2024-09-21T04:15:15.675 INFO:tasks.workunit.client.0.smithi110.stdout:4/591: write d17/d1e/f79 [2225430,109273] 0 2024-09-21T04:15:15.678 INFO:tasks.workunit.client.0.smithi110.stdout:5/482: dread d1/d22/f3c [4194304,4194304] 0 2024-09-21T04:15:15.678 INFO:tasks.workunit.client.0.smithi110.stdout:9/348: dwrite d7/d10/d24/d3c/d35/f5e [0,4194304] 0 2024-09-21T04:15:15.678 INFO:tasks.workunit.client.0.smithi110.stdout:5/483: creat d1/fa7 x:0 0 0 2024-09-21T04:15:15.678 INFO:tasks.workunit.client.0.smithi110.stdout:8/459: dwrite d0/d7/f2a [0,4194304] 0 2024-09-21T04:15:15.680 INFO:tasks.workunit.client.0.smithi110.stdout:4/592: mknod d17/d34/d3d/d57/db9/cce 0 2024-09-21T04:15:15.686 INFO:tasks.workunit.client.0.smithi110.stdout:8/460: mknod d0/d7/d13/d4b/d91/c9c 0 2024-09-21T04:15:15.686 INFO:tasks.workunit.client.1.smithi148.stdout:2/481: truncate d0/d2b/f76 1088159 0 2024-09-21T04:15:15.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/593: mkdir d17/d1e/d77/dcf 0 2024-09-21T04:15:15.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/351: dwrite d3/d5/d6/d12/d43/d54/f6a [0,4194304] 0 2024-09-21T04:15:15.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/352: creat d3/d5/d6/d12/d43/d54/f71 x:0 0 0 2024-09-21T04:15:15.698 INFO:tasks.workunit.client.0.smithi110.stdout:9/349: read d7/d10/d24/d3c/d14/f54 [510101,67850] 0 2024-09-21T04:15:15.698 INFO:tasks.workunit.client.0.smithi110.stdout:1/674: getdents d2/d9/d26/d3d 0 2024-09-21T04:15:15.699 INFO:tasks.workunit.client.0.smithi110.stdout:6/380: dwrite d4/f6c [0,4194304] 0 2024-09-21T04:15:15.700 INFO:tasks.workunit.client.0.smithi110.stdout:0/685: rmdir d4/d1c 39 2024-09-21T04:15:15.701 INFO:tasks.workunit.client.0.smithi110.stdout:8/461: mkdir d0/d30/d9d 0 2024-09-21T04:15:15.704 INFO:tasks.workunit.client.0.smithi110.stdout:4/594: mknod d17/d3e/d6b/cd0 0 2024-09-21T04:15:15.704 INFO:tasks.workunit.client.0.smithi110.stdout:4/595: stat l2 0 2024-09-21T04:15:15.710 INFO:tasks.workunit.client.0.smithi110.stdout:9/350: mkdir d7/d10/d24/d3c/d14/d5c/d6f/d78 0 2024-09-21T04:15:15.712 INFO:tasks.workunit.client.1.smithi148.stdout:3/353: getdents d3/d1f 0 2024-09-21T04:15:15.712 INFO:tasks.workunit.client.1.smithi148.stdout:4/438: dwrite d4/d12/d16/d29/f3d [0,4194304] 0 2024-09-21T04:15:15.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/381: dread d4/d12/f1d [0,4194304] 0 2024-09-21T04:15:15.715 INFO:tasks.workunit.client.1.smithi148.stdout:3/354: dread d3/d1f/f3e [0,4194304] 0 2024-09-21T04:15:15.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/395: sync 2024-09-21T04:15:15.716 INFO:tasks.workunit.client.1.smithi148.stdout:5/378: sync 2024-09-21T04:15:15.719 INFO:tasks.workunit.client.0.smithi110.stdout:4/596: dread d17/d1e/f4c [0,4194304] 0 2024-09-21T04:15:15.719 INFO:tasks.workunit.client.0.smithi110.stdout:4/597: chown d17/d1e/d7b/cbc 43560 1 2024-09-21T04:15:15.719 INFO:tasks.workunit.client.1.smithi148.stdout:2/482: rmdir d0/d2f 39 2024-09-21T04:15:15.722 INFO:tasks.workunit.client.0.smithi110.stdout:0/686: fdatasync d4/d1c/d62/f39 0 2024-09-21T04:15:15.729 INFO:tasks.workunit.client.0.smithi110.stdout:8/462: readlink d0/d7/d13/d4b/l25 0 2024-09-21T04:15:15.729 INFO:tasks.workunit.client.0.smithi110.stdout:9/351: stat d7/d10/d24/d3c/d1f/d72/l44 0 2024-09-21T04:15:15.731 INFO:tasks.workunit.client.0.smithi110.stdout:1/675: mknod d2/d9/d23/cdc 0 2024-09-21T04:15:15.733 INFO:tasks.workunit.client.0.smithi110.stdout:4/598: dread d17/f2a [0,4194304] 0 2024-09-21T04:15:15.733 INFO:tasks.workunit.client.0.smithi110.stdout:4/599: getdents d17/d34/d3d 0 2024-09-21T04:15:15.733 INFO:tasks.workunit.client.0.smithi110.stdout:0/687: unlink d4/d8b/lc4 0 2024-09-21T04:15:15.736 INFO:tasks.workunit.client.0.smithi110.stdout:8/463: symlink d0/d7/d89/l9e 0 2024-09-21T04:15:15.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/439: dwrite d4/d12/f65 [0,4194304] 0 2024-09-21T04:15:15.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/440: write d4/fb [3821080,43464] 0 2024-09-21T04:15:15.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/441: write d4/d12/d16/d26/f70 [60824,30846] 0 2024-09-21T04:15:15.739 INFO:tasks.workunit.client.0.smithi110.stdout:4/600: write d17/d3e/f40 [1714539,26906] 0 2024-09-21T04:15:15.741 INFO:tasks.workunit.client.0.smithi110.stdout:9/352: mknod d7/d10/d24/d3c/d14/d5c/d6f/d78/c79 0 2024-09-21T04:15:15.742 INFO:tasks.workunit.client.0.smithi110.stdout:0/688: unlink d4/dd/d10/d29/l4c 0 2024-09-21T04:15:15.742 INFO:tasks.workunit.client.0.smithi110.stdout:0/689: chown d4/f56 5467688 1 2024-09-21T04:15:15.745 INFO:tasks.workunit.client.1.smithi148.stdout:3/355: creat d3/d5/d6/d12/f72 x:0 0 0 2024-09-21T04:15:15.745 INFO:tasks.workunit.client.1.smithi148.stdout:3/356: write d3/f55 [545007,61423] 0 2024-09-21T04:15:15.747 INFO:tasks.workunit.client.1.smithi148.stdout:8/396: symlink dc/d13/d76/d42/l7e 0 2024-09-21T04:15:15.748 INFO:tasks.workunit.client.0.smithi110.stdout:1/676: dread d2/d9/d23/d4e/f7f [4194304,4194304] 0 2024-09-21T04:15:15.748 INFO:tasks.workunit.client.1.smithi148.stdout:0/382: sync 2024-09-21T04:15:15.748 INFO:tasks.workunit.client.1.smithi148.stdout:7/410: sync 2024-09-21T04:15:15.748 INFO:tasks.workunit.client.1.smithi148.stdout:7/411: readlink d5/d8/l11 0 2024-09-21T04:15:15.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/352: sync 2024-09-21T04:15:15.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/355: sync 2024-09-21T04:15:15.749 INFO:tasks.workunit.client.1.smithi148.stdout:1/353: truncate db/d46/d4c/f5b 442829 0 2024-09-21T04:15:15.749 INFO:tasks.workunit.client.0.smithi110.stdout:0/690: mkdir d4/dd/d10/d29/d75/dd2 0 2024-09-21T04:15:15.750 INFO:tasks.workunit.client.1.smithi148.stdout:2/483: dwrite d0/d9/dc/d13/d25/d4e/f6b [0,4194304] 0 2024-09-21T04:15:15.752 INFO:tasks.workunit.client.0.smithi110.stdout:6/382: dwrite d4/d12/d16/d66/d7a/d54/f72 [0,4194304] 0 2024-09-21T04:15:15.752 INFO:tasks.workunit.client.0.smithi110.stdout:6/383: chown d4/d12/d16/c20 1 1 2024-09-21T04:15:15.754 INFO:tasks.workunit.client.0.smithi110.stdout:0/691: symlink d4/dd/d10/d8a/db5/ld3 0 2024-09-21T04:15:15.755 INFO:tasks.workunit.client.0.smithi110.stdout:8/464: dread d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:15.757 INFO:tasks.workunit.client.1.smithi148.stdout:8/397: read dc/d13/d28/f48 [2777912,22706] 0 2024-09-21T04:15:15.757 INFO:tasks.workunit.client.0.smithi110.stdout:1/677: getdents d2/d9/d49/d7a/da8 0 2024-09-21T04:15:15.760 INFO:tasks.workunit.client.1.smithi148.stdout:4/442: mkdir d4/d12/d78 0 2024-09-21T04:15:15.762 INFO:tasks.workunit.client.1.smithi148.stdout:0/383: mknod d5/d6/d7/d11/c78 0 2024-09-21T04:15:15.763 INFO:tasks.workunit.client.1.smithi148.stdout:0/384: readlink d5/d6/d2a/l3b 0 2024-09-21T04:15:15.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/354: mkdir db/d24/d2e/d71 0 2024-09-21T04:15:15.774 INFO:tasks.workunit.client.0.smithi110.stdout:4/601: dwrite d17/f2a [0,4194304] 0 2024-09-21T04:15:15.776 INFO:tasks.workunit.client.1.smithi148.stdout:9/371: sync 2024-09-21T04:15:15.776 INFO:tasks.workunit.client.1.smithi148.stdout:9/372: write d5/d60/f3f [4233240,120927] 0 2024-09-21T04:15:15.777 INFO:tasks.workunit.client.1.smithi148.stdout:2/484: dwrite d0/d9/dc/d13/d25/f79 [0,4194304] 0 2024-09-21T04:15:15.781 INFO:tasks.workunit.client.0.smithi110.stdout:0/692: dwrite d4/d1c/d62/d69/f77 [0,4194304] 0 2024-09-21T04:15:15.784 INFO:tasks.workunit.client.0.smithi110.stdout:8/465: creat d0/d30/d34/d3e/d5d/f9f x:0 0 0 2024-09-21T04:15:15.788 INFO:tasks.workunit.client.0.smithi110.stdout:1/678: mknod d2/d9/d49/d7a/cdd 0 2024-09-21T04:15:15.789 INFO:tasks.workunit.client.0.smithi110.stdout:4/602: creat d17/d34/d3d/d20/dca/fd1 x:0 0 0 2024-09-21T04:15:15.790 INFO:tasks.workunit.client.0.smithi110.stdout:4/603: chown d17/d34/d3d/d20/d43/d51/d7a/fa7 748740 1 2024-09-21T04:15:15.791 INFO:tasks.workunit.client.0.smithi110.stdout:0/693: rmdir d4/d47/d2c/d3a 39 2024-09-21T04:15:15.793 INFO:tasks.workunit.client.1.smithi148.stdout:4/443: mknod d4/d12/d16/d29/d61/d6d/c79 0 2024-09-21T04:15:15.793 INFO:tasks.workunit.client.0.smithi110.stdout:6/384: dwrite d4/f79 [0,4194304] 0 2024-09-21T04:15:15.795 INFO:tasks.workunit.client.1.smithi148.stdout:0/385: dwrite d5/d6/d2a/d2b/f35 [0,4194304] 0 2024-09-21T04:15:15.795 INFO:tasks.workunit.client.0.smithi110.stdout:8/466: rename d0/d7/d13/f94 to d0/d7/d13/d1c/fa0 0 2024-09-21T04:15:15.796 INFO:tasks.workunit.client.0.smithi110.stdout:4/604: dread d17/d34/d3d/d20/d43/f8c [0,4194304] 0 2024-09-21T04:15:15.797 INFO:tasks.workunit.client.1.smithi148.stdout:7/412: dwrite d5/d14/f4b [0,4194304] 0 2024-09-21T04:15:15.798 INFO:tasks.workunit.client.1.smithi148.stdout:1/355: dwrite db/dd/f23 [0,4194304] 0 2024-09-21T04:15:15.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/356: stat db/dd/c38 0 2024-09-21T04:15:15.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/357: write db/d24/f2b [417106,38203] 0 2024-09-21T04:15:15.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/358: creat db/d24/d4f/f72 x:0 0 0 2024-09-21T04:15:15.799 INFO:tasks.workunit.client.0.smithi110.stdout:1/679: creat d2/d9/d49/d5c/d91/fde x:0 0 0 2024-09-21T04:15:15.801 INFO:tasks.workunit.client.0.smithi110.stdout:0/694: unlink d4/dd/d10/d29/c46 0 2024-09-21T04:15:15.805 INFO:tasks.workunit.client.1.smithi148.stdout:2/485: mkdir d0/d9/dc/d13/d25/d4e/d95 0 2024-09-21T04:15:15.806 INFO:tasks.workunit.client.0.smithi110.stdout:6/385: symlink d4/d12/d17/d2e/d40/l7b 0 2024-09-21T04:15:15.807 INFO:tasks.workunit.client.1.smithi148.stdout:7/413: dread d5/f3d [4194304,4194304] 0 2024-09-21T04:15:15.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/373: read d5/d2b/d32/f5a [513912,102797] 0 2024-09-21T04:15:15.807 INFO:tasks.workunit.client.1.smithi148.stdout:6/356: rename d0/d1a/d23/d16/d4f/d55 to d0/d1a/d23/dc/d68 0 2024-09-21T04:15:15.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/444: dwrite d4/d12/d16/d26/f70 [0,4194304] 0 2024-09-21T04:15:15.808 INFO:tasks.workunit.client.0.smithi110.stdout:8/467: symlink d0/d30/d54/la1 0 2024-09-21T04:15:15.808 INFO:tasks.workunit.client.0.smithi110.stdout:8/468: chown d0/d7/d13/d1c/d41 151 1 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/386: dread d4/d12/d16/f5b [0,4194304] 0 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/387: chown d4/d12/f2b 5356 1 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/388: chown d4/d12/d16/d66/d7a/d4b/f5e 6615939 1 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/389: truncate d4/d12/d16/f50 941941 0 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/390: write d4/d12/d16/f56 [219235,90398] 0 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:4/605: getdents d17/d34/d3d/d20/d43/d51 0 2024-09-21T04:15:15.816 INFO:tasks.workunit.client.0.smithi110.stdout:6/391: chown d4/d12/d16/d66/d7a/d4b/c68 7 1 2024-09-21T04:15:15.822 INFO:tasks.workunit.client.0.smithi110.stdout:1/680: symlink d2/db7/d86/ldf 0 2024-09-21T04:15:15.822 INFO:tasks.workunit.client.0.smithi110.stdout:1/681: dread - d2/d9/d26/d3d/fc5 zero size 2024-09-21T04:15:15.822 INFO:tasks.workunit.client.0.smithi110.stdout:1/682: chown d2/d9/d23/d3b/d9c/c9f 804 1 2024-09-21T04:15:15.825 INFO:tasks.workunit.client.0.smithi110.stdout:0/695: rmdir d4/d1c/d62/d69/d9b 39 2024-09-21T04:15:15.825 INFO:tasks.workunit.client.0.smithi110.stdout:0/696: fsync d4/dd/d10/f1a 0 2024-09-21T04:15:15.829 INFO:tasks.workunit.client.0.smithi110.stdout:6/392: mknod d4/d12/d16/c7c 0 2024-09-21T04:15:15.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/406: sync 2024-09-21T04:15:15.833 INFO:tasks.workunit.client.1.smithi148.stdout:6/357: dwrite d0/d1a/d23/dc/d68/f66 [0,4194304] 0 2024-09-21T04:15:15.833 INFO:tasks.workunit.client.1.smithi148.stdout:6/358: read d0/d1a/d23/d16/d4f/f54 [1616547,83188] 0 2024-09-21T04:15:15.835 INFO:tasks.workunit.client.0.smithi110.stdout:4/606: link d17/d34/d3d/d20/d43/d51/d7a/d96/fcc d17/d34/d3d/d57/fd2 0 2024-09-21T04:15:15.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/359: dread d0/d1a/d23/f3d [0,4194304] 0 2024-09-21T04:15:15.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/360: creat d0/d34/d38/d57/d58/f69 x:0 0 0 2024-09-21T04:15:15.835 INFO:tasks.workunit.client.1.smithi148.stdout:6/361: fsync d0/d1a/d23/f12 0 2024-09-21T04:15:15.836 INFO:tasks.workunit.client.1.smithi148.stdout:6/362: write d0/d1a/d23/d16/d4f/f54 [886358,112899] 0 2024-09-21T04:15:15.836 INFO:tasks.workunit.client.1.smithi148.stdout:6/363: chown d0/d1a/d23/dc/d68/f51 0 1 2024-09-21T04:15:15.836 INFO:tasks.workunit.client.0.smithi110.stdout:6/393: dread d4/de/f1f [0,4194304] 0 2024-09-21T04:15:15.836 INFO:tasks.workunit.client.0.smithi110.stdout:8/469: dwrite d0/d7/d13/d1c/d41/f43 [0,4194304] 0 2024-09-21T04:15:15.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/359: rmdir db/dd/d21/d6d 0 2024-09-21T04:15:15.837 INFO:tasks.workunit.client.1.smithi148.stdout:1/360: creat db/d46/d4c/f73 x:0 0 0 2024-09-21T04:15:15.839 INFO:tasks.workunit.client.1.smithi148.stdout:7/414: creat d5/d8/f89 x:0 0 0 2024-09-21T04:15:15.839 INFO:tasks.workunit.client.0.smithi110.stdout:7/407: mkdir dd/d1c/d7d 0 2024-09-21T04:15:15.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/374: creat d5/d2b/f91 x:0 0 0 2024-09-21T04:15:15.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/375: stat d5/d60/d28/d2f/l5f 0 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:6/394: dread d4/d12/d17/d33/d3e/f53 [0,4194304] 0 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:6/395: dread - d4/d12/d17/d2e/f47 zero size 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:0/697: dwrite d4/dd/d60/fa4 [0,4194304] 0 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/607: mkdir d17/d3e/dd3 0 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/608: creat d17/d1e/d21/fd4 x:0 0 0 2024-09-21T04:15:15.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/609: dread - d17/d34/d3d/d57/db9/fc8 zero size 2024-09-21T04:15:15.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/386: rename d5/d6/d7/d11/f34 to d5/d6/d2a/f79 0 2024-09-21T04:15:15.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/387: stat d5 0 2024-09-21T04:15:15.853 INFO:tasks.workunit.client.0.smithi110.stdout:6/396: dread d4/d12/f2b [0,4194304] 0 2024-09-21T04:15:15.865 INFO:tasks.workunit.client.0.smithi110.stdout:6/397: write d4/d12/d17/f2a [3251626,7787] 0 2024-09-21T04:15:15.868 INFO:tasks.workunit.client.0.smithi110.stdout:0/698: mknod d4/dd/d10/d8a/db5/cd4 0 2024-09-21T04:15:15.868 INFO:tasks.workunit.client.0.smithi110.stdout:4/610: dwrite d17/d3e/f40 [4194304,4194304] 0 2024-09-21T04:15:15.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/364: link d0/c4 d0/d34/d38/c6a 0 2024-09-21T04:15:15.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/365: truncate d0/d34/d38/f3a 336930 0 2024-09-21T04:15:15.871 INFO:tasks.workunit.client.0.smithi110.stdout:8/470: symlink d0/la2 0 2024-09-21T04:15:15.877 INFO:tasks.workunit.client.1.smithi148.stdout:7/415: mkdir d5/d8/d6f/d7d/d8a 0 2024-09-21T04:15:15.878 INFO:tasks.workunit.client.1.smithi148.stdout:4/445: dwrite d4/d23/d2a/f2f [0,4194304] 0 2024-09-21T04:15:15.878 INFO:tasks.workunit.client.1.smithi148.stdout:4/446: read - d4/d12/d2c/d2d/f6b zero size 2024-09-21T04:15:15.878 INFO:tasks.workunit.client.1.smithi148.stdout:4/447: fsync d4/d12/f5b 0 2024-09-21T04:15:15.880 INFO:tasks.workunit.client.0.smithi110.stdout:6/398: unlink d4/d12/d17/d2e/d40/f5d 0 2024-09-21T04:15:15.883 INFO:tasks.workunit.client.1.smithi148.stdout:2/486: rename d0/d9/dc/d13/d25/c4d to d0/d9/dc/d13/d25/d4e/d95/c96 0 2024-09-21T04:15:15.883 INFO:tasks.workunit.client.1.smithi148.stdout:2/487: fdatasync d0/d9/d16/f37 0 2024-09-21T04:15:15.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/683: rename d2/d9/d23/d3b/d93 to d2/d9/d23/de0 0 2024-09-21T04:15:15.885 INFO:tasks.workunit.client.0.smithi110.stdout:8/471: write d0/d7/d13/d4b/f17 [7698868,1926] 0 2024-09-21T04:15:15.885 INFO:tasks.workunit.client.0.smithi110.stdout:4/611: dread d17/f58 [0,4194304] 0 2024-09-21T04:15:15.885 INFO:tasks.workunit.client.0.smithi110.stdout:4/612: creat d17/d34/d3d/d57/db9/fd5 x:0 0 0 2024-09-21T04:15:15.887 INFO:tasks.workunit.client.1.smithi148.stdout:6/366: unlink d0/d34/d38/d57/d58/f5c 0 2024-09-21T04:15:15.890 INFO:tasks.workunit.client.0.smithi110.stdout:6/399: symlink d4/d12/d16/l7d 0 2024-09-21T04:15:15.890 INFO:tasks.workunit.client.0.smithi110.stdout:6/400: write d4/d12/d16/f56 [460892,69647] 0 2024-09-21T04:15:15.892 INFO:tasks.workunit.client.1.smithi148.stdout:1/361: mknod db/c74 0 2024-09-21T04:15:15.892 INFO:tasks.workunit.client.1.smithi148.stdout:1/362: write db/d24/d2a/f45 [272442,74732] 0 2024-09-21T04:15:15.892 INFO:tasks.workunit.client.1.smithi148.stdout:1/363: truncate db/dd/f12 1582404 0 2024-09-21T04:15:15.901 INFO:tasks.workunit.client.0.smithi110.stdout:8/472: mkdir d0/d30/d34/d3e/d32/da3 0 2024-09-21T04:15:15.901 INFO:tasks.workunit.client.0.smithi110.stdout:7/408: dwrite dd/d1c/d39/f5a [0,4194304] 0 2024-09-21T04:15:15.901 INFO:tasks.workunit.client.0.smithi110.stdout:7/409: write dd/d4d/f59 [1106627,118924] 0 2024-09-21T04:15:15.902 INFO:tasks.workunit.client.0.smithi110.stdout:4/613: truncate d17/d1e/d7b/fb8 1961431 0 2024-09-21T04:15:15.902 INFO:tasks.workunit.client.0.smithi110.stdout:4/614: write d17/d34/d3d/d20/d43/d51/d52/f59 [2110885,4658] 0 2024-09-21T04:15:15.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/615: creat d17/d34/d3d/d20/dca/fd6 x:0 0 0 2024-09-21T04:15:15.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/616: write d17/d34/d3d/d20/d43/d51/d7a/fc3 [373292,81206] 0 2024-09-21T04:15:15.906 INFO:tasks.workunit.client.0.smithi110.stdout:0/699: dwrite d4/fb [4194304,4194304] 0 2024-09-21T04:15:15.906 INFO:tasks.workunit.client.0.smithi110.stdout:0/700: readlink d4/dd/d53/d66/lae 0 2024-09-21T04:15:15.907 INFO:tasks.workunit.client.1.smithi148.stdout:4/448: creat d4/d12/d75/f7a x:0 0 0 2024-09-21T04:15:15.909 INFO:tasks.workunit.client.1.smithi148.stdout:0/388: rename d5 to d5/d6/d7/d22/d7a 22 2024-09-21T04:15:15.909 INFO:tasks.workunit.client.1.smithi148.stdout:9/376: rename d5/d60/d4c/l7a to d5/d2b/d4e/l92 0 2024-09-21T04:15:15.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/488: symlink d0/l97 0 2024-09-21T04:15:15.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/489: stat d0/d9/dc/d13/d50 0 2024-09-21T04:15:15.911 INFO:tasks.workunit.client.1.smithi148.stdout:6/367: getdents d0/d1a/d23/d53 0 2024-09-21T04:15:15.914 INFO:tasks.workunit.client.1.smithi148.stdout:1/364: creat db/f75 x:0 0 0 2024-09-21T04:15:15.915 INFO:tasks.workunit.client.1.smithi148.stdout:1/365: chown db/dd/f12 181777228 1 2024-09-21T04:15:15.917 INFO:tasks.workunit.client.1.smithi148.stdout:4/449: symlink d4/d23/l7b 0 2024-09-21T04:15:15.917 INFO:tasks.workunit.client.1.smithi148.stdout:2/490: dread d0/d9/d38/f41 [0,4194304] 0 2024-09-21T04:15:15.917 INFO:tasks.workunit.client.1.smithi148.stdout:2/491: creat d0/d2b/f98 x:0 0 0 2024-09-21T04:15:15.918 INFO:tasks.workunit.client.1.smithi148.stdout:4/450: read d4/d3a/f53 [339228,121020] 0 2024-09-21T04:15:15.919 INFO:tasks.workunit.client.0.smithi110.stdout:1/684: dwrite d2/d9/d23/de0/f9e [0,4194304] 0 2024-09-21T04:15:15.920 INFO:tasks.workunit.client.0.smithi110.stdout:7/410: unlink dd/d1c/d39/d6d/c72 0 2024-09-21T04:15:15.920 INFO:tasks.workunit.client.0.smithi110.stdout:7/411: write dd/d1c/d39/f76 [435772,87842] 0 2024-09-21T04:15:15.920 INFO:tasks.workunit.client.0.smithi110.stdout:7/412: stat dd/d48/d55/d7a/l67 0 2024-09-21T04:15:15.922 INFO:tasks.workunit.client.1.smithi148.stdout:0/389: dread d5/d6/d7/d22/f73 [0,4194304] 0 2024-09-21T04:15:15.922 INFO:tasks.workunit.client.1.smithi148.stdout:0/390: chown d5/d6 64284874 1 2024-09-21T04:15:15.922 INFO:tasks.workunit.client.1.smithi148.stdout:0/391: dread - d5/d6/d2a/d45/f6e zero size 2024-09-21T04:15:15.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/377: mknod d5/d60/d4c/c93 0 2024-09-21T04:15:15.927 INFO:tasks.workunit.client.1.smithi148.stdout:9/378: fsync d5/ff 0 2024-09-21T04:15:15.927 INFO:tasks.workunit.client.1.smithi148.stdout:9/379: chown f0 104 1 2024-09-21T04:15:15.929 INFO:tasks.workunit.client.1.smithi148.stdout:8/398: sync 2024-09-21T04:15:15.929 INFO:tasks.workunit.client.1.smithi148.stdout:5/379: sync 2024-09-21T04:15:15.929 INFO:tasks.workunit.client.1.smithi148.stdout:8/399: write dc/d13/f1c [570434,25919] 0 2024-09-21T04:15:15.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/368: creat d0/f6b x:0 0 0 2024-09-21T04:15:15.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/369: fsync d0/f21 0 2024-09-21T04:15:15.932 INFO:tasks.workunit.client.1.smithi148.stdout:5/380: dread d3/d24/f43 [0,4194304] 0 2024-09-21T04:15:15.933 INFO:tasks.workunit.client.1.smithi148.stdout:1/366: mkdir db/d46/d76 0 2024-09-21T04:15:15.939 INFO:tasks.workunit.client.1.smithi148.stdout:0/392: dread d5/d6/d7/f4b [0,4194304] 0 2024-09-21T04:15:15.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/393: chown d5/d6/d7/d18/f32 1 1 2024-09-21T04:15:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:15 smithi148 ceph-mon[34957]: [21/Sep/2024:04:15:14] ENGINE Bus STARTING 2024-09-21T04:15:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:15 smithi148 ceph-mon[34957]: [21/Sep/2024:04:15:14] ENGINE Serving on https://172.21.15.148:7150 2024-09-21T04:15:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:15 smithi148 ceph-mon[34957]: [21/Sep/2024:04:15:14] ENGINE Error in HTTPServer.serve 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:15:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-09-21T04:15:15.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:15:15.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-09-21T04:15:15.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:15:15.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:15 smithi148 ceph-mon[34957]: [21/Sep/2024:04:15:14] ENGINE Serving on http://172.21.15.148:8765 2024-09-21T04:15:15.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:15 smithi148 ceph-mon[34957]: [21/Sep/2024:04:15:14] ENGINE Bus STARTED 2024-09-21T04:15:15.942 INFO:tasks.workunit.client.1.smithi148.stdout:3/357: sync 2024-09-21T04:15:15.942 INFO:tasks.workunit.client.1.smithi148.stdout:3/358: write d3/d5/d6/d12/f72 [1030100,4317] 0 2024-09-21T04:15:15.943 INFO:tasks.workunit.client.1.smithi148.stdout:3/359: fsync d3/d1f/f3e 0 2024-09-21T04:15:15.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/492: unlink d0/d9/d16/d55/d6f/l49 0 2024-09-21T04:15:15.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/493: chown d0/d2b/d48/c78 3718272 1 2024-09-21T04:15:15.947 INFO:tasks.workunit.client.1.smithi148.stdout:2/494: read - d0/f68 zero size 2024-09-21T04:15:15.947 INFO:tasks.workunit.client.1.smithi148.stdout:4/451: symlink d4/d23/d2a/l7c 0 2024-09-21T04:15:15.947 INFO:tasks.workunit.client.1.smithi148.stdout:4/452: dread - d4/d12/d2c/d2d/f76 zero size 2024-09-21T04:15:15.951 INFO:tasks.workunit.client.1.smithi148.stdout:6/370: symlink d0/d1a/d23/d32/l6c 0 2024-09-21T04:15:15.951 INFO:tasks.workunit.client.1.smithi148.stdout:5/381: symlink d3/d4a/l8b 0 2024-09-21T04:15:15.951 INFO:tasks.workunit.client.1.smithi148.stdout:5/382: write d3/d14/f84 [289616,65399] 0 2024-09-21T04:15:15.952 INFO:tasks.workunit.client.1.smithi148.stdout:5/383: fsync d3/d34/f47 0 2024-09-21T04:15:15.952 INFO:tasks.workunit.client.1.smithi148.stdout:5/384: stat d3/d4a/l53 0 2024-09-21T04:15:15.952 INFO:tasks.workunit.client.1.smithi148.stdout:5/385: fsync d3/d14/d1a/f33 0 2024-09-21T04:15:15.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/394: mkdir d5/d6/d7/d11/d7b 0 2024-09-21T04:15:15.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/395: readlink d5/d6/d2a/l3b 0 2024-09-21T04:15:15.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/453: symlink d4/d12/d16/l7d 0 2024-09-21T04:15:15.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/380: rmdir d5/d39 39 2024-09-21T04:15:15.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/381: chown d5/d1f/d81 1722513 1 2024-09-21T04:15:15.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/416: dwrite d5/d8/f22 [0,4194304] 0 2024-09-21T04:15:15.963 INFO:tasks.workunit.client.0.smithi110.stdout:7/413: rename dd/d14/d17/f5b to dd/f7e 0 2024-09-21T04:15:15.963 INFO:tasks.workunit.client.1.smithi148.stdout:6/371: rename d0/d1a/d23/dc/c15 to d0/d8/d18/c6d 0 2024-09-21T04:15:15.965 INFO:tasks.workunit.client.1.smithi148.stdout:5/386: symlink d3/d14/d1f/d26/d2e/d56/l8c 0 2024-09-21T04:15:15.965 INFO:tasks.workunit.client.1.smithi148.stdout:5/387: fdatasync d3/d14/d1a/f33 0 2024-09-21T04:15:15.965 INFO:tasks.workunit.client.1.smithi148.stdout:5/388: chown d3/d37/l21 421 1 2024-09-21T04:15:15.967 INFO:tasks.workunit.client.1.smithi148.stdout:4/454: mkdir d4/d12/d16/d29/d61/d7e 0 2024-09-21T04:15:15.969 INFO:tasks.workunit.client.1.smithi148.stdout:2/495: dread d0/d9/dc/f46 [0,4194304] 0 2024-09-21T04:15:15.970 INFO:tasks.workunit.client.1.smithi148.stdout:0/396: rename d5/d6/d7/d22/l6f to d5/d6/d7/l7c 0 2024-09-21T04:15:15.970 INFO:tasks.workunit.client.1.smithi148.stdout:0/397: creat d5/d6/d7/d11/d48/f7d x:0 0 0 2024-09-21T04:15:15.971 INFO:tasks.workunit.client.1.smithi148.stdout:9/382: getdents d5/d1e 0 2024-09-21T04:15:15.971 INFO:tasks.workunit.client.1.smithi148.stdout:9/383: fdatasync d5/d2b/d32/f82 0 2024-09-21T04:15:15.971 INFO:tasks.workunit.client.1.smithi148.stdout:9/384: chown d5/ff 893 1 2024-09-21T04:15:15.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/496: rename d0/d9/f39 to d0/d9/dc/f99 0 2024-09-21T04:15:15.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/497: chown d0/d9/d16/l34 6087 1 2024-09-21T04:15:15.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/498: chown d0/d9/dc/d13/d25 58793 1 2024-09-21T04:15:15.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/499: fdatasync d0/d9/dc/d13/f1a 0 2024-09-21T04:15:15.978 INFO:tasks.workunit.client.1.smithi148.stdout:2/500: symlink d0/d9/d16/d55/d71/l9a 0 2024-09-21T04:15:15.978 INFO:tasks.workunit.client.1.smithi148.stdout:2/501: truncate d0/d9/d16/f60 894769 0 2024-09-21T04:15:15.980 INFO:tasks.workunit.client.1.smithi148.stdout:5/389: read d3/d14/d1f/f50 [463302,89914] 0 2024-09-21T04:15:15.983 INFO:tasks.workunit.client.1.smithi148.stdout:9/385: write d5/d1e/f2d [2380969,37556] 0 2024-09-21T04:15:15.983 INFO:tasks.workunit.client.1.smithi148.stdout:5/390: getdents d3/d34/d3c 0 2024-09-21T04:15:15.985 INFO:tasks.workunit.client.1.smithi148.stdout:9/386: mknod d5/d1e/c94 0 2024-09-21T04:15:15.986 INFO:tasks.workunit.client.1.smithi148.stdout:9/387: rmdir d5/d1e 39 2024-09-21T04:15:15.986 INFO:tasks.workunit.client.1.smithi148.stdout:9/388: getdents d5/d1e/d37/d7b 0 2024-09-21T04:15:15.987 INFO:tasks.workunit.client.1.smithi148.stdout:0/398: write d5/d6/d7/d22/f24 [1071535,37931] 0 2024-09-21T04:15:15.988 INFO:tasks.workunit.client.0.smithi110.stdout:4/617: dwrite d17/d3e/f9c [0,4194304] 0 2024-09-21T04:15:15.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/389: mkdir d5/d2b/d4e/d95 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:6/401: dwrite d4/f3d [4194304,4194304] 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:8/473: dwrite d0/d7/d13/d1c/d41/f43 [0,4194304] 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:8/474: stat d0/d7/d13/l2c 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:8/475: chown d0/d7/d89/l9e 1910 1 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/701: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/702: write d4/dd/d10/d8a/fcd [763499,99876] 0 2024-09-21T04:15:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/703: chown d4/fbc 1 1 2024-09-21T04:15:15.995 INFO:tasks.workunit.client.0.smithi110.stdout:6/402: rename d4/d12/d16/d66/d7a/d54/f72 to d4/d12/d16/f7e 0 2024-09-21T04:15:15.995 INFO:tasks.workunit.client.0.smithi110.stdout:6/403: truncate d4/f7 4964381 0 2024-09-21T04:15:15.998 INFO:tasks.workunit.client.0.smithi110.stdout:8/476: mknod d0/d7/d23/d82/ca4 0 2024-09-21T04:15:16.000 INFO:tasks.workunit.client.1.smithi148.stdout:0/399: write d5/d6/f1f [594084,84768] 0 2024-09-21T04:15:16.000 INFO:tasks.workunit.client.0.smithi110.stdout:0/704: truncate d4/dd/f8e 1093521 0 2024-09-21T04:15:16.000 INFO:tasks.workunit.client.0.smithi110.stdout:0/705: chown d4/dd/d10/f13 1 1 2024-09-21T04:15:16.001 INFO:tasks.workunit.client.0.smithi110.stdout:6/404: truncate d4/d12/f1d 4283647 0 2024-09-21T04:15:16.002 INFO:tasks.workunit.client.1.smithi148.stdout:0/400: mknod d5/d6/d2a/c7e 0 2024-09-21T04:15:16.002 INFO:tasks.workunit.client.0.smithi110.stdout:1/685: dwrite d2/d9/d49/d4f/d53/f8c [4194304,4194304] 0 2024-09-21T04:15:16.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/400: dwrite dc/d13/d39/f51 [0,4194304] 0 2024-09-21T04:15:16.003 INFO:tasks.workunit.client.1.smithi148.stdout:1/367: dwrite db/d46/f48 [0,4194304] 0 2024-09-21T04:15:16.005 INFO:tasks.workunit.client.1.smithi148.stdout:3/360: dwrite d3/d5/d5c/f58 [0,4194304] 0 2024-09-21T04:15:16.006 INFO:tasks.workunit.client.0.smithi110.stdout:8/477: creat d0/d30/d34/fa5 x:0 0 0 2024-09-21T04:15:16.006 INFO:tasks.workunit.client.0.smithi110.stdout:0/706: rename d4/dd/d53/d66/lae to d4/d1c/d62/d69/d9b/ld5 0 2024-09-21T04:15:16.006 INFO:tasks.workunit.client.0.smithi110.stdout:6/405: write d4/de/f1f [1357109,48381] 0 2024-09-21T04:15:16.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/361: dread d3/d1f/f3e [0,4194304] 0 2024-09-21T04:15:16.008 INFO:tasks.workunit.client.1.smithi148.stdout:7/417: dwrite d5/dc/f5d [4194304,4194304] 0 2024-09-21T04:15:16.008 INFO:tasks.workunit.client.1.smithi148.stdout:6/372: dwrite d0/d8/d18/f4d [0,4194304] 0 2024-09-21T04:15:16.009 INFO:tasks.workunit.client.1.smithi148.stdout:8/401: truncate dc/d13/d28/f48 1606034 0 2024-09-21T04:15:16.010 INFO:tasks.workunit.client.1.smithi148.stdout:4/455: getdents d4/d23 0 2024-09-21T04:15:16.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/368: stat db/d24/d2e/l44 0 2024-09-21T04:15:16.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/369: readlink db/d24/l3a 0 2024-09-21T04:15:16.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/370: chown db/d24/d2a/f45 2514800 1 2024-09-21T04:15:16.018 INFO:tasks.workunit.client.1.smithi148.stdout:2/502: dwrite d0/d9/f29 [0,4194304] 0 2024-09-21T04:15:16.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/478: rename d0/d30/d54/f66 to d0/d7/d6f/d86/fa6 0 2024-09-21T04:15:16.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/479: read d0/d30/d54/f59 [847584,82908] 0 2024-09-21T04:15:16.023 INFO:tasks.workunit.client.0.smithi110.stdout:3/381: sync 2024-09-21T04:15:16.023 INFO:tasks.workunit.client.0.smithi110.stdout:3/382: fdatasync d0/d12/d2a/f3e 0 2024-09-21T04:15:16.029 INFO:tasks.workunit.client.0.smithi110.stdout:3/383: dread d0/fe [0,4194304] 0 2024-09-21T04:15:16.029 INFO:tasks.workunit.client.0.smithi110.stdout:3/384: read d0/d12/d16/d1a/f35 [2353142,81311] 0 2024-09-21T04:15:16.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/401: dwrite d5/d6/d7/d11/d48/f7d [0,4194304] 0 2024-09-21T04:15:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.027+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc010000bf0 con 0x7fc01803f990 2024-09-21T04:15:16.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.028+0000 7fc0227fc640 1 -- 172.21.15.110:0/4087857242 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fc010000bf0 con 0x7fc01803f990 2024-09-21T04:15:16.032 INFO:tasks.workunit.client.1.smithi148.stdout:1/371: dwrite db/d24/d2a/f42 [0,4194304] 0 2024-09-21T04:15:16.032 INFO:tasks.workunit.client.0.smithi110.stdout:0/707: dwrite d4/d47/d2c/d3a/f43 [0,4194304] 0 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.033+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 msgr2=0x7fc018041e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.033+0000 7fc0333f3640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 0x7fc018041e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc02c10e4d0 tx=0x7fc028009260 comp rx=0 tx=0).stop 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.033+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 msgr2=0x7fc02c110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc02402c0b0 tx=0x7fc02402dbc0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 shutdown_connections 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fc01803f990 0x7fc018041e50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc02c109de0 0x7fc02c110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 --2- 172.21.15.110:0/4087857242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c1074b0 0x7fc02c110550 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 >> 172.21.15.110:0/4087857242 conn(0x7fc02c06cf90 msgr2=0x7fc02c06f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:16.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 shutdown_connections 2024-09-21T04:15:16.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.034+0000 7fc0333f3640 1 -- 172.21.15.110:0/4087857242 wait complete. 2024-09-21T04:15:16.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/362: mkdir d3/d5/d6/d12/d15/d73 0 2024-09-21T04:15:16.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/363: fsync d3/d5/d6/d12/d43/d54/f71 0 2024-09-21T04:15:16.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/364: dread - d3/d5/d6/d12/da/d21/f4e zero size 2024-09-21T04:15:16.043 INFO:tasks.workunit.client.0.smithi110.stdout:0/708: dread d4/dd/d60/f7e [0,4194304] 0 2024-09-21T04:15:16.048 INFO:tasks.workunit.client.1.smithi148.stdout:7/418: mkdir d5/d14/d25/d40/d8b 0 2024-09-21T04:15:16.048 INFO:tasks.workunit.client.1.smithi148.stdout:2/503: dwrite d0/d9/dc/d13/d25/d4e/f6b [4194304,4194304] 0 2024-09-21T04:15:16.048 INFO:tasks.workunit.client.1.smithi148.stdout:1/372: dwrite db/d24/d2a/f42 [4194304,4194304] 0 2024-09-21T04:15:16.054 INFO:tasks.workunit.client.0.smithi110.stdout:1/686: rmdir d2 39 2024-09-21T04:15:16.054 INFO:tasks.workunit.client.0.smithi110.stdout:1/687: dread - d2/d9/d26/d3d/d8f/fa3 zero size 2024-09-21T04:15:16.055 INFO:tasks.workunit.client.0.smithi110.stdout:1/688: write d2/d9/d23/f75 [368359,85295] 0 2024-09-21T04:15:16.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/373: symlink d0/d34/d38/d57/l6e 0 2024-09-21T04:15:16.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/374: fsync d0/d1a/d23/f12 0 2024-09-21T04:15:16.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/375: readlink d0/l65 0 2024-09-21T04:15:16.061 INFO:tasks.workunit.client.1.smithi148.stdout:8/402: mknod dc/d13/d76/d5d/d78/c7f 0 2024-09-21T04:15:16.061 INFO:tasks.workunit.client.1.smithi148.stdout:8/403: read dc/d13/d3c/f49 [417974,91347] 0 2024-09-21T04:15:16.061 INFO:tasks.workunit.client.1.smithi148.stdout:8/404: fsync fb 0 2024-09-21T04:15:16.061 INFO:tasks.workunit.client.1.smithi148.stdout:8/405: fdatasync dc/d43/f47 0 2024-09-21T04:15:16.066 INFO:tasks.workunit.client.1.smithi148.stdout:2/504: dwrite d0/d2b/d48/f5c [0,4194304] 0 2024-09-21T04:15:16.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/406: rename d4/d12/d17/d2e/d40/d65 to d4/d12/d16/d66/d7a/d4b/d7f 0 2024-09-21T04:15:16.070 INFO:tasks.workunit.client.0.smithi110.stdout:3/385: creat d0/d22/d29/f8b x:0 0 0 2024-09-21T04:15:16.077 INFO:tasks.workunit.client.0.smithi110.stdout:0/709: mkdir d4/d1c/d62/d91/dd6 0 2024-09-21T04:15:16.077 INFO:tasks.workunit.client.0.smithi110.stdout:0/710: write d4/dd/d10/d8a/d92/fb6 [1878528,48681] 0 2024-09-21T04:15:16.080 INFO:tasks.workunit.client.1.smithi148.stdout:0/402: link d5/d6/d7/d11/f17 d5/d6/d2a/d45/d70/f7f 0 2024-09-21T04:15:16.085 INFO:tasks.workunit.client.0.smithi110.stdout:6/407: dwrite d4/d12/f1d [0,4194304] 0 2024-09-21T04:15:16.085 INFO:tasks.workunit.client.0.smithi110.stdout:6/408: fsync d4/fb 0 2024-09-21T04:15:16.085 INFO:tasks.workunit.client.0.smithi110.stdout:6/409: chown d4/d12/f34 25 1 2024-09-21T04:15:16.088 INFO:tasks.workunit.client.0.smithi110.stdout:8/480: mkdir d0/d7/d13/d4b/da7 0 2024-09-21T04:15:16.093 INFO:tasks.workunit.client.0.smithi110.stdout:6/410: dread f3 [0,4194304] 0 2024-09-21T04:15:16.095 INFO:tasks.workunit.client.0.smithi110.stdout:3/386: creat d0/d51/d7b/f8c x:0 0 0 2024-09-21T04:15:16.096 INFO:tasks.workunit.client.1.smithi148.stdout:3/365: getdents d3/d5/d6/d12/da/d23/d28 0 2024-09-21T04:15:16.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/711: mkdir d4/dd/d10/d8a/db5/dd7 0 2024-09-21T04:15:16.098 INFO:tasks.workunit.client.1.smithi148.stdout:7/419: mkdir d5/d14/d25/d40/d8c 0 2024-09-21T04:15:16.103 INFO:tasks.workunit.client.1.smithi148.stdout:1/373: link db/dd/c32 db/d46/c77 0 2024-09-21T04:15:16.103 INFO:tasks.workunit.client.0.smithi110.stdout:8/481: fsync d0/d7/d13/d4b/f27 0 2024-09-21T04:15:16.105 INFO:tasks.workunit.client.0.smithi110.stdout:1/689: rmdir d2/d9/d23/de0/dbb 0 2024-09-21T04:15:16.105 INFO:tasks.workunit.client.1.smithi148.stdout:6/376: symlink d0/d34/l6f 0 2024-09-21T04:15:16.110 INFO:tasks.workunit.client.0.smithi110.stdout:8/482: mkdir d0/d30/d54/d6b/da8 0 2024-09-21T04:15:16.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/387: write d0/d12/d3c/f5e [836176,24415] 0 2024-09-21T04:15:16.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/388: creat d0/d51/d7b/f8d x:0 0 0 2024-09-21T04:15:16.111 INFO:tasks.workunit.client.0.smithi110.stdout:1/690: mkdir d2/d88/de1 0 2024-09-21T04:15:16.111 INFO:tasks.workunit.client.0.smithi110.stdout:1/691: write d2/d9/d49/d7a/da8/fc6 [877578,97122] 0 2024-09-21T04:15:16.113 INFO:tasks.workunit.client.0.smithi110.stdout:0/712: write d4/dd/d10/f99 [2729228,15705] 0 2024-09-21T04:15:16.114 INFO:tasks.workunit.client.0.smithi110.stdout:8/483: write d0/d7/d13/d4b/f7d [644526,45299] 0 2024-09-21T04:15:16.117 INFO:tasks.workunit.client.0.smithi110.stdout:6/411: dwrite d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:16.117 INFO:tasks.workunit.client.0.smithi110.stdout:6/412: write d4/d12/f2b [4271665,125840] 0 2024-09-21T04:15:16.119 INFO:tasks.workunit.client.1.smithi148.stdout:2/505: unlink d0/d2b/d48/f57 0 2024-09-21T04:15:16.120 INFO:tasks.workunit.client.1.smithi148.stdout:2/506: dread d0/d9/d38/f4f [0,4194304] 0 2024-09-21T04:15:16.120 INFO:tasks.workunit.client.1.smithi148.stdout:2/507: fsync d0/d9/dc/d13/d4c/f8a 0 2024-09-21T04:15:16.120 INFO:tasks.workunit.client.1.smithi148.stdout:2/508: dread - d0/d2b/f98 zero size 2024-09-21T04:15:16.121 INFO:tasks.workunit.client.0.smithi110.stdout:3/389: mknod d0/d12/d3c/c8e 0 2024-09-21T04:15:16.125 INFO:tasks.workunit.client.0.smithi110.stdout:1/692: symlink d2/d9/d49/d5c/le2 0 2024-09-21T04:15:16.129 INFO:tasks.workunit.client.0.smithi110.stdout:0/713: creat d4/dd/d10/d29/d5f/fd8 x:0 0 0 2024-09-21T04:15:16.129 INFO:tasks.workunit.client.0.smithi110.stdout:0/714: write d4/d47/fb2 [1635649,10236] 0 2024-09-21T04:15:16.130 INFO:tasks.workunit.client.0.smithi110.stdout:0/715: chown d4/d47/fb2 362519 1 2024-09-21T04:15:16.131 INFO:tasks.workunit.client.0.smithi110.stdout:8/484: getdents d0/d7/d13/d1c/d41 0 2024-09-21T04:15:16.132 INFO:tasks.workunit.client.0.smithi110.stdout:2/246: sync 2024-09-21T04:15:16.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/247: dread d2/f3b [0,4194304] 0 2024-09-21T04:15:16.134 INFO:tasks.workunit.client.1.smithi148.stdout:3/366: mkdir d3/d5/d6/d12/da/d74 0 2024-09-21T04:15:16.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/420: mknod d5/dc/d28/c8d 0 2024-09-21T04:15:16.137 INFO:tasks.workunit.client.1.smithi148.stdout:1/374: link db/d24/c56 db/d46/d4c/c78 0 2024-09-21T04:15:16.137 INFO:tasks.workunit.client.1.smithi148.stdout:1/375: write db/d24/f2b [1435734,56451] 0 2024-09-21T04:15:16.138 INFO:tasks.workunit.client.0.smithi110.stdout:6/413: truncate d4/de/f11 3771962 0 2024-09-21T04:15:16.138 INFO:tasks.workunit.client.0.smithi110.stdout:6/414: stat d4/d12/d16/d66 0 2024-09-21T04:15:16.139 INFO:tasks.workunit.client.1.smithi148.stdout:6/377: symlink d0/d34/d38/l70 0 2024-09-21T04:15:16.141 INFO:tasks.workunit.client.1.smithi148.stdout:3/367: write d3/d5/d6/d12/da/d52/f4a [3522676,1404] 0 2024-09-21T04:15:16.142 INFO:tasks.workunit.client.1.smithi148.stdout:8/406: getdents dc/d13/d39/d6e 0 2024-09-21T04:15:16.142 INFO:tasks.workunit.client.1.smithi148.stdout:8/407: stat dc/d13/c69 0 2024-09-21T04:15:16.142 INFO:tasks.workunit.client.1.smithi148.stdout:8/408: write dc/f18 [308299,38566] 0 2024-09-21T04:15:16.142 INFO:tasks.workunit.client.1.smithi148.stdout:8/409: stat dc/d43/f47 0 2024-09-21T04:15:16.142 INFO:tasks.workunit.client.0.smithi110.stdout:6/415: dread d4/fc [0,4194304] 0 2024-09-21T04:15:16.143 INFO:tasks.workunit.client.1.smithi148.stdout:8/410: dread dc/d13/d3c/f49 [0,4194304] 0 2024-09-21T04:15:16.143 INFO:tasks.workunit.client.1.smithi148.stdout:8/411: fdatasync dc/d13/d3c/f5f 0 2024-09-21T04:15:16.144 INFO:tasks.workunit.client.1.smithi148.stdout:4/456: rename d4/d23 to d4/d66/d7f 0 2024-09-21T04:15:16.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/509: truncate d0/d9/dc/f99 1482668 0 2024-09-21T04:15:16.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/510: creat d0/d2b/f9b x:0 0 0 2024-09-21T04:15:16.147 INFO:tasks.workunit.client.1.smithi148.stdout:7/421: unlink d5/d8/f89 0 2024-09-21T04:15:16.148 INFO:tasks.workunit.client.0.smithi110.stdout:1/693: symlink d2/d9/d49/d7a/le3 0 2024-09-21T04:15:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/412: read dc/f14 [1173525,77561] 0 2024-09-21T04:15:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/413: write dc/d13/f24 [482355,59346] 0 2024-09-21T04:15:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:8/414: getdents dc/d43/d4c 0 2024-09-21T04:15:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/376: mknod db/d24/c79 0 2024-09-21T04:15:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/377: write db/f64 [1035700,61836] 0 2024-09-21T04:15:16.155 INFO:tasks.workunit.client.0.smithi110.stdout:0/716: mknod d4/d1c/d62/cd9 0 2024-09-21T04:15:16.155 INFO:tasks.workunit.client.0.smithi110.stdout:0/717: readlink d4/dd/d10/d29/l95 0 2024-09-21T04:15:16.155 INFO:tasks.workunit.client.0.smithi110.stdout:0/718: chown d4/d47/l87 1159 1 2024-09-21T04:15:16.155 INFO:tasks.workunit.client.0.smithi110.stdout:0/719: readlink d4/dd/l54 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/368: truncate d3/d5/d6/d12/da/d23/f20 2144255 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/369: creat d3/d5/d6/d12/f75 x:0 0 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/370: dread - d3/d5/d6/d12/f66 zero size 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/371: readlink d3/l45 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/372: write d3/d5/d6/d12/da/d23/d28/f42 [4251176,1296] 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/373: fsync d3/f49 0 2024-09-21T04:15:16.156 INFO:tasks.workunit.client.1.smithi148.stdout:3/374: chown d3/l51 10 1 2024-09-21T04:15:16.159 INFO:tasks.workunit.client.1.smithi148.stdout:0/403: rename d5/d6/d7/d22/f59 to d5/d6/d7/d11/d48/f80 0 2024-09-21T04:15:16.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/485: dwrite d0/d30/d34/d3e/d32/f7c [0,4194304] 0 2024-09-21T04:15:16.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/486: fsync d0/d30/d54/d6b/f75 0 2024-09-21T04:15:16.163 INFO:tasks.workunit.client.0.smithi110.stdout:6/416: mkdir d4/d12/d16/d6d/d80 0 2024-09-21T04:15:16.163 INFO:tasks.workunit.client.1.smithi148.stdout:1/378: read db/f1b [2484717,102165] 0 2024-09-21T04:15:16.164 INFO:tasks.workunit.client.1.smithi148.stdout:2/511: creat d0/f9c x:0 0 0 2024-09-21T04:15:16.165 INFO:tasks.workunit.client.1.smithi148.stdout:2/512: fdatasync d0/d9/dc/fd 0 2024-09-21T04:15:16.166 INFO:tasks.workunit.client.0.smithi110.stdout:1/694: creat d2/d9/d23/d4e/fe4 x:0 0 0 2024-09-21T04:15:16.167 INFO:tasks.workunit.client.1.smithi148.stdout:0/404: write d5/d6/d7/d22/f46 [7518320,27015] 0 2024-09-21T04:15:16.167 INFO:tasks.workunit.client.1.smithi148.stdout:6/378: dwrite d0/d1a/d23/d16/f3f [0,4194304] 0 2024-09-21T04:15:16.167 INFO:tasks.workunit.client.1.smithi148.stdout:6/379: write d0/d8/f10 [4362787,113437] 0 2024-09-21T04:15:16.167 INFO:tasks.workunit.client.1.smithi148.stdout:7/422: symlink d5/d14/d25/l8e 0 2024-09-21T04:15:16.168 INFO:tasks.workunit.client.1.smithi148.stdout:4/457: dwrite d4/d12/d16/d29/f33 [0,4194304] 0 2024-09-21T04:15:16.168 INFO:tasks.workunit.client.0.smithi110.stdout:0/720: creat d4/d8b/fda x:0 0 0 2024-09-21T04:15:16.168 INFO:tasks.workunit.client.0.smithi110.stdout:0/721: creat d4/d47/d2c/fdb x:0 0 0 2024-09-21T04:15:16.168 INFO:tasks.workunit.client.0.smithi110.stdout:0/722: stat d4/d1c/c73 0 2024-09-21T04:15:16.170 INFO:tasks.workunit.client.1.smithi148.stdout:8/415: mkdir dc/d13/d76/d50/d80 0 2024-09-21T04:15:16.171 INFO:tasks.workunit.client.0.smithi110.stdout:2/248: getdents d2/d1a/d1f/d3f 0 2024-09-21T04:15:16.175 INFO:tasks.workunit.client.1.smithi148.stdout:3/375: mkdir d3/d5/d6/d12/d76 0 2024-09-21T04:15:16.175 INFO:tasks.workunit.client.0.smithi110.stdout:9/353: sync 2024-09-21T04:15:16.175 INFO:tasks.workunit.client.0.smithi110.stdout:5/484: sync 2024-09-21T04:15:16.175 INFO:tasks.workunit.client.0.smithi110.stdout:5/485: dread - d1/d49/d4d/d1e/d4a/f9f zero size 2024-09-21T04:15:16.175 INFO:tasks.workunit.client.0.smithi110.stdout:5/486: write d1/d39/fa4 [458005,104580] 0 2024-09-21T04:15:16.177 INFO:tasks.workunit.client.1.smithi148.stdout:1/379: creat db/dd/d21/f7a x:0 0 0 2024-09-21T04:15:16.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.173+0000 7f4bb7a08640 1 -- 172.21.15.110:0/609447092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100050 msgr2=0x7f4bb0100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.173+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/609447092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100050 0x7f4bb0100450 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009880 tx=0x7f4bac02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.174+0000 7f4bb7a08640 1 -- 172.21.15.110:0/609447092 shutdown_connections 2024-09-21T04:15:16.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.174+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/609447092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100a20 0x7f4bb00fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.174+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/609447092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100050 0x7f4bb0100450 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.174+0000 7f4bb7a08640 1 -- 172.21.15.110:0/609447092 >> 172.21.15.110:0/609447092 conn(0x7f4bb00f99b0 msgr2=0x7f4bb00fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:16.179 INFO:tasks.workunit.client.1.smithi148.stdout:5/391: sync 2024-09-21T04:15:16.179 INFO:tasks.workunit.client.1.smithi148.stdout:9/390: sync 2024-09-21T04:15:16.179 INFO:tasks.workunit.client.1.smithi148.stdout:5/392: fdatasync d3/d14/d1f/d26/d2e/d3d/f61 0 2024-09-21T04:15:16.180 INFO:tasks.workunit.client.0.smithi110.stdout:2/249: write d2/d1a/d1f/d3f/f1d [2302693,101820] 0 2024-09-21T04:15:16.180 INFO:tasks.workunit.client.0.smithi110.stdout:2/250: fsync d2/f16 0 2024-09-21T04:15:16.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.180+0000 7f4bb7a08640 1 -- 172.21.15.110:0/609447092 shutdown_connections 2024-09-21T04:15:16.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/376: dread d3/d5/d6/d12/da/d52/f4a [0,4194304] 0 2024-09-21T04:15:16.183 INFO:tasks.workunit.client.0.smithi110.stdout:8/487: rename d0/d7/d13/f1e to d0/d30/d34/d3e/d32/da3/fa9 0 2024-09-21T04:15:16.183 INFO:tasks.workunit.client.0.smithi110.stdout:8/488: chown d0/l15 6 1 2024-09-21T04:15:16.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.180+0000 7f4bb7a08640 1 -- 172.21.15.110:0/609447092 wait complete. 2024-09-21T04:15:16.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.181+0000 7f4bb7a08640 1 Processor -- start 2024-09-21T04:15:16.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.181+0000 7f4bb7a08640 1 -- start start 2024-09-21T04:15:16.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.181+0000 7f4bb7a08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb7a08640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100a20 0x7f4bb010ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb57a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb57a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:58514/0 (socket says 172.21.15.110:58514) 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb57a4640 1 -- 172.21.15.110:0/1670429014 learned_addr learned my addr 172.21.15.110:0/1670429014 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb010f200 con 0x7f4bb0100a20 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb010f370 con 0x7f4bb0100050 2024-09-21T04:15:16.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.182+0000 7f4bb4fa3640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100a20 0x7f4bb010ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4bb57a4640 1 -- 172.21.15.110:0/1670429014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100a20 msgr2=0x7f4bb010ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4bb57a4640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100a20 0x7f4bb010ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4bb57a4640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009560 con 0x7f4bb0100050 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4bb57a4640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4bac006010 tx=0x7f4bac0055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bac004aa0 con 0x7f4bb0100050 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bac004c00 con 0x7f4bb0100050 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.183+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bac003ac0 con 0x7f4bb0100050 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.184+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb010fdc0 con 0x7f4bb0100050 2024-09-21T04:15:16.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.184+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb01101f0 con 0x7f4bb0100050 2024-09-21T04:15:16.188 INFO:tasks.workunit.client.0.smithi110.stdout:1/695: rmdir d2/d9/d26/d3d 39 2024-09-21T04:15:16.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.186+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 51935+0+0 (secure 0 0 0) 0x7f4bac003d30 con 0x7f4bb0100050 2024-09-21T04:15:16.189 INFO:tasks.workunit.client.1.smithi148.stdout:9/391: dread d5/d2b/d32/f5b [0,4194304] 0 2024-09-21T04:15:16.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.186+0000 7f4ba67fc640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 0x7f4b9c041fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.187+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4bac0777d0 con 0x7f4bb0100050 2024-09-21T04:15:16.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.187+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b94005280 con 0x7f4bb0100050 2024-09-21T04:15:16.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.187+0000 7f4bb4fa3640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 0x7f4b9c041fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.188+0000 7f4bb4fa3640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 0x7f4b9c041fe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005d00 tx=0x7f4ba0005c50 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/354: symlink d7/d10/d24/d3c/d1f/l7a 0 2024-09-21T04:15:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/355: chown d7/d10/d24/d3c/d4f/f64 688236 1 2024-09-21T04:15:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/356: stat d7/d10/d24/d3c/d14/d5c/d6f/l70 0 2024-09-21T04:15:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/357: truncate d7/d26/f5b 555470 0 2024-09-21T04:15:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.193+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f4bac04d950 con 0x7f4bb0100050 2024-09-21T04:15:16.196 INFO:tasks.workunit.client.1.smithi148.stdout:0/405: truncate d5/d6/d7/d18/f32 1207274 0 2024-09-21T04:15:16.196 INFO:tasks.workunit.client.1.smithi148.stdout:0/406: readlink d5/d6/d7/d11/l5f 0 2024-09-21T04:15:16.197 INFO:tasks.workunit.client.0.smithi110.stdout:5/487: dwrite d1/d39/f79 [0,4194304] 0 2024-09-21T04:15:16.198 INFO:tasks.workunit.client.1.smithi148.stdout:6/380: mknod d0/d1a/d23/dc/d68/d5b/c71 0 2024-09-21T04:15:16.198 INFO:tasks.workunit.client.1.smithi148.stdout:6/381: fsync d0/d1a/d23/f3d 0 2024-09-21T04:15:16.198 INFO:tasks.workunit.client.1.smithi148.stdout:6/382: dread - d0/d1a/d23/f61 zero size 2024-09-21T04:15:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/383: dread d0/d1a/d23/f3d [0,4194304] 0 2024-09-21T04:15:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/384: write d0/d1a/d23/f19 [1051994,40624] 0 2024-09-21T04:15:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:7/423: mknod d5/d14/d25/d40/d8b/c8f 0 2024-09-21T04:15:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:7/424: read d5/d8/f9 [288670,74366] 0 2024-09-21T04:15:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:7/425: write d5/f5a [649901,43673] 0 2024-09-21T04:15:16.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/489: mknod d0/d7/d23/d82/caa 0 2024-09-21T04:15:16.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/490: stat d0/d7/d13/l2c 0 2024-09-21T04:15:16.203 INFO:tasks.workunit.client.0.smithi110.stdout:6/417: rename d4/d12/d16/d6d/d80 to d4/d12/d17/d81 0 2024-09-21T04:15:16.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/458: getdents d4/d66 0 2024-09-21T04:15:16.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/459: chown d4/d66/d7f/c5e 5218 1 2024-09-21T04:15:16.207 INFO:tasks.workunit.client.0.smithi110.stdout:1/696: getdents d2 0 2024-09-21T04:15:16.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/416: mkdir dc/d13/d3c/d81 0 2024-09-21T04:15:16.208 INFO:tasks.workunit.client.1.smithi148.stdout:8/417: write dc/d13/d28/f48 [1330905,2731] 0 2024-09-21T04:15:16.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/380: creat db/d24/d2e/f7b x:0 0 0 2024-09-21T04:15:16.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/381: truncate db/d46/f53 1356120 0 2024-09-21T04:15:16.212 INFO:tasks.workunit.client.0.smithi110.stdout:9/358: unlink d7/d10/d24/d3c/d1f/l7a 0 2024-09-21T04:15:16.214 INFO:tasks.workunit.client.0.smithi110.stdout:8/491: creat d0/d7/d13/d4b/fab x:0 0 0 2024-09-21T04:15:16.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/377: truncate d3/d5/f1e 2502512 0 2024-09-21T04:15:16.217 INFO:tasks.workunit.client.1.smithi148.stdout:3/378: chown d3/d5/d6/d12/d15/c3c 12008 1 2024-09-21T04:15:16.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/379: dread d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:16.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/380: chown d3/d5/d6/d12/da/d23/f20 829431 1 2024-09-21T04:15:16.220 INFO:tasks.workunit.client.0.smithi110.stdout:6/418: mknod d4/d12/d17/d2e/c82 0 2024-09-21T04:15:16.221 INFO:tasks.workunit.client.1.smithi148.stdout:2/513: rmdir d0/d9/dc/d13/d25/d4e 39 2024-09-21T04:15:16.222 INFO:tasks.workunit.client.0.smithi110.stdout:6/419: dread d4/d12/d17/d2e/d38/f4f [0,4194304] 0 2024-09-21T04:15:16.222 INFO:tasks.workunit.client.0.smithi110.stdout:0/723: rename d4/dd/d53/fb9 to d4/dd/d10/d29/fdc 0 2024-09-21T04:15:16.222 INFO:tasks.workunit.client.0.smithi110.stdout:0/724: write d4/d47/fab [749325,71562] 0 2024-09-21T04:15:16.226 INFO:tasks.workunit.client.0.smithi110.stdout:1/697: mknod d2/d9/d23/d3b/dca/ce5 0 2024-09-21T04:15:16.226 INFO:tasks.workunit.client.0.smithi110.stdout:1/698: chown d2/d9/d23/d4e/d57 253391 1 2024-09-21T04:15:16.229 INFO:tasks.workunit.client.1.smithi148.stdout:9/392: creat d5/f96 x:0 0 0 2024-09-21T04:15:16.229 INFO:tasks.workunit.client.1.smithi148.stdout:9/393: stat d5/d1e/c94 0 2024-09-21T04:15:16.229 INFO:tasks.workunit.client.0.smithi110.stdout:6/420: mkdir d4/d12/d16/d66/d7a/d83 0 2024-09-21T04:15:16.232 INFO:tasks.workunit.client.0.smithi110.stdout:8/492: rename d0/d7/d13/f97 to d0/d7/d13/d4b/da7/fac 0 2024-09-21T04:15:16.235 INFO:tasks.workunit.client.0.smithi110.stdout:0/725: unlink d4/dd/d53/d96/fb0 0 2024-09-21T04:15:16.237 INFO:tasks.workunit.client.0.smithi110.stdout:1/699: dread d2/f94 [0,4194304] 0 2024-09-21T04:15:16.237 INFO:tasks.workunit.client.0.smithi110.stdout:1/700: truncate d2/d9/d23/f29 941976 0 2024-09-21T04:15:16.237 INFO:tasks.workunit.client.0.smithi110.stdout:1/701: chown d2/ccb 44 1 2024-09-21T04:15:16.238 INFO:tasks.workunit.client.0.smithi110.stdout:0/726: write d4/dd/d53/f79 [1525945,4378] 0 2024-09-21T04:15:16.238 INFO:tasks.workunit.client.0.smithi110.stdout:0/727: readlink d4/d1c/d62/d69/lbf 0 2024-09-21T04:15:16.238 INFO:tasks.workunit.client.0.smithi110.stdout:0/728: read d4/dd/d10/d29/d45/f5b [1720289,15654] 0 2024-09-21T04:15:16.242 INFO:tasks.workunit.client.1.smithi148.stdout:6/385: mkdir d0/d1a/d23/d16/d36/d72 0 2024-09-21T04:15:16.245 INFO:tasks.workunit.client.1.smithi148.stdout:9/394: dwrite d5/d60/d28/f30 [0,4194304] 0 2024-09-21T04:15:16.246 INFO:tasks.workunit.client.1.smithi148.stdout:4/460: mknod d4/d66/d7f/c80 0 2024-09-21T04:15:16.246 INFO:tasks.workunit.client.1.smithi148.stdout:4/461: write d4/f51 [661799,125490] 0 2024-09-21T04:15:16.247 INFO:tasks.workunit.client.1.smithi148.stdout:8/418: symlink dc/d13/d76/d50/d80/l82 0 2024-09-21T04:15:16.249 INFO:tasks.workunit.client.1.smithi148.stdout:1/382: rename db/d24/c40 to db/d46/d76/c7c 0 2024-09-21T04:15:16.249 INFO:tasks.workunit.client.1.smithi148.stdout:1/383: creat db/d24/d4f/f7d x:0 0 0 2024-09-21T04:15:16.250 INFO:tasks.workunit.client.1.smithi148.stdout:3/381: rmdir d3/d5/d6/d12/da/d23/d28 39 2024-09-21T04:15:16.251 INFO:tasks.workunit.client.1.smithi148.stdout:2/514: rmdir d0/d9/d16/d55 39 2024-09-21T04:15:16.252 INFO:tasks.workunit.client.1.smithi148.stdout:2/515: truncate d0/f9c 580591 0 2024-09-21T04:15:16.252 INFO:tasks.workunit.client.1.smithi148.stdout:2/516: write d0/d9/d16/f6a [248924,11518] 0 2024-09-21T04:15:16.252 INFO:tasks.workunit.client.0.smithi110.stdout:9/359: rmdir d7/d10 39 2024-09-21T04:15:16.252 INFO:tasks.workunit.client.0.smithi110.stdout:0/729: dread d4/dd/d10/d29/d45/fca [0,4194304] 0 2024-09-21T04:15:16.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/421: rename d4/d12/d17/d33/f55 to d4/de/f84 0 2024-09-21T04:15:16.255 INFO:tasks.workunit.client.1.smithi148.stdout:4/462: write d4/d12/d16/f1f [724888,122784] 0 2024-09-21T04:15:16.255 INFO:tasks.workunit.client.1.smithi148.stdout:0/407: getdents d5/d6/d2a/d2b 0 2024-09-21T04:15:16.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/408: stat d5/d6/d2a/d45 0 2024-09-21T04:15:16.256 INFO:tasks.workunit.client.0.smithi110.stdout:8/493: mkdir d0/d30/d9d/dad 0 2024-09-21T04:15:16.256 INFO:tasks.workunit.client.0.smithi110.stdout:8/494: readlink d0/l12 0 2024-09-21T04:15:16.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/386: rmdir d0/d1a/d23/d16/d36/d72 0 2024-09-21T04:15:16.259 INFO:tasks.workunit.client.0.smithi110.stdout:1/702: mkdir d2/d9/d23/d3b/d9c/de6 0 2024-09-21T04:15:16.266 INFO:tasks.workunit.client.1.smithi148.stdout:9/395: creat d5/d2b/d4e/f97 x:0 0 0 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/360: creat d7/d10/d24/d3c/d1f/d43/f7b x:0 0 0 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/361: readlink d7/d10/d24/d3c/d1f/d72/d59/l65 0 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/362: dread - d7/d10/d24/d3c/d1f/d72/f63 zero size 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:0/730: rmdir d4/d47/d2c/d3a 39 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/495: dread d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/496: write d0/d7/d13/f5a [1415540,78763] 0 2024-09-21T04:15:16.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/497: chown d0/d7/d13/f57 925570100 1 2024-09-21T04:15:16.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/419: rename fb to dc/d13/d76/d5d/f83 0 2024-09-21T04:15:16.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/420: write dc/d13/d3c/f4b [5125110,72270] 0 2024-09-21T04:15:16.269 INFO:tasks.workunit.client.0.smithi110.stdout:6/422: rename d4/d12/d17/d33/d76/l60 to d4/d12/d16/d6d/l85 0 2024-09-21T04:15:16.271 INFO:tasks.workunit.client.1.smithi148.stdout:1/384: rmdir db/d46/d76 39 2024-09-21T04:15:16.271 INFO:tasks.workunit.client.1.smithi148.stdout:1/385: chown db/dd/d21/f5c 41 1 2024-09-21T04:15:16.272 INFO:tasks.workunit.client.0.smithi110.stdout:1/703: dread d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:16.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/517: creat d0/d9/dc/d13/d25/d4e/d8f/f9d x:0 0 0 2024-09-21T04:15:16.273 INFO:tasks.workunit.client.1.smithi148.stdout:9/396: creat d5/d39/f98 x:0 0 0 2024-09-21T04:15:16.274 INFO:tasks.workunit.client.0.smithi110.stdout:9/363: creat d7/d10/d24/d3c/d1f/d68/f7c x:0 0 0 2024-09-21T04:15:16.274 INFO:tasks.workunit.client.0.smithi110.stdout:9/364: readlink d7/d10/d24/d3c/d1f/d43/l62 0 2024-09-21T04:15:16.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/409: dwrite d5/d6/f16 [0,4194304] 0 2024-09-21T04:15:16.276 INFO:tasks.workunit.client.1.smithi148.stdout:4/463: rename d4/d66/d7f/c5e to d4/d12/c81 0 2024-09-21T04:15:16.279 INFO:tasks.workunit.client.1.smithi148.stdout:2/518: mkdir d0/d9/d16/d9e 0 2024-09-21T04:15:16.280 INFO:tasks.workunit.client.1.smithi148.stdout:2/519: write d0/d9/dc/d13/d4c/f8a [582366,6380] 0 2024-09-21T04:15:16.280 INFO:tasks.workunit.client.0.smithi110.stdout:8/498: mknod d0/d7/d89/cae 0 2024-09-21T04:15:16.281 INFO:tasks.workunit.client.1.smithi148.stdout:1/386: read db/dd/f23 [4060954,33691] 0 2024-09-21T04:15:16.282 INFO:tasks.workunit.client.1.smithi148.stdout:2/520: dread d0/d9/dc/f99 [0,4194304] 0 2024-09-21T04:15:16.283 INFO:tasks.workunit.client.0.smithi110.stdout:6/423: mkdir d4/d86 0 2024-09-21T04:15:16.283 INFO:tasks.workunit.client.1.smithi148.stdout:0/410: write d5/d6/f63 [1495294,110616] 0 2024-09-21T04:15:16.284 INFO:tasks.workunit.client.1.smithi148.stdout:6/387: rename d0/d1a/d23/d16/d36/l3b to d0/d1a/d23/d53/l73 0 2024-09-21T04:15:16.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/704: mkdir d2/d9/d26/d3d/d8f/de7 0 2024-09-21T04:15:16.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/521: mkdir d0/d9/dc/d13/d25/d4e/d8f/d9f 0 2024-09-21T04:15:16.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/522: readlink d0/d9/dc/d13/d25/l75 0 2024-09-21T04:15:16.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/523: truncate d0/d2b/f76 1817674 0 2024-09-21T04:15:16.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/524: write d0/d9/f6e [827723,94218] 0 2024-09-21T04:15:16.287 INFO:tasks.workunit.client.0.smithi110.stdout:0/731: dread d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:16.287 INFO:tasks.workunit.client.0.smithi110.stdout:0/732: write d4/dd/d10/d29/d75/fa9 [951109,58874] 0 2024-09-21T04:15:16.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/387: creat db/d24/d2e/d59/f7e x:0 0 0 2024-09-21T04:15:16.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/388: stat db/d46/f47 0 2024-09-21T04:15:16.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/397: rename d5/d60/d28/d73/f87 to d5/d60/d28/d2f/d8e/f99 0 2024-09-21T04:15:16.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/398: write d5/d2b/d32/f5a [837790,39428] 0 2024-09-21T04:15:16.290 INFO:tasks.workunit.client.1.smithi148.stdout:6/388: unlink d0/d8/c40 0 2024-09-21T04:15:16.295 INFO:tasks.workunit.client.1.smithi148.stdout:1/389: mknod db/dd/c7f 0 2024-09-21T04:15:16.295 INFO:tasks.workunit.client.0.smithi110.stdout:1/705: write d2/d9/d49/f77 [2636504,110826] 0 2024-09-21T04:15:16.295 INFO:tasks.workunit.client.0.smithi110.stdout:1/706: chown d2/d9/d26/f8b 4165304 1 2024-09-21T04:15:16.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/382: dwrite d3/d5/d6/d12/da/d23/f26 [4194304,4194304] 0 2024-09-21T04:15:16.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/383: write d3/f49 [368766,58337] 0 2024-09-21T04:15:16.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/421: dwrite dc/d13/d76/f5b [0,4194304] 0 2024-09-21T04:15:16.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/384: write d3/d5/d6/d12/da/d23/f20 [2149002,56500] 0 2024-09-21T04:15:16.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/525: rmdir d0/d2b 39 2024-09-21T04:15:16.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/526: write d0/d9/f17 [530869,20170] 0 2024-09-21T04:15:16.298 INFO:tasks.workunit.client.0.smithi110.stdout:6/424: write d4/d12/d16/d66/d7a/d4b/f51 [3138508,7178] 0 2024-09-21T04:15:16.298 INFO:tasks.workunit.client.0.smithi110.stdout:6/425: fsync d4/de/ff 0 2024-09-21T04:15:16.298 INFO:tasks.workunit.client.0.smithi110.stdout:9/365: dread d7/d10/d24/d3c/d1f/d68/f6b [0,4194304] 0 2024-09-21T04:15:16.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/411: dread d5/d6/f62 [0,4194304] 0 2024-09-21T04:15:16.299 INFO:tasks.workunit.client.1.smithi148.stdout:1/390: unlink db/c16 0 2024-09-21T04:15:16.300 INFO:tasks.workunit.client.0.smithi110.stdout:8/499: dread d0/f11 [4194304,4194304] 0 2024-09-21T04:15:16.310 INFO:tasks.workunit.client.0.smithi110.stdout:1/707: write d2/d9/d49/d7a/f7c [1371259,40339] 0 2024-09-21T04:15:16.315 INFO:tasks.workunit.client.1.smithi148.stdout:6/389: write d0/d8/d18/f4d [2850059,47774] 0 2024-09-21T04:15:16.315 INFO:tasks.workunit.client.1.smithi148.stdout:6/390: chown d0/d34/d38/l70 48631 1 2024-09-21T04:15:16.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/426: rename d4/de/ff to d4/d12/d16/d6d/f87 0 2024-09-21T04:15:16.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/427: dread - d4/d12/d16/d66/d7a/d4b/f5e zero size 2024-09-21T04:15:16.318 INFO:tasks.workunit.client.0.smithi110.stdout:9/366: mkdir d7/d10/d24/d3c/d14/d5c/d6f/d78/d7d 0 2024-09-21T04:15:16.318 INFO:tasks.workunit.client.0.smithi110.stdout:0/733: dwrite d4/dd/d10/d29/f6b [0,4194304] 0 2024-09-21T04:15:16.318 INFO:tasks.workunit.client.0.smithi110.stdout:0/734: write d4/d47/f23 [5084475,107867] 0 2024-09-21T04:15:16.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/422: dread dc/d13/d76/f40 [0,4194304] 0 2024-09-21T04:15:16.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/399: dwrite d5/d1e/d37/d46/f72 [0,4194304] 0 2024-09-21T04:15:16.325 INFO:tasks.workunit.client.0.smithi110.stdout:1/708: write d2/d9/d49/d4f/f67 [1488318,47284] 0 2024-09-21T04:15:16.325 INFO:tasks.workunit.client.0.smithi110.stdout:1/709: chown d2/d9/d49/d5c/c80 0 1 2024-09-21T04:15:16.329 INFO:tasks.workunit.client.1.smithi148.stdout:8/423: read dc/d13/d76/f5b [1803156,63592] 0 2024-09-21T04:15:16.329 INFO:tasks.workunit.client.1.smithi148.stdout:8/424: chown dc/d13/d76/f40 296 1 2024-09-21T04:15:16.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/425: dread dc/f7d [0,4194304] 0 2024-09-21T04:15:16.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/426: read dc/fe [2777731,84094] 0 2024-09-21T04:15:16.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/427: chown dc/d13/d28/l7a 5896563 1 2024-09-21T04:15:16.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/428: readlink dc/d13/d76/d5d/l74 0 2024-09-21T04:15:16.335 INFO:tasks.workunit.client.1.smithi148.stdout:2/527: dwrite d0/d9/dc/d13/f8b [0,4194304] 0 2024-09-21T04:15:16.335 INFO:tasks.workunit.client.1.smithi148.stdout:2/528: chown d0/d9/d16/d9e 5 1 2024-09-21T04:15:16.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/385: mknod d3/d1f/d4d/c77 0 2024-09-21T04:15:16.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/386: write d3/d5/d6/d12/f66 [557803,120497] 0 2024-09-21T04:15:16.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/387: getdents d3/d5/d6/d12/da/d74 0 2024-09-21T04:15:16.336 INFO:tasks.workunit.client.0.smithi110.stdout:6/428: truncate d4/f3d 598154 0 2024-09-21T04:15:16.336 INFO:tasks.workunit.client.0.smithi110.stdout:6/429: truncate d4/f13 5270142 0 2024-09-21T04:15:16.336 INFO:tasks.workunit.client.0.smithi110.stdout:6/430: fsync d4/d12/d16/f5b 0 2024-09-21T04:15:16.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/431: write d4/d12/d16/f50 [1874281,3341] 0 2024-09-21T04:15:16.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/432: stat d4/f6a 0 2024-09-21T04:15:16.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/433: chown d4/d12/d16/f50 39747 1 2024-09-21T04:15:16.337 INFO:tasks.workunit.client.1.smithi148.stdout:3/388: read d3/d1f/f3e [38987,42301] 0 2024-09-21T04:15:16.337 INFO:tasks.workunit.client.1.smithi148.stdout:3/389: write d3/f49 [283219,122392] 0 2024-09-21T04:15:16.339 INFO:tasks.workunit.client.0.smithi110.stdout:9/367: rmdir d7/d10/d24/d3c/d14/d5c/d6f/d78 39 2024-09-21T04:15:16.340 INFO:tasks.workunit.client.1.smithi148.stdout:6/391: dwrite d0/d1a/d23/dc/d68/f66 [0,4194304] 0 2024-09-21T04:15:16.344 INFO:tasks.workunit.client.0.smithi110.stdout:0/735: rename d4/c33 to d4/dd/d10/d29/d5f/dc3/cdd 0 2024-09-21T04:15:16.347 INFO:tasks.workunit.client.1.smithi148.stdout:0/412: getdents d5/d6/d2a 0 2024-09-21T04:15:16.347 INFO:tasks.workunit.client.1.smithi148.stdout:0/413: readlink d5/d6/d7/l33 0 2024-09-21T04:15:16.348 INFO:tasks.workunit.client.1.smithi148.stdout:1/391: rename db/l22 to db/dd/d21/l80 0 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:1/392: dread db/f64 [0,4194304] 0 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:1/393: chown db/c52 25320 1 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:6/392: dwrite d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:6/393: write d0/d8/f49 [410973,127677] 0 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:6/394: chown d0/d1a/d23/d16/d4f/f54 579 1 2024-09-21T04:15:16.355 INFO:tasks.workunit.client.1.smithi148.stdout:6/395: write d0/d1a/d23/f19 [2114291,16588] 0 2024-09-21T04:15:16.356 INFO:tasks.workunit.client.1.smithi148.stdout:0/414: write d5/d6/d3e/f26 [565124,73901] 0 2024-09-21T04:15:16.356 INFO:tasks.workunit.client.1.smithi148.stdout:0/415: chown d5/d6/d2a/d2b/f39 108 1 2024-09-21T04:15:16.356 INFO:tasks.workunit.client.1.smithi148.stdout:0/416: creat d5/d6/d7/d22/f81 x:0 0 0 2024-09-21T04:15:16.363 INFO:tasks.workunit.client.0.smithi110.stdout:1/710: creat d2/d9/d23/d4e/d57/dc9/fe8 x:0 0 0 2024-09-21T04:15:16.363 INFO:tasks.workunit.client.0.smithi110.stdout:1/711: write d2/d9/d26/f8b [2597330,121511] 0 2024-09-21T04:15:16.365 INFO:tasks.workunit.client.0.smithi110.stdout:8/500: truncate d0/d7/d13/d4b/f7e 3767627 0 2024-09-21T04:15:16.371 INFO:tasks.workunit.client.1.smithi148.stdout:9/400: creat d5/d60/d28/d38/f9a x:0 0 0 2024-09-21T04:15:16.372 INFO:tasks.workunit.client.0.smithi110.stdout:6/434: symlink d4/l88 0 2024-09-21T04:15:16.372 INFO:tasks.workunit.client.0.smithi110.stdout:6/435: dread - d4/f6a zero size 2024-09-21T04:15:16.372 INFO:tasks.workunit.client.0.smithi110.stdout:1/712: read d2/d9/d49/f5b [2187932,88735] 0 2024-09-21T04:15:16.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/429: mknod dc/d13/d28/c84 0 2024-09-21T04:15:16.379 INFO:tasks.workunit.client.1.smithi148.stdout:9/401: write d5/d1e/d37/d46/f7e [939407,80407] 0 2024-09-21T04:15:16.380 INFO:tasks.workunit.client.1.smithi148.stdout:2/529: symlink d0/d9/dc/d13/d25/d4e/d82/la0 0 2024-09-21T04:15:16.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/430: read dc/d13/d3c/f4b [4685673,79407] 0 2024-09-21T04:15:16.381 INFO:tasks.workunit.client.0.smithi110.stdout:0/736: link d4/dd/d53/f5a d4/d47/fde 0 2024-09-21T04:15:16.382 INFO:tasks.workunit.client.0.smithi110.stdout:1/713: write d2/d9/d26/d3d/f6f [623640,17107] 0 2024-09-21T04:15:16.382 INFO:tasks.workunit.client.0.smithi110.stdout:1/714: write d2/d9/d26/d3d/fd9 [494752,68268] 0 2024-09-21T04:15:16.383 INFO:tasks.workunit.client.1.smithi148.stdout:3/390: symlink d3/d5/d6/d12/d15/l78 0 2024-09-21T04:15:16.385 INFO:tasks.workunit.client.0.smithi110.stdout:8/501: creat d0/d7/d23/d52/d8b/faf x:0 0 0 2024-09-21T04:15:16.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/431: dread dc/d43/f47 [0,4194304] 0 2024-09-21T04:15:16.386 INFO:tasks.workunit.client.0.smithi110.stdout:9/368: rmdir d7/d10/d24/d3c/d1f/d43 39 2024-09-21T04:15:16.386 INFO:tasks.workunit.client.0.smithi110.stdout:9/369: readlink d7/l36 0 2024-09-21T04:15:16.393 INFO:tasks.workunit.client.1.smithi148.stdout:2/530: write d0/d9/dc/f46 [2516604,38034] 0 2024-09-21T04:15:16.393 INFO:tasks.workunit.client.1.smithi148.stdout:2/531: fdatasync d0/d9/dc/d13/d25/d2d/f2e 0 2024-09-21T04:15:16.398 INFO:tasks.workunit.client.0.smithi110.stdout:9/370: creat d7/d26/f7e x:0 0 0 2024-09-21T04:15:16.400 INFO:tasks.workunit.client.0.smithi110.stdout:1/715: write d2/d9/f6c [2961549,117922] 0 2024-09-21T04:15:16.400 INFO:tasks.workunit.client.0.smithi110.stdout:6/436: dwrite d4/f13 [0,4194304] 0 2024-09-21T04:15:16.404 INFO:tasks.workunit.client.0.smithi110.stdout:8/502: write d0/d7/d13/d1c/d41/f43 [3022373,116515] 0 2024-09-21T04:15:16.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/618: sync 2024-09-21T04:15:16.406 INFO:tasks.workunit.client.0.smithi110.stdout:7/414: sync 2024-09-21T04:15:16.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/619: write d17/d1e/d21/f5b [1490337,44113] 0 2024-09-21T04:15:16.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/620: read d17/d34/d3d/d57/f9f [1866785,100463] 0 2024-09-21T04:15:16.407 INFO:tasks.workunit.client.0.smithi110.stdout:4/621: stat d17/f58 0 2024-09-21T04:15:16.411 INFO:tasks.workunit.client.1.smithi148.stdout:3/391: dwrite d3/d5/d6/d12/da/d23/d28/d65/f67 [4194304,4194304] 0 2024-09-21T04:15:16.413 INFO:tasks.workunit.client.1.smithi148.stdout:1/394: getdents db/d46/d76 0 2024-09-21T04:15:16.413 INFO:tasks.workunit.client.1.smithi148.stdout:1/395: chown db/d24/f25 28093028 1 2024-09-21T04:15:16.415 INFO:tasks.workunit.client.0.smithi110.stdout:4/622: symlink d17/d34/d3d/d20/d43/d51/d52/d98/ld7 0 2024-09-21T04:15:16.419 INFO:tasks.workunit.client.1.smithi148.stdout:6/396: creat d0/d1a/d23/dc/d68/d5b/f74 x:0 0 0 2024-09-21T04:15:16.419 INFO:tasks.workunit.client.0.smithi110.stdout:0/737: dwrite d4/dd/d53/f5a [4194304,4194304] 0 2024-09-21T04:15:16.419 INFO:tasks.workunit.client.0.smithi110.stdout:0/738: write d4/dd/d10/d29/d45/f5b [2494501,66294] 0 2024-09-21T04:15:16.422 INFO:tasks.workunit.client.1.smithi148.stdout:2/532: dread d0/d9/dc/d13/d25/d4e/f6b [4194304,4194304] 0 2024-09-21T04:15:16.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/417: mkdir d5/d82 0 2024-09-21T04:15:16.424 INFO:tasks.workunit.client.1.smithi148.stdout:0/418: chown d5/d6/f63 1 1 2024-09-21T04:15:16.424 INFO:tasks.workunit.client.0.smithi110.stdout:4/623: rmdir d17/d1e/d7b 39 2024-09-21T04:15:16.425 INFO:tasks.workunit.client.0.smithi110.stdout:4/624: fdatasync d17/f32 0 2024-09-21T04:15:16.425 INFO:tasks.workunit.client.0.smithi110.stdout:4/625: creat d17/d34/d3d/d20/d43/d51/d7a/d96/fd8 x:0 0 0 2024-09-21T04:15:16.431 INFO:tasks.workunit.client.0.smithi110.stdout:9/371: dwrite d7/d10/d24/d3c/d1f/d43/f60 [0,4194304] 0 2024-09-21T04:15:16.433 INFO:tasks.workunit.client.1.smithi148.stdout:9/402: truncate d5/d1f/f4b 29335 0 2024-09-21T04:15:16.434 INFO:tasks.workunit.client.1.smithi148.stdout:8/432: mkdir dc/d13/d3c/d81/d85 0 2024-09-21T04:15:16.434 INFO:tasks.workunit.client.1.smithi148.stdout:8/433: readlink dc/d13/d28/l36 0 2024-09-21T04:15:16.434 INFO:tasks.workunit.client.1.smithi148.stdout:8/434: chown dc/d13/d39/d6e/f6f 4095 1 2024-09-21T04:15:16.437 INFO:tasks.workunit.client.1.smithi148.stdout:1/396: symlink db/d24/d2a/l81 0 2024-09-21T04:15:16.438 INFO:tasks.workunit.client.0.smithi110.stdout:4/626: link fa d17/d34/d3d/d57/fd9 0 2024-09-21T04:15:16.440 INFO:tasks.workunit.client.1.smithi148.stdout:6/397: creat d0/d34/d38/d57/f75 x:0 0 0 2024-09-21T04:15:16.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/426: sync 2024-09-21T04:15:16.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/427: fdatasync d5/dc/d28/f57 0 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.1.smithi148.stdout:4/464: sync 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.1.smithi148.stdout:6/398: write d0/d1a/d23/dc/d68/d5b/f4a [5467480,116901] 0 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.1.smithi148.stdout:2/533: rename d0/d9/dc/d13/d25/d2d to d0/d9/dc/d13/d25/d4e/da1 0 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.0.smithi110.stdout:6/437: dwrite d4/f3d [0,4194304] 0 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.0.smithi110.stdout:6/438: chown d4/d12/d17/d2e/d38 3521 1 2024-09-21T04:15:16.442 INFO:tasks.workunit.client.0.smithi110.stdout:6/439: write d4/d12/d16/f56 [702211,21330] 0 2024-09-21T04:15:16.446 INFO:tasks.workunit.client.0.smithi110.stdout:1/716: dwrite d2/d9/d23/f64 [0,4194304] 0 2024-09-21T04:15:16.446 INFO:tasks.workunit.client.0.smithi110.stdout:8/503: dwrite d0/d30/d54/f98 [0,4194304] 0 2024-09-21T04:15:16.448 INFO:tasks.workunit.client.1.smithi148.stdout:5/393: sync 2024-09-21T04:15:16.449 INFO:tasks.workunit.client.1.smithi148.stdout:0/419: creat d5/d6/d7/f83 x:0 0 0 2024-09-21T04:15:16.450 INFO:tasks.workunit.client.0.smithi110.stdout:9/372: getdents d7/d10 0 2024-09-21T04:15:16.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/627: mkdir d17/d3e/dda 0 2024-09-21T04:15:16.454 INFO:tasks.workunit.client.1.smithi148.stdout:9/403: symlink d5/d60/d4c/l9b 0 2024-09-21T04:15:16.455 INFO:tasks.workunit.client.1.smithi148.stdout:4/465: write d4/f6 [6110765,60763] 0 2024-09-21T04:15:16.456 INFO:tasks.workunit.client.0.smithi110.stdout:6/440: dread d4/d12/d16/d66/f67 [0,4194304] 0 2024-09-21T04:15:16.456 INFO:tasks.workunit.client.1.smithi148.stdout:7/428: read d5/d14/d25/d40/d7b/f84 [4019206,30741] 0 2024-09-21T04:15:16.457 INFO:tasks.workunit.client.1.smithi148.stdout:8/435: unlink dc/d13/d76/f5b 0 2024-09-21T04:15:16.460 INFO:tasks.workunit.client.1.smithi148.stdout:5/394: dread d3/d14/d1a/f4d [0,4194304] 0 2024-09-21T04:15:16.460 INFO:tasks.workunit.client.1.smithi148.stdout:1/397: mkdir db/d46/d4c/d82 0 2024-09-21T04:15:16.464 INFO:tasks.workunit.client.1.smithi148.stdout:1/398: dread db/d46/f53 [0,4194304] 0 2024-09-21T04:15:16.469 INFO:tasks.workunit.client.1.smithi148.stdout:6/399: mknod d0/d1a/c76 0 2024-09-21T04:15:16.469 INFO:tasks.workunit.client.1.smithi148.stdout:6/400: chown d0/d1a/d23/d16/f5d 19982 1 2024-09-21T04:15:16.469 INFO:tasks.workunit.client.1.smithi148.stdout:6/401: dread - d0/d1a/d23/d16/f52 zero size 2024-09-21T04:15:16.476 INFO:tasks.workunit.client.1.smithi148.stdout:0/420: mkdir d5/d6/d2a/d2b/d84 0 2024-09-21T04:15:16.477 INFO:tasks.workunit.client.0.smithi110.stdout:8/504: creat d0/d30/d54/fb0 x:0 0 0 2024-09-21T04:15:16.483 INFO:tasks.workunit.client.1.smithi148.stdout:9/404: mknod d5/d60/d4c/d5e/c9c 0 2024-09-21T04:15:16.487 INFO:tasks.workunit.client.1.smithi148.stdout:6/402: dwrite d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/403: chown d0/d1a/d23/c6 233262 1 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/404: write d0/d1a/d23/ff [46000,125003] 0 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/405: dread - d0/d1a/d23/dc/d68/d5b/f74 zero size 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/406: fdatasync d0/d1a/d23/f19 0 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/407: stat d0/d8/f24 0 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/408: truncate d0/d1a/d23/f3d 1994126 0 2024-09-21T04:15:16.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/409: fdatasync d0/f21 0 2024-09-21T04:15:16.492 INFO:tasks.workunit.client.1.smithi148.stdout:4/466: unlink d4/d66/d7f/d2a/f2f 0 2024-09-21T04:15:16.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/436: rmdir dc/d13/d76/d50 39 2024-09-21T04:15:16.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/437: chown dc/d13/d76/d5d 107365 1 2024-09-21T04:15:16.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/438: fdatasync dc/d13/d39/f3b 0 2024-09-21T04:15:16.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/439: write dc/d13/d76/d42/f4a [418391,53589] 0 2024-09-21T04:15:16.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/440: readlink dc/l4d 0 2024-09-21T04:15:16.494 INFO:tasks.workunit.client.0.smithi110.stdout:9/373: truncate d7/d10/d24/d3c/f20 2620970 0 2024-09-21T04:15:16.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/395: link d3/d14/d1a/f33 d3/d34/f8d 0 2024-09-21T04:15:16.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/396: write d3/d14/d1f/f7e [4312662,63431] 0 2024-09-21T04:15:16.500 INFO:tasks.workunit.client.0.smithi110.stdout:4/628: unlink d17/d34/d3d/d57/fd9 0 2024-09-21T04:15:16.502 INFO:tasks.workunit.client.0.smithi110.stdout:6/441: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d89 0 2024-09-21T04:15:16.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/441: dread f8 [0,4194304] 0 2024-09-21T04:15:16.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/442: fsync dc/f68 0 2024-09-21T04:15:16.503 INFO:tasks.workunit.client.1.smithi148.stdout:1/399: creat db/dd/f83 x:0 0 0 2024-09-21T04:15:16.504 INFO:tasks.workunit.client.0.smithi110.stdout:1/717: symlink d2/d9/d49/le9 0 2024-09-21T04:15:16.505 INFO:tasks.workunit.client.1.smithi148.stdout:2/534: rmdir d0/d9/d16/d55/d71 39 2024-09-21T04:15:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:8/505: rename d0/d7/d13/d46/f68 to d0/d7/d13/fb1 0 2024-09-21T04:15:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:8/506: dread - d0/d7/d23/f69 zero size 2024-09-21T04:15:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:8/507: chown d0/d7/f2a 96518499 1 2024-09-21T04:15:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:8/508: write d0/d30/d34/d3e/d5d/f62 [489878,75852] 0 2024-09-21T04:15:16.508 INFO:tasks.workunit.client.1.smithi148.stdout:0/421: creat d5/f85 x:0 0 0 2024-09-21T04:15:16.509 INFO:tasks.workunit.client.1.smithi148.stdout:0/422: creat d5/d6/d3e/f86 x:0 0 0 2024-09-21T04:15:16.510 INFO:tasks.workunit.client.0.smithi110.stdout:4/629: rmdir d17/d34/d3d/d20/d43/d51/d52/d98 39 2024-09-21T04:15:16.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/410: dwrite d0/d8/d18/f25 [0,4194304] 0 2024-09-21T04:15:16.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/405: symlink d5/d60/d4c/l9d 0 2024-09-21T04:15:16.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/406: write f0 [106463,125451] 0 2024-09-21T04:15:16.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/467: mknod d4/d66/c82 0 2024-09-21T04:15:16.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/718: creat d2/d9/d49/d7a/fea x:0 0 0 2024-09-21T04:15:16.520 INFO:tasks.workunit.client.0.smithi110.stdout:3/390: sync 2024-09-21T04:15:16.522 INFO:tasks.workunit.client.1.smithi148.stdout:7/429: rename d5/dc to d5/d8/d6f/d7d/d90 0 2024-09-21T04:15:16.522 INFO:tasks.workunit.client.1.smithi148.stdout:7/430: dread - d5/d14/d25/d40/f41 zero size 2024-09-21T04:15:16.522 INFO:tasks.workunit.client.1.smithi148.stdout:7/431: write d5/d8/d6f/d7d/d90/f10 [108365,80734] 0 2024-09-21T04:15:16.522 INFO:tasks.workunit.client.1.smithi148.stdout:7/432: read - d5/d8/d6f/d7d/d90/d28/f6d zero size 2024-09-21T04:15:16.526 INFO:tasks.workunit.client.0.smithi110.stdout:6/442: dread d4/de/f11 [0,4194304] 0 2024-09-21T04:15:16.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/374: rename d7/d10/d24/d3c/d1f/d43/l47 to d7/d10/d24/d3c/d14/d5c/d6f/d78/l7f 0 2024-09-21T04:15:16.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/375: chown d7/c6d 709080 1 2024-09-21T04:15:16.529 INFO:tasks.workunit.client.0.smithi110.stdout:8/509: link d0/d7/d13/d1c/fa0 d0/d7/d89/fb2 0 2024-09-21T04:15:16.529 INFO:tasks.workunit.client.0.smithi110.stdout:8/510: readlink d0/d30/l4d 0 2024-09-21T04:15:16.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/397: creat d3/d34/d3c/d5d/f8e x:0 0 0 2024-09-21T04:15:16.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/398: chown d3/d14/d1f/l65 7825551 1 2024-09-21T04:15:16.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/376: mknod d7/d10/d24/d3c/d11/c80 0 2024-09-21T04:15:16.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/377: truncate d7/d10/d24/d3c/d1f/d43/f6c 495081 0 2024-09-21T04:15:16.535 INFO:tasks.workunit.client.0.smithi110.stdout:8/511: write d0/d7/d13/d4b/f1f [402645,30221] 0 2024-09-21T04:15:16.535 INFO:tasks.workunit.client.0.smithi110.stdout:8/512: creat d0/d30/d54/fb3 x:0 0 0 2024-09-21T04:15:16.535 INFO:tasks.workunit.client.1.smithi148.stdout:9/407: read d5/d60/d28/d38/f47 [1536941,2272] 0 2024-09-21T04:15:16.536 INFO:tasks.workunit.client.0.smithi110.stdout:1/719: symlink d2/d9/d23/leb 0 2024-09-21T04:15:16.536 INFO:tasks.workunit.client.0.smithi110.stdout:1/720: dread - d2/d9/d49/d4f/d53/fd4 zero size 2024-09-21T04:15:16.536 INFO:tasks.workunit.client.0.smithi110.stdout:1/721: truncate d2/d9/d26/d3d/d8f/fbc 4401517 0 2024-09-21T04:15:16.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/423: dwrite d5/d6/d7/f43 [0,4194304] 0 2024-09-21T04:15:16.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/424: chown d5/d6/d2a/d2b/d84 206257909 1 2024-09-21T04:15:16.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/425: stat d5/d6/d7/d11/d48/f6b 0 2024-09-21T04:15:16.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/426: read d5/d6/d2a/d2b/f39 [18386,56979] 0 2024-09-21T04:15:16.538 INFO:tasks.workunit.client.0.smithi110.stdout:9/378: write d7/d10/d24/f38 [90096,6770] 0 2024-09-21T04:15:16.541 INFO:tasks.workunit.client.0.smithi110.stdout:8/513: mknod d0/d7/d23/cb4 0 2024-09-21T04:15:16.545 INFO:tasks.workunit.client.0.smithi110.stdout:8/514: rename d0/d7/d13/l2c to d0/d7/d6f/lb5 0 2024-09-21T04:15:16.546 INFO:tasks.workunit.client.0.smithi110.stdout:4/630: dwrite d17/d1e/d7b/f5e [0,4194304] 0 2024-09-21T04:15:16.546 INFO:tasks.workunit.client.0.smithi110.stdout:4/631: write d17/d34/d3d/fa6 [516767,38095] 0 2024-09-21T04:15:16.546 INFO:tasks.workunit.client.0.smithi110.stdout:4/632: write d17/d34/d3d/d20/d43/d51/d7a/d96/fd8 [78756,83861] 0 2024-09-21T04:15:16.546 INFO:tasks.workunit.client.0.smithi110.stdout:4/633: stat d17/d34/fb2 0 2024-09-21T04:15:16.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/411: creat d0/d34/d38/d43/d63/f77 x:0 0 0 2024-09-21T04:15:16.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/412: truncate d0/d8/d18/f60 620036 0 2024-09-21T04:15:16.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/413: readlink d0/d1a/d23/dc/l56 0 2024-09-21T04:15:16.550 INFO:tasks.workunit.client.0.smithi110.stdout:1/722: write d2/f45 [2898918,91139] 0 2024-09-21T04:15:16.550 INFO:tasks.workunit.client.1.smithi148.stdout:7/433: unlink d5/d14/d25/d79/f85 0 2024-09-21T04:15:16.552 INFO:tasks.workunit.client.1.smithi148.stdout:8/443: getdents dc/d43 0 2024-09-21T04:15:16.552 INFO:tasks.workunit.client.1.smithi148.stdout:8/444: chown dc/d13/d76/d5d/d78/c7f 86004 1 2024-09-21T04:15:16.552 INFO:tasks.workunit.client.1.smithi148.stdout:8/445: dread - dc/d13/d28/f2f zero size 2024-09-21T04:15:16.553 INFO:tasks.workunit.client.1.smithi148.stdout:5/399: creat d3/d14/d1f/d26/d2e/f8f x:0 0 0 2024-09-21T04:15:16.554 INFO:tasks.workunit.client.1.smithi148.stdout:2/535: mkdir d0/d9/da2 0 2024-09-21T04:15:16.556 INFO:tasks.workunit.client.1.smithi148.stdout:9/408: mkdir d5/d1f/d81/d9e 0 2024-09-21T04:15:16.557 INFO:tasks.workunit.client.0.smithi110.stdout:1/723: symlink d2/d9/d23/lec 0 2024-09-21T04:15:16.557 INFO:tasks.workunit.client.0.smithi110.stdout:1/724: write d2/d9/d23/d4e/f61 [2277692,88769] 0 2024-09-21T04:15:16.561 INFO:tasks.workunit.client.1.smithi148.stdout:7/434: dread f4 [0,4194304] 0 2024-09-21T04:15:16.562 INFO:tasks.workunit.client.0.smithi110.stdout:9/379: write d7/d10/f13 [5854600,100946] 0 2024-09-21T04:15:16.565 INFO:tasks.workunit.client.1.smithi148.stdout:8/446: creat dc/d13/d6a/f86 x:0 0 0 2024-09-21T04:15:16.565 INFO:tasks.workunit.client.0.smithi110.stdout:4/634: dread d17/d34/d3d/fa3 [0,4194304] 0 2024-09-21T04:15:16.566 INFO:tasks.workunit.client.0.smithi110.stdout:4/635: creat d17/d34/d3d/d20/d43/d51/d7a/d96/fdb x:0 0 0 2024-09-21T04:15:16.567 INFO:tasks.workunit.client.0.smithi110.stdout:9/380: link d7/d10/d24/f52 d7/d10/d24/d3c/d11/f81 0 2024-09-21T04:15:16.567 INFO:tasks.workunit.client.0.smithi110.stdout:9/381: readlink d7/d10/d24/d3c/d14/d5c/d6f/l70 0 2024-09-21T04:15:16.569 INFO:tasks.workunit.client.1.smithi148.stdout:5/400: truncate d3/d14/d1a/f57 847840 0 2024-09-21T04:15:16.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/401: fdatasync d3/fa 0 2024-09-21T04:15:16.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/409: creat d5/d2b/d32/f9f x:0 0 0 2024-09-21T04:15:16.573 INFO:tasks.workunit.client.1.smithi148.stdout:7/435: dread d5/d8/d6f/d7d/f68 [0,4194304] 0 2024-09-21T04:15:16.573 INFO:tasks.workunit.client.1.smithi148.stdout:7/436: creat d5/d8/f91 x:0 0 0 2024-09-21T04:15:16.574 INFO:tasks.workunit.client.0.smithi110.stdout:1/725: write d2/d9/d23/d4e/f7f [1215184,129398] 0 2024-09-21T04:15:16.574 INFO:tasks.workunit.client.1.smithi148.stdout:8/447: dread dc/d13/d28/f48 [0,4194304] 0 2024-09-21T04:15:16.575 INFO:tasks.workunit.client.0.smithi110.stdout:4/636: link d17/c81 d17/d3e/dbd/cdc 0 2024-09-21T04:15:16.577 INFO:tasks.workunit.client.1.smithi148.stdout:5/402: link d3/d24/l55 d3/d34/d78/l90 0 2024-09-21T04:15:16.578 INFO:tasks.workunit.client.0.smithi110.stdout:9/382: creat d7/d10/d24/d3c/d14/d5c/d6f/d78/f82 x:0 0 0 2024-09-21T04:15:16.578 INFO:tasks.workunit.client.0.smithi110.stdout:9/383: truncate d7/d10/d24/d3c/f5a 4548119 0 2024-09-21T04:15:16.579 INFO:tasks.workunit.client.0.smithi110.stdout:3/391: dwrite d0/d12/d16/d1a/f4e [0,4194304] 0 2024-09-21T04:15:16.580 INFO:tasks.workunit.client.1.smithi148.stdout:9/410: unlink d5/d1e/d37/d46/f6c 0 2024-09-21T04:15:16.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/726: dread d2/d9/d23/d3b/f55 [0,4194304] 0 2024-09-21T04:15:16.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/727: chown d2/d9/d49/d5c/cc0 60 1 2024-09-21T04:15:16.582 INFO:tasks.workunit.client.0.smithi110.stdout:4/637: creat d17/d1e/d7b/dc5/fdd x:0 0 0 2024-09-21T04:15:16.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/384: mknod d7/d10/d24/d3c/d1f/d72/c83 0 2024-09-21T04:15:16.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/385: chown d7/d10/d24/d3c 1305690 1 2024-09-21T04:15:16.587 INFO:tasks.workunit.client.1.smithi148.stdout:8/448: rmdir dc/d13/d76/d5d/d78 39 2024-09-21T04:15:16.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.589+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b94000bf0 con 0x7f4b9c03fb20 2024-09-21T04:15:16.594 INFO:tasks.workunit.client.0.smithi110.stdout:6/443: dwrite d4/d12/d16/f5b [4194304,4194304] 0 2024-09-21T04:15:16.594 INFO:tasks.workunit.client.0.smithi110.stdout:6/444: stat d4/d12/d17/d2e/l58 0 2024-09-21T04:15:16.594 INFO:tasks.workunit.client.0.smithi110.stdout:6/445: readlink d4/l1a 0 2024-09-21T04:15:16.594 INFO:tasks.workunit.client.0.smithi110.stdout:6/446: readlink d4/d12/d17/d33/l4a 0 2024-09-21T04:15:16.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/468: dwrite d4/d12/d16/d29/f5a [4194304,4194304] 0 2024-09-21T04:15:16.600 INFO:tasks.workunit.client.0.smithi110.stdout:3/392: truncate d0/d12/d16/d1a/d5c/f66 3577562 0 2024-09-21T04:15:16.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/392: sync 2024-09-21T04:15:16.602 INFO:tasks.workunit.client.0.smithi110.stdout:8/515: dwrite d0/d7/d13/fb1 [0,4194304] 0 2024-09-21T04:15:16.603 INFO:tasks.workunit.client.1.smithi148.stdout:9/411: truncate d5/d1e/d37/f3b 2977972 0 2024-09-21T04:15:16.605 INFO:tasks.workunit.client.0.smithi110.stdout:3/393: dread d0/d12/d3c/d5a/d70/f84 [0,4194304] 0 2024-09-21T04:15:16.607 INFO:tasks.workunit.client.0.smithi110.stdout:6/447: unlink d4/d12/d16/c31 0 2024-09-21T04:15:16.609 INFO:tasks.workunit.client.1.smithi148.stdout:4/469: mkdir d4/d12/d83 0 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.607+0000 7f4ba67fc640 1 -- 172.21.15.110:0/1670429014 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4b94000bf0 con 0x7f4b9c03fb20 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (6m) 2m ago 8m 25.3M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (8m) 2m ago 8m 7524k - 18.2.4-871-g6a1bdf85 0adab276c97e b0186c860d81 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (7m) 24s ago 7m 7415k - 18.2.4-871-g6a1bdf85 0adab276c97e 6e1e8fb80a91 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 2m ago 8m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (7m) 24s ago 7m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (6m) 2m ago 8m 84.8M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:15:16.610 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (2m) 2m ago 2m 16.3M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (2m) 2m ago 2m 11.8M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (2m) 24s ago 2m 14.4M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (2m) 24s ago 2m 16.0M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (9m) 2m ago 9m 486M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (27s) 24s ago 7m 59.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0d777a7727d1 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 2m ago 9m 47.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (7m) 24s ago 7m 36.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (8m) 2m ago 8m 16.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 24s ago 7m 15.4M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 2m ago 6m 57.7M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 2m ago 5m 54.6M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 2m ago 5m 53.1M 3946M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (4m) 24s ago 4m 121M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (4m) 24s ago 4m 114M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:15:16.611 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (4m) 24s ago 4m 112M 4970M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:15:16.612 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 2m ago 8m 59.4M - 2.43.0 a07b618ecd1d 2d686ff7a6ad 2024-09-21T04:15:16.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/393: mknod d3/d5/d6/d12/d43/d54/c79 0 2024-09-21T04:15:16.613 INFO:tasks.workunit.client.1.smithi148.stdout:7/437: dread d5/d8/d6f/d7d/d90/d28/f57 [0,4194304] 0 2024-09-21T04:15:16.614 INFO:tasks.workunit.client.0.smithi110.stdout:3/394: mknod d0/d12/d3c/d5a/c8f 0 2024-09-21T04:15:16.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/414: dwrite d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:16.614 INFO:tasks.workunit.client.1.smithi148.stdout:5/403: dwrite d3/f12 [0,4194304] 0 2024-09-21T04:15:16.615 INFO:tasks.workunit.client.1.smithi148.stdout:5/404: stat d3/d14/d1f/d26/d2e/c62 0 2024-09-21T04:15:16.615 INFO:tasks.workunit.client.1.smithi148.stdout:9/412: mknod d5/d60/d28/d38/ca0 0 2024-09-21T04:15:16.615 INFO:tasks.workunit.client.1.smithi148.stdout:9/413: chown d5/f96 136967 1 2024-09-21T04:15:16.616 INFO:tasks.workunit.client.1.smithi148.stdout:9/414: read - d5/d2b/f91 zero size 2024-09-21T04:15:16.616 INFO:tasks.workunit.client.1.smithi148.stdout:9/415: fsync d5/d1e/f2d 0 2024-09-21T04:15:16.616 INFO:tasks.workunit.client.1.smithi148.stdout:4/470: truncate d4/d12/f5b 2940363 0 2024-09-21T04:15:16.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.612+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 msgr2=0x7f4b9c041fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.612+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 0x7f4b9c041fe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005d00 tx=0x7f4ba0005c50 comp rx=0 tx=0).stop 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.612+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 msgr2=0x7f4bb010e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.612+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4bac006010 tx=0x7f4bac0055a0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 shutdown_connections 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f4b9c03fb20 0x7f4b9c041fe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bb0100a20 0x7f4bb010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 --2- 172.21.15.110:0/1670429014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb0100050 0x7f4bb010e6f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 >> 172.21.15.110:0/1670429014 conn(0x7f4bb00f99b0 msgr2=0x7f4bb01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 shutdown_connections 2024-09-21T04:15:16.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.613+0000 7f4bb7a08640 1 -- 172.21.15.110:0/1670429014 wait complete. 2024-09-21T04:15:16.618 INFO:tasks.workunit.client.1.smithi148.stdout:5/405: dread d3/d24/f43 [0,4194304] 0 2024-09-21T04:15:16.618 INFO:tasks.workunit.client.0.smithi110.stdout:3/395: unlink d0/d12/d2a/f53 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:7/438: symlink d5/d8/d6f/d7d/d8a/l92 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/427: dread d5/d6/d3e/f3a [0,4194304] 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/428: truncate d5/d6/d2a/d2b/f2c 4389946 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/429: write d5/d6/d2a/d2b/f5b [1873318,61207] 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/430: write d5/d6/f1f [1458459,1485] 0 2024-09-21T04:15:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/431: write d5/d6/f1f [845597,55455] 0 2024-09-21T04:15:16.621 INFO:tasks.workunit.client.0.smithi110.stdout:8/516: dread d0/f3d [0,4194304] 0 2024-09-21T04:15:16.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/415: link d0/d34/d38/d57/d58/f69 d0/d34/d38/d43/f78 0 2024-09-21T04:15:16.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/394: dread d3/d5/f35 [0,4194304] 0 2024-09-21T04:15:16.629 INFO:tasks.workunit.client.0.smithi110.stdout:1/728: dwrite d2/d9/d26/f2d [0,4194304] 0 2024-09-21T04:15:16.629 INFO:tasks.workunit.client.1.smithi148.stdout:1/400: dwrite db/dd/d21/f7a [0,4194304] 0 2024-09-21T04:15:16.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/401: chown db/d46/f48 56881786 1 2024-09-21T04:15:16.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/402: read - db/d24/d4f/f7d zero size 2024-09-21T04:15:16.632 INFO:tasks.workunit.client.0.smithi110.stdout:9/386: dwrite d7/d10/d24/d3c/d1f/d72/f63 [0,4194304] 0 2024-09-21T04:15:16.636 INFO:tasks.workunit.client.1.smithi148.stdout:4/471: symlink d4/d12/d16/d29/d61/d6c/l84 0 2024-09-21T04:15:16.636 INFO:tasks.workunit.client.1.smithi148.stdout:4/472: fsync d4/d3a/f53 0 2024-09-21T04:15:16.637 INFO:tasks.workunit.client.0.smithi110.stdout:4/638: dwrite d17/d1e/d21/f5b [0,4194304] 0 2024-09-21T04:15:16.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/439: rename d5/d14/d25/d40/d8c to d5/d14/d25/d40/d7b/d93 0 2024-09-21T04:15:16.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/440: dread - d5/d8/d6f/d7d/f7e zero size 2024-09-21T04:15:16.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/441: write d5/d14/d25/d40/d7b/f82 [265483,91407] 0 2024-09-21T04:15:16.640 INFO:tasks.workunit.client.1.smithi148.stdout:0/432: creat d5/d82/f87 x:0 0 0 2024-09-21T04:15:16.642 INFO:tasks.workunit.client.1.smithi148.stdout:6/416: mknod d0/d34/d38/d43/d63/c79 0 2024-09-21T04:15:16.645 INFO:tasks.workunit.client.1.smithi148.stdout:3/395: symlink d3/d5/d6/d12/l7a 0 2024-09-21T04:15:16.649 INFO:tasks.workunit.client.0.smithi110.stdout:6/448: dwrite d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:16.649 INFO:tasks.workunit.client.0.smithi110.stdout:8/517: link d0/d7/d13/d46/l80 d0/d30/d34/d3e/d60/lb6 0 2024-09-21T04:15:16.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/406: dwrite d3/d34/d3c/d5d/f8e [0,4194304] 0 2024-09-21T04:15:16.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/729: mkdir d2/db7/ded 0 2024-09-21T04:15:16.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/407: write d3/d37/f5f [991501,74954] 0 2024-09-21T04:15:16.652 INFO:tasks.workunit.client.1.smithi148.stdout:1/403: symlink db/d24/d2e/d59/l84 0 2024-09-21T04:15:16.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/404: chown db/d46/d4c/c54 0 1 2024-09-21T04:15:16.654 INFO:tasks.workunit.client.1.smithi148.stdout:1/405: dread db/f64 [0,4194304] 0 2024-09-21T04:15:16.654 INFO:tasks.workunit.client.1.smithi148.stdout:1/406: truncate db/d24/f33 710922 0 2024-09-21T04:15:16.655 INFO:tasks.workunit.client.1.smithi148.stdout:9/416: getdents d5/d60/d28/d38 0 2024-09-21T04:15:16.660 INFO:tasks.workunit.client.0.smithi110.stdout:9/387: rename d7/d10/d24/l49 to d7/d10/d24/l84 0 2024-09-21T04:15:16.661 INFO:tasks.workunit.client.1.smithi148.stdout:4/473: rename d4/d12/d16/d29/d61/d6d to d4/d12/d75/d85 0 2024-09-21T04:15:16.666 INFO:tasks.workunit.client.1.smithi148.stdout:7/442: mknod d5/d8/d6f/d7d/d8a/c94 0 2024-09-21T04:15:16.666 INFO:tasks.workunit.client.1.smithi148.stdout:7/443: truncate d5/d8/d6f/d7d/f7f 1260914 0 2024-09-21T04:15:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:16 smithi110 ceph-mon[27723]: pgmap v5: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:16 smithi110 ceph-mon[27723]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:16 smithi110 ceph-mon[27723]: mgrmap e24: smithi148.zjrahi(active, since 4s) 2024-09-21T04:15:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:16 smithi110 ceph-mon[27723]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:16.669 INFO:tasks.workunit.client.1.smithi148.stdout:0/433: creat d5/d6/f88 x:0 0 0 2024-09-21T04:15:16.669 INFO:tasks.workunit.client.1.smithi148.stdout:3/396: dwrite d3/d5/d6/d12/da/d23/d28/d65/f67 [0,4194304] 0 2024-09-21T04:15:16.669 INFO:tasks.workunit.client.1.smithi148.stdout:3/397: chown d3/d5/d6/d12/da 0 1 2024-09-21T04:15:16.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/474: dread d4/d66/d7f/d2a/f73 [0,4194304] 0 2024-09-21T04:15:16.677 INFO:tasks.workunit.client.1.smithi148.stdout:5/408: symlink d3/d34/d78/l91 0 2024-09-21T04:15:16.677 INFO:tasks.workunit.client.1.smithi148.stdout:5/409: truncate d3/f58 1630532 0 2024-09-21T04:15:16.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/417: dwrite d5/d1e/f49 [0,4194304] 0 2024-09-21T04:15:16.678 INFO:tasks.workunit.client.0.smithi110.stdout:8/518: mknod d0/d7/d23/cb7 0 2024-09-21T04:15:16.680 INFO:tasks.workunit.client.1.smithi148.stdout:1/407: symlink db/d46/d4c/l85 0 2024-09-21T04:15:16.680 INFO:tasks.workunit.client.1.smithi148.stdout:1/408: creat db/d46/d4c/f86 x:0 0 0 2024-09-21T04:15:16.680 INFO:tasks.workunit.client.1.smithi148.stdout:3/398: dread d3/d5/d6/d12/da/f3d [0,4194304] 0 2024-09-21T04:15:16.681 INFO:tasks.workunit.client.0.smithi110.stdout:1/730: rename d2/d88 to d2/d9/d49/d7a/dee 0 2024-09-21T04:15:16.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/444: link d5/d14/d25/d40/l5b d5/l95 0 2024-09-21T04:15:16.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/445: dread - d5/d8/d6f/d7d/d90/d28/f70 zero size 2024-09-21T04:15:16.684 INFO:tasks.workunit.client.0.smithi110.stdout:9/388: symlink d7/d10/d24/d3c/d14/d5c/d6f/l85 0 2024-09-21T04:15:16.684 INFO:tasks.workunit.client.0.smithi110.stdout:9/389: write d7/d10/d24/d3c/f20 [375786,37940] 0 2024-09-21T04:15:16.686 INFO:tasks.workunit.client.0.smithi110.stdout:6/449: getdents d4/d12/d17/d2e/d40 0 2024-09-21T04:15:16.687 INFO:tasks.workunit.client.1.smithi148.stdout:5/410: write f0 [3159228,41224] 0 2024-09-21T04:15:16.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/731: mknod d2/d9/d23/d3b/cef 0 2024-09-21T04:15:16.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/732: dread - d2/d9/d49/d4f/d87/fd1 zero size 2024-09-21T04:15:16.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/733: dread - d2/d9/d49/d7a/dee/fcf zero size 2024-09-21T04:15:16.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/734: write d2/d9/d23/d4e/f61 [2761525,4428] 0 2024-09-21T04:15:16.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/735: write d2/d9/d23/d3b/d9c/fa4 [587328,80946] 0 2024-09-21T04:15:16.692 INFO:tasks.workunit.client.0.smithi110.stdout:9/390: symlink d7/d10/d24/d3c/d4f/l86 0 2024-09-21T04:15:16.692 INFO:tasks.workunit.client.0.smithi110.stdout:9/391: read d7/d26/f5b [167062,7397] 0 2024-09-21T04:15:16.695 INFO:tasks.workunit.client.0.smithi110.stdout:1/736: dread d2/d9/d49/d7a/f7c [0,4194304] 0 2024-09-21T04:15:16.695 INFO:tasks.workunit.client.0.smithi110.stdout:1/737: chown d2/d9/d49/f99 723 1 2024-09-21T04:15:16.697 INFO:tasks.workunit.client.0.smithi110.stdout:6/450: dread d4/f6c [0,4194304] 0 2024-09-21T04:15:16.698 INFO:tasks.workunit.client.0.smithi110.stdout:1/738: dread d2/d9/d26/d3d/f58 [0,4194304] 0 2024-09-21T04:15:16.698 INFO:tasks.workunit.client.0.smithi110.stdout:9/392: unlink d7/d10/d24/f32 0 2024-09-21T04:15:16.699 INFO:tasks.workunit.client.1.smithi148.stdout:6/417: rmdir d0/d1a/d23 39 2024-09-21T04:15:16.699 INFO:tasks.workunit.client.1.smithi148.stdout:6/418: truncate d0/f21 9183720 0 2024-09-21T04:15:16.701 INFO:tasks.workunit.client.0.smithi110.stdout:6/451: symlink d4/d12/d17/d81/l8a 0 2024-09-21T04:15:16.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/251: sync 2024-09-21T04:15:16.704 INFO:tasks.workunit.client.0.smithi110.stdout:5/488: sync 2024-09-21T04:15:16.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/252: stat d2/d1a/d1f/d29 0 2024-09-21T04:15:16.704 INFO:tasks.workunit.client.0.smithi110.stdout:5/489: dread - d1/d49/d4d/d1e/d4a/f83 zero size 2024-09-21T04:15:16.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/739: unlink d2/d9/d23/f38 0 2024-09-21T04:15:16.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/740: chown d2/d9/d26/l6d 21358 1 2024-09-21T04:15:16.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/741: chown d2/d9/d49/d5c/d91/fa5 291883 1 2024-09-21T04:15:16.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/742: fdatasync d2/d9/d23/d4e/f7f 0 2024-09-21T04:15:16.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/452: symlink d4/d12/d17/d2e/d38/l8b 0 2024-09-21T04:15:16.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/453: creat d4/f8c x:0 0 0 2024-09-21T04:15:16.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/454: read - d4/d12/d17/d2e/f47 zero size 2024-09-21T04:15:16.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/455: stat d4/d12/d17/d33/f64 0 2024-09-21T04:15:16.708 INFO:tasks.workunit.client.0.smithi110.stdout:7/415: sync 2024-09-21T04:15:16.708 INFO:tasks.workunit.client.1.smithi148.stdout:9/418: unlink d5/d60/d28/f75 0 2024-09-21T04:15:16.709 INFO:tasks.workunit.client.0.smithi110.stdout:5/490: dread d1/dc/f1a [0,4194304] 0 2024-09-21T04:15:16.710 INFO:tasks.workunit.client.0.smithi110.stdout:5/491: chown d1/c2 264103886 1 2024-09-21T04:15:16.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/456: creat d4/d12/d16/d66/d7a/d4b/f8d x:0 0 0 2024-09-21T04:15:16.715 INFO:tasks.workunit.client.0.smithi110.stdout:7/416: dread dd/d1c/d39/f5a [0,4194304] 0 2024-09-21T04:15:16.715 INFO:tasks.workunit.client.0.smithi110.stdout:7/417: write dd/d4d/f5e [859747,45209] 0 2024-09-21T04:15:16.717 INFO:tasks.workunit.client.0.smithi110.stdout:6/457: creat d4/d12/d17/f8e x:0 0 0 2024-09-21T04:15:16.717 INFO:tasks.workunit.client.0.smithi110.stdout:6/458: dread - d4/d12/d17/f8e zero size 2024-09-21T04:15:16.728 INFO:tasks.workunit.client.1.smithi148.stdout:3/399: creat d3/d5/f7b x:0 0 0 2024-09-21T04:15:16.728 INFO:tasks.workunit.client.1.smithi148.stdout:7/446: creat d5/d8/d64/f96 x:0 0 0 2024-09-21T04:15:16.731 INFO:tasks.workunit.client.0.smithi110.stdout:9/393: dwrite d7/d10/d24/d3c/d1f/d43/f6c [0,4194304] 0 2024-09-21T04:15:16.733 INFO:tasks.workunit.client.1.smithi148.stdout:0/434: rmdir d5 39 2024-09-21T04:15:16.737 INFO:tasks.workunit.client.1.smithi148.stdout:5/411: mknod d3/d14/d1f/c92 0 2024-09-21T04:15:16.738 INFO:tasks.workunit.client.1.smithi148.stdout:6/419: unlink d0/d1a/d23/d16/f2c 0 2024-09-21T04:15:16.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/394: truncate d7/d10/d24/f38 116369 0 2024-09-21T04:15:16.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/395: fsync f2 0 2024-09-21T04:15:16.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/396: truncate d7/d10/d24/d3c/d14/d31/f6a 751675 0 2024-09-21T04:15:16.742 INFO:tasks.workunit.client.1.smithi148.stdout:3/400: rmdir d3/d5/d6/d12/d43/d6e 0 2024-09-21T04:15:16.744 INFO:tasks.workunit.client.0.smithi110.stdout:9/397: symlink d7/d10/d24/d3c/d1f/d72/l87 0 2024-09-21T04:15:16.745 INFO:tasks.workunit.client.1.smithi148.stdout:5/412: link d3/d14/d1a/f57 d3/d34/d3c/d5d/f93 0 2024-09-21T04:15:16.745 INFO:tasks.workunit.client.0.smithi110.stdout:2/253: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:15:16.746 INFO:tasks.workunit.client.0.smithi110.stdout:2/254: write d2/d1a/d1f/d3f/f2b [1001826,87229] 0 2024-09-21T04:15:16.746 INFO:tasks.workunit.client.0.smithi110.stdout:2/255: readlink d2/l23 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.0.smithi110.stdout:2/256: chown d2/d1a/d1f 921736797 1 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.0.smithi110.stdout:2/257: fdatasync d2/f44 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.0.smithi110.stdout:2/258: fsync d2/d1a/d1f/d3f/f43 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.1.smithi148.stdout:0/435: dread d5/d6/d2a/d2b/f35 [0,4194304] 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.1.smithi148.stdout:6/420: creat d0/d1a/d23/d16/d36/f7a x:0 0 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.1.smithi148.stdout:6/421: readlink d0/d34/l6f 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.1.smithi148.stdout:6/422: write d0/d34/d38/d57/d58/f69 [1007206,16560] 0 2024-09-21T04:15:16.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/419: getdents d5/d2b/d4e 0 2024-09-21T04:15:16.748 INFO:tasks.workunit.client.1.smithi148.stdout:3/401: creat d3/d5/d6/d12/d15/d73/f7c x:0 0 0 2024-09-21T04:15:16.750 INFO:tasks.workunit.client.0.smithi110.stdout:2/259: rename d2/f35 to d2/d1a/d1f/d29/f45 0 2024-09-21T04:15:16.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/413: rename d3/d14/d1f/d26/d2e/d56/f19 to d3/d14/d1f/d26/f94 0 2024-09-21T04:15:16.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/414: fdatasync d3/d24/f76 0 2024-09-21T04:15:16.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/415: chown d3/l6d 249248343 1 2024-09-21T04:15:16.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/436: creat d5/d6/d2a/d2b/d84/f89 x:0 0 0 2024-09-21T04:15:16.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/437: fsync f4 0 2024-09-21T04:15:16.756 INFO:tasks.workunit.client.0.smithi110.stdout:2/260: creat d2/d1a/d1f/d3f/f46 x:0 0 0 2024-09-21T04:15:16.757 INFO:tasks.workunit.client.0.smithi110.stdout:2/261: fdatasync d2/f44 0 2024-09-21T04:15:16.759 INFO:tasks.workunit.client.1.smithi148.stdout:0/438: creat d5/d6/d7/d22/f8a x:0 0 0 2024-09-21T04:15:16.759 INFO:tasks.workunit.client.1.smithi148.stdout:0/439: fdatasync d5/d6/d7/d11/d48/f7d 0 2024-09-21T04:15:16.759 INFO:tasks.workunit.client.1.smithi148.stdout:7/447: dread d5/d8/f16 [4194304,4194304] 0 2024-09-21T04:15:16.759 INFO:tasks.workunit.client.1.smithi148.stdout:7/448: chown d5 389648 1 2024-09-21T04:15:16.762 INFO:tasks.workunit.client.0.smithi110.stdout:6/459: dread d4/de/f44 [0,4194304] 0 2024-09-21T04:15:16.762 INFO:tasks.workunit.client.1.smithi148.stdout:6/423: write d0/d1a/d23/dc/d68/f66 [3149729,49698] 0 2024-09-21T04:15:16.762 INFO:tasks.workunit.client.1.smithi148.stdout:6/424: chown d0/l35 30861 1 2024-09-21T04:15:16.762 INFO:tasks.workunit.client.1.smithi148.stdout:6/425: fsync d0/d1a/d23/d53/f5f 0 2024-09-21T04:15:16.763 INFO:tasks.workunit.client.1.smithi148.stdout:0/440: mkdir d5/d6/d2a/d3f/d8b 0 2024-09-21T04:15:16.764 INFO:tasks.workunit.client.1.smithi148.stdout:9/420: dread d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:16.765 INFO:tasks.workunit.client.1.smithi148.stdout:7/449: symlink d5/d14/d25/d40/d7b/l97 0 2024-09-21T04:15:16.766 INFO:tasks.workunit.client.0.smithi110.stdout:5/492: dwrite d1/d39/f88 [4194304,4194304] 0 2024-09-21T04:15:16.767 INFO:tasks.workunit.client.0.smithi110.stdout:8/519: dwrite d0/d7/d13/d1c/f50 [0,4194304] 0 2024-09-21T04:15:16.767 INFO:tasks.workunit.client.1.smithi148.stdout:1/409: dwrite db/d46/f47 [0,4194304] 0 2024-09-21T04:15:16.767 INFO:tasks.workunit.client.1.smithi148.stdout:1/410: dread - db/d24/d4f/f72 zero size 2024-09-21T04:15:16.767 INFO:tasks.workunit.client.1.smithi148.stdout:1/411: write db/d24/d4f/f7d [715646,43541] 0 2024-09-21T04:15:16.767 INFO:tasks.workunit.client.1.smithi148.stdout:1/412: fdatasync db/f1b 0 2024-09-21T04:15:16.768 INFO:tasks.workunit.client.0.smithi110.stdout:6/460: mkdir d4/d86/d8f 0 2024-09-21T04:15:16.768 INFO:tasks.workunit.client.1.smithi148.stdout:5/416: dread d3/d14/d1f/d26/f94 [0,4194304] 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:1/743: dwrite d2/d9/d49/fbe [0,4194304] 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:7/418: dwrite dd/d1c/f75 [0,4194304] 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:9/398: dread d7/d10/d24/d3c/d1f/d43/f60 [0,4194304] 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:1/744: write d2/f2e [915358,36363] 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:7/419: creat dd/d1c/d39/d4a/f7f x:0 0 0 2024-09-21T04:15:16.769 INFO:tasks.workunit.client.0.smithi110.stdout:7/420: write dd/d1c/d39/f3c [2035599,32324] 0 2024-09-21T04:15:16.770 INFO:tasks.workunit.client.1.smithi148.stdout:6/426: creat d0/d34/d38/d43/f7b x:0 0 0 2024-09-21T04:15:16.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/493: creat d1/d49/d6f/fa8 x:0 0 0 2024-09-21T04:15:16.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/494: chown d1/d49/d4d/d1e/l9a 6 1 2024-09-21T04:15:16.774 INFO:tasks.workunit.client.1.smithi148.stdout:0/441: symlink d5/d6/d2a/d3f/d6d/l8c 0 2024-09-21T04:15:16.774 INFO:tasks.workunit.client.1.smithi148.stdout:0/442: chown d5/d6/d7/d11/d48 7114 1 2024-09-21T04:15:16.774 INFO:tasks.workunit.client.1.smithi148.stdout:0/443: read d5/d6/d3e/f1b [817118,18856] 0 2024-09-21T04:15:16.776 INFO:tasks.workunit.client.1.smithi148.stdout:9/421: creat d5/d2b/d32/d67/d80/fa1 x:0 0 0 2024-09-21T04:15:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/422: read - d5/d39/f77 zero size 2024-09-21T04:15:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/423: chown d5/d60/d28/l89 254642 1 2024-09-21T04:15:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/424: dread - d5/d39/f98 zero size 2024-09-21T04:15:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/425: fdatasync d5/d39/f6f 0 2024-09-21T04:15:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/426: read d5/d2b/f4f [2172573,66323] 0 2024-09-21T04:15:16.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/427: stat d5/d60/d69 0 2024-09-21T04:15:16.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/428: chown d5/c76 266908 1 2024-09-21T04:15:16.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/429: truncate d5/d1e/d37/f8d 125803 0 2024-09-21T04:15:16.780 INFO:tasks.workunit.client.1.smithi148.stdout:0/444: read d5/d6/d7/f4b [4066818,6473] 0 2024-09-21T04:15:16.782 INFO:tasks.workunit.client.0.smithi110.stdout:9/399: dread d7/d10/d24/d3c/d1f/d72/f63 [0,4194304] 0 2024-09-21T04:15:16.782 INFO:tasks.workunit.client.0.smithi110.stdout:8/520: dread d0/d30/d54/d6b/f75 [0,4194304] 0 2024-09-21T04:15:16.782 INFO:tasks.workunit.client.0.smithi110.stdout:6/461: mkdir d4/d12/d17/d81/d90 0 2024-09-21T04:15:16.782 INFO:tasks.workunit.client.0.smithi110.stdout:6/462: fsync d4/f3d 0 2024-09-21T04:15:16.783 INFO:tasks.workunit.client.0.smithi110.stdout:1/745: dread d2/d9/d23/d4e/fad [0,4194304] 0 2024-09-21T04:15:16.783 INFO:tasks.workunit.client.0.smithi110.stdout:1/746: read - d2/db7/d86/fb9 zero size 2024-09-21T04:15:16.783 INFO:tasks.workunit.client.0.smithi110.stdout:1/747: chown d2/d9/d26/d3d/fd9 14 1 2024-09-21T04:15:16.783 INFO:tasks.workunit.client.0.smithi110.stdout:1/748: write d2/d9/f1f [82082,52591] 0 2024-09-21T04:15:16.785 INFO:tasks.workunit.client.1.smithi148.stdout:7/450: rename d5/d8/d6f/d7d/d90/d28/f53 to d5/d8/d6f/d7d/f98 0 2024-09-21T04:15:16.785 INFO:tasks.workunit.client.1.smithi148.stdout:1/413: link db/l49 db/d46/d4c/l87 0 2024-09-21T04:15:16.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.783+0000 7fefc6bf5640 1 -- 172.21.15.110:0/1340257881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 msgr2=0x7fefc0071c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.783+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/1340257881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc0071c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fefb8005fd0 tx=0x7fefb802b510 comp rx=0 tx=0).stop 2024-09-21T04:15:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.784+0000 7fefc6bf5640 1 -- 172.21.15.110:0/1340257881 shutdown_connections 2024-09-21T04:15:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.784+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/1340257881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc0107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.784+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/1340257881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc0071c90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.784+0000 7fefc6bf5640 1 -- 172.21.15.110:0/1340257881 >> 172.21.15.110:0/1340257881 conn(0x7fefc006cf90 msgr2=0x7fefc006f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.786+0000 7fefc6bf5640 1 -- 172.21.15.110:0/1340257881 shutdown_connections 2024-09-21T04:15:16.789 INFO:tasks.workunit.client.1.smithi148.stdout:3/402: dwrite d3/d5/d6/d12/da/d23/d28/d46/f5b [0,4194304] 0 2024-09-21T04:15:16.790 INFO:tasks.workunit.client.1.smithi148.stdout:3/403: readlink d3/d5/d6/d2b/l56 0 2024-09-21T04:15:16.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.787+0000 7fefc6bf5640 1 -- 172.21.15.110:0/1340257881 wait complete. 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 Processor -- start 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 -- start start 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc01a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01a4eb0 con 0x7fefc0071890 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.788+0000 7fefc6bf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01a5020 con 0x7fefc01077d0 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.789+0000 7fefbffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.789+0000 7fefbffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:58524/0 (socket says 172.21.15.110:58524) 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.789+0000 7fefbffff640 1 -- 172.21.15.110:0/3767295695 learned_addr learned my addr 172.21.15.110:0/3767295695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.789+0000 7fefc4991640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc01a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.789+0000 7fefbffff640 1 -- 172.21.15.110:0/3767295695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 msgr2=0x7fefc01a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:16.793 INFO:tasks.workunit.client.0.smithi110.stdout:2/262: dwrite d2/d1a/d1f/d3f/f1d [0,4194304] 0 2024-09-21T04:15:16.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.790+0000 7fefbffff640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc01a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:16.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.790+0000 7fefbffff640 1 -- 172.21.15.110:0/3767295695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8007490 con 0x7fefc01077d0 2024-09-21T04:15:16.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.790+0000 7fefc4991640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc01a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:15:16.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.790+0000 7fefbffff640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d6f0 tx=0x7fefb000dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:16.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.791+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb0009a30 con 0x7fefc01077d0 2024-09-21T04:15:16.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.791+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefb0009b90 con 0x7fefc01077d0 2024-09-21T04:15:16.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.792+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb0003ea0 con 0x7fefc01077d0 2024-09-21T04:15:16.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.792+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc01a9ac0 con 0x7fefc01077d0 2024-09-21T04:15:16.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.793+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc01aa090 con 0x7fefc01077d0 2024-09-21T04:15:16.796 INFO:tasks.workunit.client.0.smithi110.stdout:7/421: write dd/d1c/d39/d4a/d52/f60 [8851317,127616] 0 2024-09-21T04:15:16.796 INFO:tasks.workunit.client.0.smithi110.stdout:7/422: read dd/d1c/d39/f3c [2697558,2286] 0 2024-09-21T04:15:16.796 INFO:tasks.workunit.client.0.smithi110.stdout:7/423: dread - dd/d1c/d39/d4a/f7f zero size 2024-09-21T04:15:16.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/427: fdatasync d0/d1a/d23/dc/d68/f66 0 2024-09-21T04:15:16.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/428: read - d0/d1a/d23/d16/d36/f7a zero size 2024-09-21T04:15:16.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/429: chown d0/d1a/d23/d16/f3f 81965781 1 2024-09-21T04:15:16.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/430: chown d0/l35 11530 1 2024-09-21T04:15:16.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.794+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 24) v1 ==== 51935+0+0 (secure 0 0 0) 0x7fefb0010460 con 0x7fefc01077d0 2024-09-21T04:15:16.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.794+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc010a100 con 0x7fefc01077d0 2024-09-21T04:15:16.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.795+0000 7fefbdffb640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 0x7fefa8042010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:16.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.795+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fefb0054fd0 con 0x7fefc01077d0 2024-09-21T04:15:16.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.795+0000 7fefc4991640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 0x7fefa8042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:16.799 INFO:tasks.workunit.client.1.smithi148.stdout:9/430: dwrite d5/ff [4194304,4194304] 0 2024-09-21T04:15:16.799 INFO:tasks.workunit.client.1.smithi148.stdout:9/431: write d5/d2b/f91 [930955,37871] 0 2024-09-21T04:15:16.799 INFO:tasks.workunit.client.1.smithi148.stdout:9/432: chown d5/d39/l61 1894803874 1 2024-09-21T04:15:16.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.797+0000 7fefc4991640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 0x7fefa8042010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefb802ba30 tx=0x7fefb8000f40 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:16.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/417: dwrite f0 [0,4194304] 0 2024-09-21T04:15:16.801 INFO:tasks.workunit.client.0.smithi110.stdout:9/400: mkdir d7/d10/d88 0 2024-09-21T04:15:16.802 INFO:tasks.workunit.client.1.smithi148.stdout:0/445: creat d5/d6/d2a/d3f/f8d x:0 0 0 2024-09-21T04:15:16.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:16.801+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fefb001c690 con 0x7fefc01077d0 2024-09-21T04:15:16.804 INFO:tasks.workunit.client.0.smithi110.stdout:6/463: symlink d4/d12/d17/d33/d76/l91 0 2024-09-21T04:15:16.806 INFO:tasks.workunit.client.1.smithi148.stdout:7/451: creat d5/d14/d25/d40/d8b/f99 x:0 0 0 2024-09-21T04:15:16.807 INFO:tasks.workunit.client.0.smithi110.stdout:8/521: dwrite d0/d30/f58 [0,4194304] 0 2024-09-21T04:15:16.807 INFO:tasks.workunit.client.0.smithi110.stdout:8/522: write d0/d30/d34/d3e/d32/da3/fa9 [353802,90694] 0 2024-09-21T04:15:16.808 INFO:tasks.workunit.client.0.smithi110.stdout:1/749: mkdir d2/d9/d49/d4f/df0 0 2024-09-21T04:15:16.808 INFO:tasks.workunit.client.0.smithi110.stdout:1/750: fsync d2/d9/d26/fce 0 2024-09-21T04:15:16.812 INFO:tasks.workunit.client.0.smithi110.stdout:2/263: symlink d2/l47 0 2024-09-21T04:15:16.814 INFO:tasks.workunit.client.1.smithi148.stdout:3/404: mknod d3/d5/d6/d12/da/d23/c7d 0 2024-09-21T04:15:16.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/446: dread d5/d6/f15 [0,4194304] 0 2024-09-21T04:15:16.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/447: fdatasync d5/d6/d2a/d2b/f2c 0 2024-09-21T04:15:16.816 INFO:tasks.workunit.client.1.smithi148.stdout:6/431: truncate d0/d1a/d23/d16/f3f 3847202 0 2024-09-21T04:15:16.816 INFO:tasks.workunit.client.1.smithi148.stdout:6/432: chown d0/d34/d38/d57 7 1 2024-09-21T04:15:16.820 INFO:tasks.workunit.client.0.smithi110.stdout:7/424: mkdir dd/d1c/d39/d6d/d80 0 2024-09-21T04:15:16.820 INFO:tasks.workunit.client.0.smithi110.stdout:2/264: dread d2/d1a/d1f/f38 [0,4194304] 0 2024-09-21T04:15:16.820 INFO:tasks.workunit.client.1.smithi148.stdout:9/433: mkdir d5/d1f/d81/d9e/da2 0 2024-09-21T04:15:16.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/405: dread d3/d5/d6/d12/da/d52/f4a [0,4194304] 0 2024-09-21T04:15:16.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/406: readlink d3/d5/d6/d12/da/l1b 0 2024-09-21T04:15:16.824 INFO:tasks.workunit.client.0.smithi110.stdout:6/464: dwrite d4/d12/f2b [0,4194304] 0 2024-09-21T04:15:16.824 INFO:tasks.workunit.client.1.smithi148.stdout:5/418: truncate d3/d14/d1f/f50 2110525 0 2024-09-21T04:15:16.825 INFO:tasks.workunit.client.0.smithi110.stdout:5/495: getdents d1/d7/d58 0 2024-09-21T04:15:16.826 INFO:tasks.workunit.client.1.smithi148.stdout:0/448: dread d5/d6/d7/f4f [0,4194304] 0 2024-09-21T04:15:16.826 INFO:tasks.workunit.client.1.smithi148.stdout:0/449: truncate d5/d6/d7/f43 4305092 0 2024-09-21T04:15:16.829 INFO:tasks.workunit.client.1.smithi148.stdout:7/452: getdents d5/d14/d25 0 2024-09-21T04:15:16.830 INFO:tasks.workunit.client.1.smithi148.stdout:7/453: write d5/d8/d6f/d7d/d90/f6e [392874,89415] 0 2024-09-21T04:15:16.832 INFO:tasks.workunit.client.1.smithi148.stdout:1/414: getdents db/d24/d2e 0 2024-09-21T04:15:16.838 INFO:tasks.workunit.client.1.smithi148.stdout:6/433: truncate d0/d1a/d23/ff 474753 0 2024-09-21T04:15:16.838 INFO:tasks.workunit.client.1.smithi148.stdout:6/434: chown d0/d34/l6f 331 1 2024-09-21T04:15:16.838 INFO:tasks.workunit.client.1.smithi148.stdout:6/435: chown d0/d8/d18/c6d 1 1 2024-09-21T04:15:16.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/434: creat d5/d2b/d4e/fa3 x:0 0 0 2024-09-21T04:15:16.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/435: creat d5/d2b/d32/fa4 x:0 0 0 2024-09-21T04:15:16.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/436: truncate d5/d2b/d4e/f97 323608 0 2024-09-21T04:15:16.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/437: read d5/d60/d28/d38/f52 [3485318,36091] 0 2024-09-21T04:15:16.845 INFO:tasks.workunit.client.1.smithi148.stdout:0/450: dwrite d5/d6/d2a/d2b/f2c [0,4194304] 0 2024-09-21T04:15:16.845 INFO:tasks.workunit.client.1.smithi148.stdout:0/451: chown d5/f85 6315583 1 2024-09-21T04:15:16.845 INFO:tasks.workunit.client.1.smithi148.stdout:0/452: chown d5/d6/d3e/f86 25 1 2024-09-21T04:15:16.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/407: creat d3/d5/d6/d12/d76/f7e x:0 0 0 2024-09-21T04:15:16.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/408: readlink d3/d5/l69 0 2024-09-21T04:15:16.849 INFO:tasks.workunit.client.1.smithi148.stdout:5/419: rename d3/d14/d41/f49 to d3/d34/d3c/d5d/f95 0 2024-09-21T04:15:16.849 INFO:tasks.workunit.client.1.smithi148.stdout:5/420: write d3/f58 [2636310,15399] 0 2024-09-21T04:15:16.849 INFO:tasks.workunit.client.1.smithi148.stdout:1/415: dwrite db/d24/d2e/f63 [0,4194304] 0 2024-09-21T04:15:16.849 INFO:tasks.workunit.client.1.smithi148.stdout:5/421: creat d3/d14/d41/f96 x:0 0 0 2024-09-21T04:15:16.852 INFO:tasks.workunit.client.0.smithi110.stdout:9/401: unlink d7/d10/d24/d3c/d1f/d43/c4c 0 2024-09-21T04:15:16.857 INFO:tasks.workunit.client.1.smithi148.stdout:7/454: creat d5/d14/d25/d40/d7b/f9a x:0 0 0 2024-09-21T04:15:16.857 INFO:tasks.workunit.client.1.smithi148.stdout:7/455: dread - d5/d14/f78 zero size 2024-09-21T04:15:16.857 INFO:tasks.workunit.client.1.smithi148.stdout:7/456: readlink d5/l1f 0 2024-09-21T04:15:16.859 INFO:tasks.workunit.client.1.smithi148.stdout:6/436: truncate d0/d1a/d23/d16/f1f 1682748 0 2024-09-21T04:15:16.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/437: creat d0/d34/d38/d43/d63/f7c x:0 0 0 2024-09-21T04:15:16.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/438: fsync d0/d8/d18/f4d 0 2024-09-21T04:15:16.861 INFO:tasks.workunit.client.0.smithi110.stdout:8/523: mknod d0/d7/d89/cb8 0 2024-09-21T04:15:16.861 INFO:tasks.workunit.client.0.smithi110.stdout:8/524: chown d0/d30/d34/d3e/d5d/c6d 106 1 2024-09-21T04:15:16.863 INFO:tasks.workunit.client.0.smithi110.stdout:8/525: dread d0/d7/d23/d82/f83 [0,4194304] 0 2024-09-21T04:15:16.863 INFO:tasks.workunit.client.0.smithi110.stdout:1/751: mknod d2/d9/d23/d4e/cf1 0 2024-09-21T04:15:16.863 INFO:tasks.workunit.client.0.smithi110.stdout:1/752: write d2/d9/d49/d7a/fea [293814,36373] 0 2024-09-21T04:15:16.864 INFO:tasks.workunit.client.1.smithi148.stdout:9/438: creat d5/fa5 x:0 0 0 2024-09-21T04:15:16.865 INFO:tasks.workunit.client.0.smithi110.stdout:7/425: mknod dd/d1c/d39/d6d/c81 0 2024-09-21T04:15:16.865 INFO:tasks.workunit.client.0.smithi110.stdout:1/753: dread d2/db7/f5f [0,4194304] 0 2024-09-21T04:15:16.866 INFO:tasks.workunit.client.1.smithi148.stdout:0/453: rename d5/d6/d7/d11/c5c to d5/d6/d7/d11/c8e 0 2024-09-21T04:15:16.866 INFO:tasks.workunit.client.1.smithi148.stdout:0/454: fsync d5/d6/f1f 0 2024-09-21T04:15:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:2/265: rename d2/d1a/d1f/d29/fd to d2/d1a/d1f/d29/f48 0 2024-09-21T04:15:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:2/266: fsync d2/d1a/d1f/d3f/f1d 0 2024-09-21T04:15:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:2/267: write d2/f3b [868124,74437] 0 2024-09-21T04:15:16.868 INFO:tasks.workunit.client.1.smithi148.stdout:3/409: symlink d3/d5/d6/d12/da/l7f 0 2024-09-21T04:15:16.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/465: mknod d4/d12/c92 0 2024-09-21T04:15:16.872 INFO:tasks.workunit.client.0.smithi110.stdout:5/496: mkdir d1/d7/d18/da9 0 2024-09-21T04:15:16.873 INFO:tasks.workunit.client.0.smithi110.stdout:5/497: fdatasync d1/d22/d51/f70 0 2024-09-21T04:15:16.874 INFO:tasks.workunit.client.1.smithi148.stdout:0/455: dread d5/d6/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:16.875 INFO:tasks.workunit.client.0.smithi110.stdout:9/402: mknod d7/d10/d24/d3c/d14/d5c/d6f/d78/d7d/c89 0 2024-09-21T04:15:16.879 INFO:tasks.workunit.client.0.smithi110.stdout:8/526: creat d0/d30/d34/d3e/d32/fb9 x:0 0 0 2024-09-21T04:15:16.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/498: dread d1/d49/d4d/d1e/d5a/d64/f73 [0,4194304] 0 2024-09-21T04:15:16.883 INFO:tasks.workunit.client.0.smithi110.stdout:7/426: mkdir dd/d48/d55/d7a/d82 0 2024-09-21T04:15:16.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/754: creat d2/d9/d23/d3b/d9c/de6/ff2 x:0 0 0 2024-09-21T04:15:16.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/755: write d2/d9/d23/f75 [237347,23398] 0 2024-09-21T04:15:16.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/756: chown d2/f2e 8820555 1 2024-09-21T04:15:16.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/757: readlink d2/d9/d26/ld7 0 2024-09-21T04:15:16.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/466: creat d4/d12/d16/d66/f93 x:0 0 0 2024-09-21T04:15:16.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/467: truncate d4/f6a 70571 0 2024-09-21T04:15:16.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/468: write d4/d12/d16/d66/d7a/d4b/f8d [386301,2451] 0 2024-09-21T04:15:16.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/469: chown d4/d12/d16/d66/d7a 0 1 2024-09-21T04:15:16.888 INFO:tasks.workunit.client.0.smithi110.stdout:6/470: fsync d4/f79 0 2024-09-21T04:15:16.890 INFO:tasks.workunit.client.0.smithi110.stdout:9/403: creat d7/d10/d24/d3c/d14/d5c/d6f/d78/d7d/f8a x:0 0 0 2024-09-21T04:15:16.891 INFO:tasks.workunit.client.0.smithi110.stdout:9/404: dread d7/d26/f5b [0,4194304] 0 2024-09-21T04:15:16.893 INFO:tasks.workunit.client.0.smithi110.stdout:5/499: mknod d1/d7/d18/d3f/caa 0 2024-09-21T04:15:16.893 INFO:tasks.workunit.client.0.smithi110.stdout:5/500: chown d1/d49/l52 6901 1 2024-09-21T04:15:16.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/422: unlink d3/d24/c4c 0 2024-09-21T04:15:16.897 INFO:tasks.workunit.client.0.smithi110.stdout:7/427: symlink dd/d48/d55/l83 0 2024-09-21T04:15:16.897 INFO:tasks.workunit.client.0.smithi110.stdout:7/428: chown dd/d14/d17/f3a 1 1 2024-09-21T04:15:16.899 INFO:tasks.workunit.client.1.smithi148.stdout:7/457: mkdir d5/d8/d6f/d7d/d90/d9b 0 2024-09-21T04:15:16.902 INFO:tasks.workunit.client.0.smithi110.stdout:1/758: mkdir d2/df3 0 2024-09-21T04:15:16.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/439: mknod d5/d2b/d4e/d95/ca6 0 2024-09-21T04:15:16.903 INFO:tasks.workunit.client.0.smithi110.stdout:6/471: mknod d4/d12/d17/d2e/c94 0 2024-09-21T04:15:16.905 INFO:tasks.workunit.client.1.smithi148.stdout:3/410: rename d3/d5/c30 to d3/d5/c80 0 2024-09-21T04:15:16.905 INFO:tasks.workunit.client.1.smithi148.stdout:3/411: write d3/d1f/f3e [1008841,27529] 0 2024-09-21T04:15:16.908 INFO:tasks.workunit.client.1.smithi148.stdout:8/449: sync 2024-09-21T04:15:16.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/536: sync 2024-09-21T04:15:16.909 INFO:tasks.workunit.client.0.smithi110.stdout:7/429: mknod dd/d4d/c84 0 2024-09-21T04:15:16.911 INFO:tasks.workunit.client.1.smithi148.stdout:4/475: sync 2024-09-21T04:15:16.911 INFO:tasks.workunit.client.1.smithi148.stdout:0/456: truncate d5/d6/d2a/d2b/f39 2882040 0 2024-09-21T04:15:16.912 INFO:tasks.workunit.client.1.smithi148.stdout:5/423: mkdir d3/d14/d1f/d26/d59/d97 0 2024-09-21T04:15:16.912 INFO:tasks.workunit.client.0.smithi110.stdout:1/759: stat d2/d9/d26/d3d/d8f/l9b 0 2024-09-21T04:15:16.912 INFO:tasks.workunit.client.0.smithi110.stdout:1/760: chown d2/d9/d23/d4e/f61 12 1 2024-09-21T04:15:16.912 INFO:tasks.workunit.client.0.smithi110.stdout:1/761: write d2/d9/d26/d3d/d8f/fa3 [489589,63183] 0 2024-09-21T04:15:16.915 INFO:tasks.workunit.client.0.smithi110.stdout:5/501: rmdir d1/d49/d4d/d1e/d5a/d64/d7b 39 2024-09-21T04:15:16.915 INFO:tasks.workunit.client.0.smithi110.stdout:5/502: getdents d1/d7/d18/d8f 0 2024-09-21T04:15:16.915 INFO:tasks.workunit.client.0.smithi110.stdout:5/503: fsync d1/d7/f13 0 2024-09-21T04:15:16.917 INFO:tasks.workunit.client.0.smithi110.stdout:2/268: dread d2/d1a/d1f/d3f/f24 [0,4194304] 0 2024-09-21T04:15:16.918 INFO:tasks.workunit.client.0.smithi110.stdout:2/269: dread - d2/d1a/d1f/d29/f45 zero size 2024-09-21T04:15:16.919 INFO:tasks.workunit.client.1.smithi148.stdout:8/450: link dc/d13/d39/f66 dc/d13/d3c/d81/d85/f87 0 2024-09-21T04:15:16.923 INFO:tasks.workunit.client.0.smithi110.stdout:7/430: creat dd/d14/d17/f85 x:0 0 0 2024-09-21T04:15:16.924 INFO:tasks.workunit.client.1.smithi148.stdout:5/424: symlink d3/d14/l98 0 2024-09-21T04:15:16.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/405: read d7/d10/f12 [2139574,12705] 0 2024-09-21T04:15:16.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/762: creat d2/d9/d49/d4f/d87/ff4 x:0 0 0 2024-09-21T04:15:16.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/763: write d2/d9/d49/d4f/d53/fd4 [763607,80789] 0 2024-09-21T04:15:16.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/764: readlink d2/l18 0 2024-09-21T04:15:16.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/472: dwrite d4/d12/d17/d33/f64 [4194304,4194304] 0 2024-09-21T04:15:16.927 INFO:tasks.workunit.client.1.smithi148.stdout:0/457: symlink d5/d6/l8f 0 2024-09-21T04:15:16.935 INFO:tasks.workunit.client.1.smithi148.stdout:6/439: dwrite d0/d1a/d23/ff [0,4194304] 0 2024-09-21T04:15:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:16 smithi148 ceph-mon[34957]: pgmap v5: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:16 smithi148 ceph-mon[34957]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:16 smithi148 ceph-mon[34957]: mgrmap e24: smithi148.zjrahi(active, since 4s) 2024-09-21T04:15:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:16 smithi148 ceph-mon[34957]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:16.945 INFO:tasks.workunit.client.1.smithi148.stdout:9/440: dwrite d5/d60/d28/d2f/d8e/f99 [0,4194304] 0 2024-09-21T04:15:16.951 INFO:tasks.workunit.client.0.smithi110.stdout:5/504: creat d1/d49/fab x:0 0 0 2024-09-21T04:15:16.954 INFO:tasks.workunit.client.0.smithi110.stdout:2/270: stat d2/d1a/c42 0 2024-09-21T04:15:16.955 INFO:tasks.workunit.client.1.smithi148.stdout:8/451: creat dc/d13/d3c/f88 x:0 0 0 2024-09-21T04:15:16.955 INFO:tasks.workunit.client.1.smithi148.stdout:8/452: chown dc/d13/d76/d5d/d78 44 1 2024-09-21T04:15:16.955 INFO:tasks.workunit.client.1.smithi148.stdout:8/453: fdatasync dc/d13/d76/f2e 0 2024-09-21T04:15:16.955 INFO:tasks.workunit.client.0.smithi110.stdout:2/271: write d2/f44 [291433,79179] 0 2024-09-21T04:15:16.957 INFO:tasks.workunit.client.1.smithi148.stdout:0/458: dwrite d5/d6/d7/d22/f73 [0,4194304] 0 2024-09-21T04:15:16.957 INFO:tasks.workunit.client.0.smithi110.stdout:7/431: link dd/d14/f73 dd/d1c/d39/d56/f86 0 2024-09-21T04:15:16.958 INFO:tasks.workunit.client.1.smithi148.stdout:2/537: rename d0/d9/dc/d13/d25/d4e to d0/d2b/da3 0 2024-09-21T04:15:16.959 INFO:tasks.workunit.client.1.smithi148.stdout:2/538: fsync d0/d2b/da3/d8f/f9d 0 2024-09-21T04:15:16.959 INFO:tasks.workunit.client.0.smithi110.stdout:9/406: unlink d7/d10/f13 0 2024-09-21T04:15:16.962 INFO:tasks.workunit.client.1.smithi148.stdout:4/476: dwrite d4/f10 [0,4194304] 0 2024-09-21T04:15:16.964 INFO:tasks.workunit.client.1.smithi148.stdout:6/440: creat d0/d59/f7d x:0 0 0 2024-09-21T04:15:16.965 INFO:tasks.workunit.client.0.smithi110.stdout:6/473: rmdir d4/d12/d16/d66/d7a/d54 39 2024-09-21T04:15:16.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/412: dwrite d3/d5/d6/d12/f75 [0,4194304] 0 2024-09-21T04:15:16.966 INFO:tasks.workunit.client.1.smithi148.stdout:9/441: symlink d5/d2b/d32/la7 0 2024-09-21T04:15:16.967 INFO:tasks.workunit.client.0.smithi110.stdout:5/505: symlink d1/d49/lac 0 2024-09-21T04:15:16.968 INFO:tasks.workunit.client.0.smithi110.stdout:1/765: dwrite d2/d9/d49/d4f/d87/ff4 [0,4194304] 0 2024-09-21T04:15:16.969 INFO:tasks.workunit.client.1.smithi148.stdout:8/454: creat dc/d13/d39/d6e/f89 x:0 0 0 2024-09-21T04:15:16.973 INFO:tasks.workunit.client.1.smithi148.stdout:0/459: creat d5/d6/d7/d11/d60/f90 x:0 0 0 2024-09-21T04:15:16.977 INFO:tasks.workunit.client.1.smithi148.stdout:5/425: rename d3/d34/d3c/f48 to d3/d34/f99 0 2024-09-21T04:15:16.979 INFO:tasks.workunit.client.1.smithi148.stdout:2/539: truncate d0/f4 1828440 0 2024-09-21T04:15:16.983 INFO:tasks.workunit.client.1.smithi148.stdout:6/441: mknod d0/d8/c7e 0 2024-09-21T04:15:16.986 INFO:tasks.workunit.client.1.smithi148.stdout:0/460: dread d5/d6/f62 [0,4194304] 0 2024-09-21T04:15:16.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/413: mknod d3/d5/d6/d12/d15/c81 0 2024-09-21T04:15:16.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/414: chown d3/d5/d6/d12/da/d23/l34 13073453 1 2024-09-21T04:15:16.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/415: chown d3/d5/d6/d2b/l2d 1243839447 1 2024-09-21T04:15:16.990 INFO:tasks.workunit.client.1.smithi148.stdout:3/416: write d3/f49 [1168155,112903] 0 2024-09-21T04:15:16.990 INFO:tasks.workunit.client.0.smithi110.stdout:2/272: unlink d2/d1a/c42 0 2024-09-21T04:15:16.993 INFO:tasks.workunit.client.1.smithi148.stdout:9/442: mkdir d5/d1f/d81/d9e/da8 0 2024-09-21T04:15:16.994 INFO:tasks.workunit.client.0.smithi110.stdout:7/432: truncate dd/f57 1429723 0 2024-09-21T04:15:16.995 INFO:tasks.workunit.client.0.smithi110.stdout:7/433: chown dd/d1c/d39/f76 24 1 2024-09-21T04:15:16.997 INFO:tasks.workunit.client.0.smithi110.stdout:5/506: mknod d1/d49/d69/cad 0 2024-09-21T04:15:16.997 INFO:tasks.workunit.client.0.smithi110.stdout:5/507: dread - d1/d22/f97 zero size 2024-09-21T04:15:16.997 INFO:tasks.workunit.client.0.smithi110.stdout:5/508: chown d1/d39 43768208 1 2024-09-21T04:15:16.998 INFO:tasks.workunit.client.1.smithi148.stdout:2/540: creat d0/d2b/da3/d8f/fa4 x:0 0 0 2024-09-21T04:15:17.001 INFO:tasks.workunit.client.1.smithi148.stdout:6/442: creat d0/d1a/d23/d27/d62/f7f x:0 0 0 2024-09-21T04:15:17.004 INFO:tasks.workunit.client.1.smithi148.stdout:0/461: rename d5/d6/d7/l7c to d5/d6/d2a/d2b/d84/l91 0 2024-09-21T04:15:17.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/455: dwrite dc/d13/d39/f51 [0,4194304] 0 2024-09-21T04:15:17.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/456: write dc/d13/d76/f40 [4158842,125753] 0 2024-09-21T04:15:17.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/457: chown dc/d13/d76/d42/l77 51512 1 2024-09-21T04:15:17.005 INFO:tasks.workunit.client.0.smithi110.stdout:6/474: dwrite d4/d12/d16/d66/d7a/d4b/f5e [0,4194304] 0 2024-09-21T04:15:17.005 INFO:tasks.workunit.client.0.smithi110.stdout:6/475: stat d4/d3c/c4c 0 2024-09-21T04:15:17.005 INFO:tasks.workunit.client.1.smithi148.stdout:5/426: dwrite d3/f12 [0,4194304] 0 2024-09-21T04:15:17.006 INFO:tasks.workunit.client.1.smithi148.stdout:2/541: dread d0/d9/dc/d13/f80 [0,4194304] 0 2024-09-21T04:15:17.006 INFO:tasks.workunit.client.1.smithi148.stdout:4/477: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:17.006 INFO:tasks.workunit.client.0.smithi110.stdout:1/766: rename d2/db7/d86/fb9 to d2/d9/d23/de0/ff5 0 2024-09-21T04:15:17.008 INFO:tasks.workunit.client.1.smithi148.stdout:9/443: link d5/l17 d5/d2b/d4e/la9 0 2024-09-21T04:15:17.010 INFO:tasks.workunit.client.0.smithi110.stdout:2/273: rmdir d2/d1a/d1f/d29/d19 39 2024-09-21T04:15:17.015 INFO:tasks.workunit.client.1.smithi148.stdout:5/427: dread d3/d24/f76 [4194304,4194304] 0 2024-09-21T04:15:17.015 INFO:tasks.workunit.client.1.smithi148.stdout:5/428: creat d3/d14/d1a/f9a x:0 0 0 2024-09-21T04:15:17.015 INFO:tasks.workunit.client.0.smithi110.stdout:5/509: mkdir d1/d49/d4d/d1e/d4a/dae 0 2024-09-21T04:15:17.015 INFO:tasks.workunit.client.0.smithi110.stdout:5/510: chown d1/d39/f79 12816289 1 2024-09-21T04:15:17.019 INFO:tasks.workunit.client.0.smithi110.stdout:0/739: sync 2024-09-21T04:15:17.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/443: rmdir d0/d34/d38 39 2024-09-21T04:15:17.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/444: chown d0/d1a/d23/l2b 1719387 1 2024-09-21T04:15:17.019 INFO:tasks.workunit.client.0.smithi110.stdout:1/767: write d2/d9/d49/d4f/d53/f8c [2070324,87218] 0 2024-09-21T04:15:17.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/458: dwrite dc/f14 [0,4194304] 0 2024-09-21T04:15:17.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/459: stat dc/d13/d28/d62 0 2024-09-21T04:15:17.023 INFO:tasks.workunit.client.0.smithi110.stdout:2/274: dread d2/f16 [0,4194304] 0 2024-09-21T04:15:17.023 INFO:tasks.workunit.client.0.smithi110.stdout:2/275: chown d2/d1a/d1f/d3f/f43 188326 1 2024-09-21T04:15:17.024 INFO:tasks.workunit.client.0.smithi110.stdout:6/476: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d89/d95 0 2024-09-21T04:15:17.025 INFO:tasks.workunit.client.0.smithi110.stdout:6/477: write d4/d12/d17/d2e/d40/f2f [2735404,64205] 0 2024-09-21T04:15:17.026 INFO:tasks.workunit.client.1.smithi148.stdout:0/462: mknod d5/d6/d2a/d45/c92 0 2024-09-21T04:15:17.027 INFO:tasks.workunit.client.0.smithi110.stdout:6/478: write d4/f6 [183757,19712] 0 2024-09-21T04:15:17.027 INFO:tasks.workunit.client.0.smithi110.stdout:6/479: chown d4/d12/d17/d33/d3e/f53 3655529 1 2024-09-21T04:15:17.027 INFO:tasks.workunit.client.0.smithi110.stdout:7/434: getdents dd/d1c/d39 0 2024-09-21T04:15:17.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/435: readlink dd/d14/d17/l29 0 2024-09-21T04:15:17.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/436: chown dd/d1c/d39/f76 110 1 2024-09-21T04:15:17.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/437: creat dd/d48/d55/f87 x:0 0 0 2024-09-21T04:15:17.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/438: chown dd/d1c/c37 72462 1 2024-09-21T04:15:17.033 INFO:tasks.workunit.client.0.smithi110.stdout:5/511: dwrite d1/d49/d4d/d1e/d5a/f67 [0,4194304] 0 2024-09-21T04:15:17.033 INFO:tasks.workunit.client.0.smithi110.stdout:5/512: creat d1/d49/d4d/d1e/d4a/faf x:0 0 0 2024-09-21T04:15:17.033 INFO:tasks.workunit.client.0.smithi110.stdout:5/513: fdatasync d1/d49/d4d/d1e/d4a/faf 0 2024-09-21T04:15:17.034 INFO:tasks.workunit.client.1.smithi148.stdout:2/542: creat d0/d9/d16/d55/d71/fa5 x:0 0 0 2024-09-21T04:15:17.035 INFO:tasks.workunit.client.0.smithi110.stdout:2/276: creat d2/d1a/d1f/d2a/d41/f49 x:0 0 0 2024-09-21T04:15:17.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/444: creat d5/d1e/faa x:0 0 0 2024-09-21T04:15:17.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/445: creat d5/d2b/fab x:0 0 0 2024-09-21T04:15:17.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/446: write d5/d60/d28/f6a [691786,50517] 0 2024-09-21T04:15:17.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/480: creat d4/d12/d17/d33/f96 x:0 0 0 2024-09-21T04:15:17.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/481: chown d4/f13 390913236 1 2024-09-21T04:15:17.045 INFO:tasks.workunit.client.1.smithi148.stdout:5/429: creat d3/d34/d3c/d5d/d75/f9b x:0 0 0 2024-09-21T04:15:17.046 INFO:tasks.workunit.client.1.smithi148.stdout:5/430: creat d3/d34/d3c/d5d/f9c x:0 0 0 2024-09-21T04:15:17.047 INFO:tasks.workunit.client.0.smithi110.stdout:7/439: symlink dd/d1c/d39/d4a/l88 0 2024-09-21T04:15:17.048 INFO:tasks.workunit.client.1.smithi148.stdout:6/445: mkdir d0/d1a/d23/d27/d80 0 2024-09-21T04:15:17.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/447: dwrite d5/d1e/d37/d46/f7e [4194304,4194304] 0 2024-09-21T04:15:17.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/448: creat d5/d1e/d37/d46/fac x:0 0 0 2024-09-21T04:15:17.055 INFO:tasks.workunit.client.1.smithi148.stdout:6/446: dread d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:17.058 INFO:tasks.workunit.client.0.smithi110.stdout:2/277: dwrite d2/d1a/d1f/d3f/f24 [4194304,4194304] 0 2024-09-21T04:15:17.062 INFO:tasks.workunit.client.1.smithi148.stdout:9/449: dread d5/d1e/f49 [4194304,4194304] 0 2024-09-21T04:15:17.062 INFO:tasks.workunit.client.1.smithi148.stdout:9/450: fsync d5/d1e/f65 0 2024-09-21T04:15:17.079 INFO:tasks.workunit.client.1.smithi148.stdout:6/447: dwrite d0/d8/d18/f4d [0,4194304] 0 2024-09-21T04:15:17.079 INFO:tasks.workunit.client.1.smithi148.stdout:6/448: chown d0/d1a/d23/d27 389859080 1 2024-09-21T04:15:17.080 INFO:tasks.workunit.client.1.smithi148.stdout:6/449: dread - d0/d1a/d23/d53/f5f zero size 2024-09-21T04:15:17.080 INFO:tasks.workunit.client.1.smithi148.stdout:6/450: fdatasync d0/d8/d18/f25 0 2024-09-21T04:15:17.080 INFO:tasks.workunit.client.1.smithi148.stdout:6/451: dread d0/d34/d38/f3a [0,4194304] 0 2024-09-21T04:15:17.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/460: truncate dc/d13/d3c/d81/d85/f87 117539 0 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/461: dread - dc/d13/d3c/f88 zero size 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/462: readlink dc/l22 0 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/463: stat f9 0 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/464: chown dc/d13/d76/d5d/f83 6912772 1 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/465: read - dc/d43/f54 zero size 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/466: readlink dc/d13/d39/d6e/l73 0 2024-09-21T04:15:17.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/467: truncate dc/f18 3491430 0 2024-09-21T04:15:17.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/468: write dc/d13/f1c [313701,49419] 0 2024-09-21T04:15:17.142 INFO:tasks.workunit.client.1.smithi148.stdout:3/417: rename d3/d5/d6/d12/da to d3/d1f/d4d/d82 0 2024-09-21T04:15:17.155 INFO:tasks.workunit.client.1.smithi148.stdout:0/463: link d5/c67 d5/d6/d2a/d2b/d84/c93 0 2024-09-21T04:15:17.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/418: dwrite d3/d1f/d4d/d82/f3d [4194304,4194304] 0 2024-09-21T04:15:17.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/419: chown d3/d1f/d4d/d82/d21/c4b 635 1 2024-09-21T04:15:17.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/420: write d3/d1f/d4d/d82/f11 [4681624,62655] 0 2024-09-21T04:15:17.161 INFO:tasks.workunit.client.0.smithi110.stdout:6/482: truncate d4/d12/f2b 1177909 0 2024-09-21T04:15:17.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.158+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fefc010e3d0 con 0x7fefc01077d0 2024-09-21T04:15:17.161 INFO:tasks.workunit.client.0.smithi110.stdout:1/768: write d2/d9/d23/f40 [1381454,96884] 0 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.159+0000 7fefbdffb640 1 -- 172.21.15.110:0/3767295695 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fefb00202f0 con 0x7fefc01077d0 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 12, 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:15:17.163 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:15:17.164 INFO:tasks.workunit.client.0.smithi110.stdout:7/440: mkdir dd/d1c/d89 0 2024-09-21T04:15:17.165 INFO:tasks.workunit.client.1.smithi148.stdout:5/431: symlink d3/d24/l9d 0 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 msgr2=0x7fefa8042010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 0x7fefa8042010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefb802ba30 tx=0x7fefb8000f40 comp rx=0 tx=0).stop 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 msgr2=0x7fefc01a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d6f0 tx=0x7fefb000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 shutdown_connections 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fefa803fb50 0x7fefa8042010 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc01077d0 0x7fefc01a48e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 --2- 172.21.15.110:0/3767295695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefc0071890 0x7fefc01a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.164+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 >> 172.21.15.110:0/3767295695 conn(0x7fefc006cf90 msgr2=0x7fefc006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.165+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 shutdown_connections 2024-09-21T04:15:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.165+0000 7fefc6bf5640 1 -- 172.21.15.110:0/3767295695 wait complete. 2024-09-21T04:15:17.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/421: dread d3/d1f/f47 [4194304,4194304] 0 2024-09-21T04:15:17.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/422: write d3/d5/f1e [2136803,26728] 0 2024-09-21T04:15:17.178 INFO:tasks.workunit.client.0.smithi110.stdout:5/514: getdents d1/d49/d4d/d1e 0 2024-09-21T04:15:17.178 INFO:tasks.workunit.client.0.smithi110.stdout:5/515: write d1/d7/d58/f60 [495858,50724] 0 2024-09-21T04:15:17.179 INFO:tasks.workunit.client.1.smithi148.stdout:8/469: write dc/d43/d4c/d4f/f79 [281437,81369] 0 2024-09-21T04:15:17.180 INFO:tasks.workunit.client.0.smithi110.stdout:6/483: rename d4/d12/d16/d66/d7a/d4b/f71 to d4/d12/d16/d66/d7a/d4b/f97 0 2024-09-21T04:15:17.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/516: mknod d1/d49/d4d/d1e/d4a/dae/cb0 0 2024-09-21T04:15:17.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/517: creat d1/fb1 x:0 0 0 2024-09-21T04:15:17.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/518: stat d1/f4c 0 2024-09-21T04:15:17.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/519: write d1/d49/d4d/d1e/d4a/faf [821081,54045] 0 2024-09-21T04:15:17.188 INFO:tasks.workunit.client.0.smithi110.stdout:6/484: creat d4/d12/d17/d2e/d38/f98 x:0 0 0 2024-09-21T04:15:17.189 INFO:tasks.workunit.client.0.smithi110.stdout:6/485: dread d4/de/f84 [0,4194304] 0 2024-09-21T04:15:17.190 INFO:tasks.workunit.client.1.smithi148.stdout:9/451: creat d5/d2b/d4e/d95/fad x:0 0 0 2024-09-21T04:15:17.191 INFO:tasks.workunit.client.0.smithi110.stdout:1/769: dwrite d2/d9/d26/d3d/f58 [0,4194304] 0 2024-09-21T04:15:17.192 INFO:tasks.workunit.client.0.smithi110.stdout:7/441: dwrite dd/d48/d55/f87 [0,4194304] 0 2024-09-21T04:15:17.196 INFO:tasks.workunit.client.0.smithi110.stdout:5/520: dread d1/f8a [0,4194304] 0 2024-09-21T04:15:17.196 INFO:tasks.workunit.client.1.smithi148.stdout:6/452: mkdir d0/d1a/d23/dc/d81 0 2024-09-21T04:15:17.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/278: truncate d2/d1a/d1f/d3f/f24 3910040 0 2024-09-21T04:15:17.198 INFO:tasks.workunit.client.0.smithi110.stdout:5/521: read d1/d22/f27 [4816,26912] 0 2024-09-21T04:15:17.198 INFO:tasks.workunit.client.0.smithi110.stdout:5/522: chown d1/d49/d4d/l1f 434 1 2024-09-21T04:15:17.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/478: rename d4/d66/d7f/d2a/c5d to d4/d12/d2c/c86 0 2024-09-21T04:15:17.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/479: write d4/d66/d7f/f4a [1247428,99692] 0 2024-09-21T04:15:17.201 INFO:tasks.workunit.client.1.smithi148.stdout:0/464: symlink d5/d82/l94 0 2024-09-21T04:15:17.203 INFO:tasks.workunit.client.0.smithi110.stdout:2/279: link d2/d1a/d1f/f38 d2/d1a/d1f/d3f/f4a 0 2024-09-21T04:15:17.203 INFO:tasks.workunit.client.0.smithi110.stdout:2/280: dread - d2/d1a/d1f/d3f/f46 zero size 2024-09-21T04:15:17.206 INFO:tasks.workunit.client.1.smithi148.stdout:1/416: sync 2024-09-21T04:15:17.206 INFO:tasks.workunit.client.1.smithi148.stdout:7/458: sync 2024-09-21T04:15:17.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/543: symlink d0/d9/la6 0 2024-09-21T04:15:17.208 INFO:tasks.workunit.client.0.smithi110.stdout:1/770: dread d2/d9/d23/de0/f9e [0,4194304] 0 2024-09-21T04:15:17.210 INFO:tasks.workunit.client.0.smithi110.stdout:5/523: unlink d1/d49/f6c 0 2024-09-21T04:15:17.210 INFO:tasks.workunit.client.0.smithi110.stdout:5/524: chown d1/d49/d4d/f9c 4863 1 2024-09-21T04:15:17.210 INFO:tasks.workunit.client.0.smithi110.stdout:5/525: write d1/d39/f79 [4573613,88312] 0 2024-09-21T04:15:17.211 INFO:tasks.workunit.client.0.smithi110.stdout:7/442: dread dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:17.214 INFO:tasks.workunit.client.1.smithi148.stdout:5/432: truncate d3/d14/f44 623276 0 2024-09-21T04:15:17.214 INFO:tasks.workunit.client.1.smithi148.stdout:8/470: dwrite dc/d13/d3c/f5f [0,4194304] 0 2024-09-21T04:15:17.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/526: read d1/d49/d4d/d1e/d5a/d64/f73 [1952074,8081] 0 2024-09-21T04:15:17.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/527: chown d1/l91 1802886 1 2024-09-21T04:15:17.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/528: dread - d1/d49/d4d/f89 zero size 2024-09-21T04:15:17.216 INFO:tasks.workunit.client.0.smithi110.stdout:1/771: link d2/d9/d23/d3b/cef d2/d9/d23/d3b/cf6 0 2024-09-21T04:15:17.220 INFO:tasks.workunit.client.1.smithi148.stdout:3/423: unlink d3/d1f/d4d/d82/d21/c4b 0 2024-09-21T04:15:17.221 INFO:tasks.workunit.client.1.smithi148.stdout:3/424: fdatasync d3/d5/d5c/f58 0 2024-09-21T04:15:17.221 INFO:tasks.workunit.client.0.smithi110.stdout:5/529: dread d1/d49/d4d/d1e/f61 [0,4194304] 0 2024-09-21T04:15:17.222 INFO:tasks.workunit.client.1.smithi148.stdout:9/452: symlink d5/d60/d28/d73/lae 0 2024-09-21T04:15:17.222 INFO:tasks.workunit.client.1.smithi148.stdout:9/453: write d5/f1b [976199,41480] 0 2024-09-21T04:15:17.224 INFO:tasks.workunit.client.1.smithi148.stdout:7/459: dwrite d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:17.224 INFO:tasks.workunit.client.1.smithi148.stdout:7/460: readlink d5/d8/d6f/d7d/l5f 0 2024-09-21T04:15:17.225 INFO:tasks.workunit.client.0.smithi110.stdout:6/486: dwrite d4/d12/d16/d66/d7a/d4b/f5e [0,4194304] 0 2024-09-21T04:15:17.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/453: creat d0/d1a/d23/f82 x:0 0 0 2024-09-21T04:15:17.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/454: fsync d0/f6b 0 2024-09-21T04:15:17.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/455: chown d0/d8/d18 0 1 2024-09-21T04:15:17.227 INFO:tasks.workunit.client.0.smithi110.stdout:5/530: getdents d1/d49/d69 0 2024-09-21T04:15:17.231 INFO:tasks.workunit.client.0.smithi110.stdout:3/396: sync 2024-09-21T04:15:17.231 INFO:tasks.workunit.client.1.smithi148.stdout:9/454: write d5/f36 [3182312,107326] 0 2024-09-21T04:15:17.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/544: mkdir d0/d9/d16/d55/d6f/da7 0 2024-09-21T04:15:17.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/545: chown d0/f1d 13 1 2024-09-21T04:15:17.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/546: chown d0/d9/d16/d55/d6f/f33 155681035 1 2024-09-21T04:15:17.236 INFO:tasks.workunit.client.0.smithi110.stdout:6/487: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99 0 2024-09-21T04:15:17.236 INFO:tasks.workunit.client.0.smithi110.stdout:6/488: readlink d4/d12/d16/l36 0 2024-09-21T04:15:17.237 INFO:tasks.workunit.client.1.smithi148.stdout:9/455: dread d5/d60/d28/d38/f47 [0,4194304] 0 2024-09-21T04:15:17.237 INFO:tasks.workunit.client.0.smithi110.stdout:2/281: fsync d2/d1a/d1f/d3f/f4a 0 2024-09-21T04:15:17.237 INFO:tasks.workunit.client.0.smithi110.stdout:7/443: dwrite dd/d1c/f2c [0,4194304] 0 2024-09-21T04:15:17.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/417: dread db/dd/f12 [0,4194304] 0 2024-09-21T04:15:17.239 INFO:tasks.workunit.client.0.smithi110.stdout:2/282: dread d2/d1a/d1f/d3f/f3a [0,4194304] 0 2024-09-21T04:15:17.240 INFO:tasks.workunit.client.0.smithi110.stdout:2/283: chown d2/d1a/d1f/f38 71 1 2024-09-21T04:15:17.240 INFO:tasks.workunit.client.0.smithi110.stdout:2/284: fsync d2/f3b 0 2024-09-21T04:15:17.240 INFO:tasks.workunit.client.0.smithi110.stdout:7/444: dread dd/d1c/f34 [0,4194304] 0 2024-09-21T04:15:17.240 INFO:tasks.workunit.client.0.smithi110.stdout:7/445: fsync dd/f57 0 2024-09-21T04:15:17.242 INFO:tasks.workunit.client.0.smithi110.stdout:5/531: symlink d1/d49/lb2 0 2024-09-21T04:15:17.242 INFO:tasks.workunit.client.0.smithi110.stdout:5/532: readlink d1/d7/d58/l76 0 2024-09-21T04:15:17.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/471: creat dc/d43/d4c/d4f/f8a x:0 0 0 2024-09-21T04:15:17.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/425: creat d3/d5/d6/d12/d43/f83 x:0 0 0 2024-09-21T04:15:17.249 INFO:tasks.workunit.client.1.smithi148.stdout:3/426: write d3/d1f/f3e [1210915,43422] 0 2024-09-21T04:15:17.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/533: dread d1/f15 [0,4194304] 0 2024-09-21T04:15:17.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/534: dread - d1/d49/d4d/f89 zero size 2024-09-21T04:15:17.252 INFO:tasks.workunit.client.0.smithi110.stdout:6/489: mkdir d4/d12/d17/d9a 0 2024-09-21T04:15:17.254 INFO:tasks.workunit.client.0.smithi110.stdout:2/285: creat d2/d1a/d1f/d29/f4b x:0 0 0 2024-09-21T04:15:17.254 INFO:tasks.workunit.client.0.smithi110.stdout:2/286: dread - d2/d1a/d1f/d29/f4b zero size 2024-09-21T04:15:17.255 INFO:tasks.workunit.client.0.smithi110.stdout:2/287: creat d2/d1a/d1f/d29/f4c x:0 0 0 2024-09-21T04:15:17.255 INFO:tasks.workunit.client.0.smithi110.stdout:2/288: creat d2/d1a/d1f/f4d x:0 0 0 2024-09-21T04:15:17.256 INFO:tasks.workunit.client.0.smithi110.stdout:7/446: mkdir dd/d14/d17/d8a 0 2024-09-21T04:15:17.256 INFO:tasks.workunit.client.0.smithi110.stdout:7/447: fdatasync dd/d14/d17/f3a 0 2024-09-21T04:15:17.257 INFO:tasks.workunit.client.0.smithi110.stdout:5/535: mknod d1/d7/d58/cb3 0 2024-09-21T04:15:17.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/490: unlink d4/d12/d17/d2e/d40/l7b 0 2024-09-21T04:15:17.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/461: creat d5/d8/d6f/d7d/d90/d28/f9c x:0 0 0 2024-09-21T04:15:17.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/462: fdatasync d5/d8/d6f/d7d/d90/f50 0 2024-09-21T04:15:17.268 INFO:tasks.workunit.client.0.smithi110.stdout:2/289: link d2/d1a/d1f/d29/c14 d2/d1a/d1f/d2a/c4e 0 2024-09-21T04:15:17.268 INFO:tasks.workunit.client.0.smithi110.stdout:2/290: creat d2/d1a/d1f/d29/f4f x:0 0 0 2024-09-21T04:15:17.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/456: creat d0/d1a/f83 x:0 0 0 2024-09-21T04:15:17.271 INFO:tasks.workunit.client.0.smithi110.stdout:7/448: creat dd/d14/d17/d8a/f8b x:0 0 0 2024-09-21T04:15:17.278 INFO:tasks.workunit.client.0.smithi110.stdout:5/536: dwrite d1/d49/d4d/d1e/f61 [0,4194304] 0 2024-09-21T04:15:17.281 INFO:tasks.workunit.client.0.smithi110.stdout:6/491: rename d4/de/f44 to d4/d86/d8f/f9b 0 2024-09-21T04:15:17.286 INFO:tasks.workunit.client.1.smithi148.stdout:4/480: rename d4/d12/d2c to d4/d87 0 2024-09-21T04:15:17.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/537: rename d1/d7/d58/f7c to d1/d49/d4d/d1e/d4a/dae/fb4 0 2024-09-21T04:15:17.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/538: truncate d1/d49/d4d/f89 911085 0 2024-09-21T04:15:17.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/539: chown d1/d49/d4d/d1e/d5a/d64/f80 137 1 2024-09-21T04:15:17.293 INFO:tasks.workunit.client.1.smithi148.stdout:6/457: dwrite d0/d1a/d23/d16/f3f [0,4194304] 0 2024-09-21T04:15:17.293 INFO:tasks.workunit.client.1.smithi148.stdout:6/458: chown d0/d1a/d23/l31 1915805 1 2024-09-21T04:15:17.294 INFO:tasks.workunit.client.0.smithi110.stdout:2/291: dwrite d2/d1a/d1f/d29/f45 [0,4194304] 0 2024-09-21T04:15:17.294 INFO:tasks.workunit.client.0.smithi110.stdout:2/292: write d2/d1a/d1f/d29/f4b [760695,28552] 0 2024-09-21T04:15:17.294 INFO:tasks.workunit.client.0.smithi110.stdout:7/449: dwrite dd/d4d/f59 [0,4194304] 0 2024-09-21T04:15:17.295 INFO:tasks.workunit.client.0.smithi110.stdout:6/492: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:17.295 INFO:tasks.workunit.client.0.smithi110.stdout:6/493: fdatasync d4/d12/d16/d66/d7a/d4b/f97 0 2024-09-21T04:15:17.296 INFO:tasks.workunit.client.1.smithi148.stdout:9/456: symlink d5/d2b/d32/d67/laf 0 2024-09-21T04:15:17.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/433: getdents d3/d14/d1f/d26/d2e 0 2024-09-21T04:15:17.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/434: truncate d3/d14/f44 726850 0 2024-09-21T04:15:17.300 INFO:tasks.workunit.client.0.smithi110.stdout:2/293: dread d2/d1a/d1f/d3f/f4a [0,4194304] 0 2024-09-21T04:15:17.300 INFO:tasks.workunit.client.0.smithi110.stdout:2/294: write d2/d1a/d1f/d2a/f21 [6353146,109425] 0 2024-09-21T04:15:17.300 INFO:tasks.workunit.client.0.smithi110.stdout:2/295: chown d2/d1a/d1f/d29/c3e 68934 1 2024-09-21T04:15:17.302 INFO:tasks.workunit.client.1.smithi148.stdout:1/418: creat db/d46/d4c/d82/f88 x:0 0 0 2024-09-21T04:15:17.303 INFO:tasks.workunit.client.0.smithi110.stdout:7/450: dread dd/d1c/d39/f76 [0,4194304] 0 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 -- 172.21.15.110:0/1984209783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c071760 msgr2=0x7f841c071b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 --2- 172.21.15.110:0/1984209783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c071760 0x7f841c071b60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f840c009880 tx=0x7f840c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 -- 172.21.15.110:0/1984209783 shutdown_connections 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 --2- 172.21.15.110:0/1984209783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c1085c0 0x7f841c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 --2- 172.21.15.110:0/1984209783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c071760 0x7f841c071b60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 -- 172.21.15.110:0/1984209783 >> 172.21.15.110:0/1984209783 conn(0x7f841c06cc80 msgr2=0x7f841c06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:17.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.302+0000 7f8421ef1640 1 -- 172.21.15.110:0/1984209783 shutdown_connections 2024-09-21T04:15:17.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.303+0000 7f8421ef1640 1 -- 172.21.15.110:0/1984209783 wait complete. 2024-09-21T04:15:17.307 INFO:tasks.workunit.client.0.smithi110.stdout:2/296: read d2/d1a/f20 [2145734,39715] 0 2024-09-21T04:15:17.308 INFO:tasks.workunit.client.1.smithi148.stdout:8/472: link dc/f21 dc/d3e/f8b 0 2024-09-21T04:15:17.308 INFO:tasks.workunit.client.1.smithi148.stdout:8/473: truncate dc/d13/d28/f2f 1000049 0 2024-09-21T04:15:17.308 INFO:tasks.workunit.client.1.smithi148.stdout:4/481: dwrite d4/d12/f5b [0,4194304] 0 2024-09-21T04:15:17.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.304+0000 7f8421ef1640 1 Processor -- start 2024-09-21T04:15:17.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.304+0000 7f8421ef1640 1 -- start start 2024-09-21T04:15:17.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.304+0000 7f8421ef1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c071760 0x7f841c10f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f8421ef1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f8421ef1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c10df00 con 0x7f841c1085c0 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f8421ef1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c10e070 con 0x7f841c071760 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48040/0 (socket says 172.21.15.110:48040) 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 -- 172.21.15.110:0/163999997 learned_addr learned my addr 172.21.15.110:0/163999997 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841b7fe640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c071760 0x7f841c10f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 -- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c071760 msgr2=0x7f841c10f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c071760 0x7f841c10f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 -- 172.21.15.110:0/163999997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009560 con 0x7f841c1085c0 2024-09-21T04:15:17.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.305+0000 7f841affd640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f841000b420 tx=0x7f841000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:17.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.306+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8410011800 con 0x7f841c1085c0 2024-09-21T04:15:17.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.306+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8410011e20 con 0x7f841c1085c0 2024-09-21T04:15:17.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.306+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c10e350 con 0x7f841c1085c0 2024-09-21T04:15:17.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.306+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f841000f2e0 con 0x7f841c1085c0 2024-09-21T04:15:17.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.306+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c19daf0 con 0x7f841c1085c0 2024-09-21T04:15:17.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/494: symlink d4/d12/d16/d66/d7a/d54/l9c 0 2024-09-21T04:15:17.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/495: chown d4/d12/d17/d2e 33191 1 2024-09-21T04:15:17.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.309+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) v1 ==== 51935+0+0 (secure 0 0 0) 0x7f8410010380 con 0x7f841c1085c0 2024-09-21T04:15:17.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.309+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c071be0 con 0x7f841c1085c0 2024-09-21T04:15:17.312 INFO:tasks.workunit.client.1.smithi148.stdout:7/463: creat d5/d8/d6f/f9d x:0 0 0 2024-09-21T04:15:17.312 INFO:tasks.workunit.client.1.smithi148.stdout:7/464: creat d5/d14/d25/d40/d7b/f9e x:0 0 0 2024-09-21T04:15:17.313 INFO:tasks.workunit.client.1.smithi148.stdout:7/465: write d5/d14/f78 [47842,99668] 0 2024-09-21T04:15:17.313 INFO:tasks.workunit.client.0.smithi110.stdout:2/297: unlink d2/d1a/c32 0 2024-09-21T04:15:17.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.310+0000 7f8418ff9640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 0x7f8404042010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.310+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8410054d80 con 0x7f841c1085c0 2024-09-21T04:15:17.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.311+0000 7f841b7fe640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 0x7f8404042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/547: getdents d0/d9 0 2024-09-21T04:15:17.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/548: creat d0/d2b/da3/d8f/fa8 x:0 0 0 2024-09-21T04:15:17.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/549: dread d0/d9/f1b [0,4194304] 0 2024-09-21T04:15:17.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/550: dread - d0/d2b/f9b zero size 2024-09-21T04:15:17.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/551: stat d0/d9/d38/f41 0 2024-09-21T04:15:17.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/552: readlink d0/d9/d16/d55/d6f/l72 0 2024-09-21T04:15:17.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/553: write d0/d2f/f84 [534594,122350] 0 2024-09-21T04:15:17.318 INFO:tasks.workunit.client.1.smithi148.stdout:6/459: unlink d0/d1a/d23/c6 0 2024-09-21T04:15:17.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.314+0000 7f841b7fe640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 0x7f8404042010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f840c000c00 tx=0x7f840c005db0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:17.321 INFO:tasks.workunit.client.1.smithi148.stdout:9/457: rename d5/d2b/d32/f9f to d5/d1e/d37/fb0 0 2024-09-21T04:15:17.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.319+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f8410011960 con 0x7f841c1085c0 2024-09-21T04:15:17.323 INFO:tasks.workunit.client.1.smithi148.stdout:5/435: symlink d3/d14/d1f/d26/d2e/d3d/l9e 0 2024-09-21T04:15:17.327 INFO:tasks.workunit.client.1.smithi148.stdout:8/474: creat dc/d43/d4c/f8c x:0 0 0 2024-09-21T04:15:17.331 INFO:tasks.workunit.client.1.smithi148.stdout:6/460: dread d0/d1a/d23/dc/d68/d5b/f4a [0,4194304] 0 2024-09-21T04:15:17.332 INFO:tasks.workunit.client.1.smithi148.stdout:9/458: write d5/d1e/f49 [7515107,110110] 0 2024-09-21T04:15:17.333 INFO:tasks.workunit.client.1.smithi148.stdout:7/466: creat d5/d14/f9f x:0 0 0 2024-09-21T04:15:17.337 INFO:tasks.workunit.client.1.smithi148.stdout:2/554: rename d0/d9/d16/d55/d71/f93 to d0/d2b/da3/d95/fa9 0 2024-09-21T04:15:17.337 INFO:tasks.workunit.client.1.smithi148.stdout:2/555: stat d0/d9/dc/d13/d50/f70 0 2024-09-21T04:15:17.337 INFO:tasks.workunit.client.1.smithi148.stdout:2/556: write d0/d9/f17 [3495962,12168] 0 2024-09-21T04:15:17.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/436: truncate d3/d14/f2d 3426169 0 2024-09-21T04:15:17.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/475: mknod dc/d13/d39/d6e/c8d 0 2024-09-21T04:15:17.340 INFO:tasks.workunit.client.1.smithi148.stdout:7/467: truncate d5/d8/f16 7381142 0 2024-09-21T04:15:17.340 INFO:tasks.workunit.client.1.smithi148.stdout:2/557: rmdir d0/d2b/da3/d95 39 2024-09-21T04:15:17.343 INFO:tasks.workunit.client.1.smithi148.stdout:6/461: dread d0/d1a/d23/d27/f2d [0,4194304] 0 2024-09-21T04:15:17.343 INFO:tasks.workunit.client.1.smithi148.stdout:1/419: dwrite db/f58 [0,4194304] 0 2024-09-21T04:15:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:7/451: dwrite dd/f7e [0,4194304] 0 2024-09-21T04:15:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:6/496: dwrite d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:6/497: creat d4/d12/d17/f9d x:0 0 0 2024-09-21T04:15:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:2/298: dwrite d2/d1a/d1f/d29/f4f [0,4194304] 0 2024-09-21T04:15:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:2/299: chown d2/d1a/d1f/d2a/c39 0 1 2024-09-21T04:15:17.346 INFO:tasks.workunit.client.1.smithi148.stdout:5/437: mknod d3/d34/d78/c9f 0 2024-09-21T04:15:17.349 INFO:tasks.workunit.client.1.smithi148.stdout:7/468: link d5/d8/cd d5/d14/d25/ca0 0 2024-09-21T04:15:17.349 INFO:tasks.workunit.client.1.smithi148.stdout:7/469: write d5/d8/d6f/d7d/f68 [4583265,7191] 0 2024-09-21T04:15:17.349 INFO:tasks.workunit.client.1.smithi148.stdout:7/470: chown d5/d8/d6f/d7d/d8a/c94 11252 1 2024-09-21T04:15:17.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/476: write dc/f68 [3941606,25658] 0 2024-09-21T04:15:17.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/477: creat dc/d43/d4c/d4f/f8e x:0 0 0 2024-09-21T04:15:17.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/478: dread - dc/d13/d6a/f86 zero size 2024-09-21T04:15:17.353 INFO:tasks.workunit.client.0.smithi110.stdout:7/452: link dd/d14/d17/f85 dd/d1c/f8c 0 2024-09-21T04:15:17.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/558: dwrite d0/f9c [0,4194304] 0 2024-09-21T04:15:17.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/559: write d0/d9/dc/f99 [1794332,84271] 0 2024-09-21T04:15:17.361 INFO:tasks.workunit.client.0.smithi110.stdout:6/498: truncate d4/f6c 3967349 0 2024-09-21T04:15:17.361 INFO:tasks.workunit.client.0.smithi110.stdout:6/499: fdatasync d4/d12/d17/d2e/f47 0 2024-09-21T04:15:17.361 INFO:tasks.workunit.client.0.smithi110.stdout:6/500: fdatasync d4/d12/f34 0 2024-09-21T04:15:17.365 INFO:tasks.workunit.client.1.smithi148.stdout:9/459: rmdir d5/d1e/d37/d7b 0 2024-09-21T04:15:17.366 INFO:tasks.workunit.client.0.smithi110.stdout:2/300: mkdir d2/d50 0 2024-09-21T04:15:17.367 INFO:tasks.workunit.client.1.smithi148.stdout:5/438: mkdir d3/d14/d1f/d26/d2e/d3d/da0 0 2024-09-21T04:15:17.369 INFO:tasks.workunit.client.1.smithi148.stdout:6/462: dwrite d0/d1a/d23/d16/f3f [0,4194304] 0 2024-09-21T04:15:17.371 INFO:tasks.workunit.client.1.smithi148.stdout:1/420: dwrite db/d24/d2e/f7b [0,4194304] 0 2024-09-21T04:15:17.371 INFO:tasks.workunit.client.1.smithi148.stdout:1/421: truncate db/d46/d4c/f5b 969798 0 2024-09-21T04:15:17.373 INFO:tasks.workunit.client.1.smithi148.stdout:7/471: creat d5/d8/d6f/d7d/d8a/fa1 x:0 0 0 2024-09-21T04:15:17.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/479: mkdir dc/d13/d76/d5d/d8f 0 2024-09-21T04:15:17.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/480: write dc/d13/d3c/d81/d85/f87 [533501,33693] 0 2024-09-21T04:15:17.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/481: dread - dc/d13/d3c/f88 zero size 2024-09-21T04:15:17.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/482: chown dc/l25 355751 1 2024-09-21T04:15:17.380 INFO:tasks.workunit.client.0.smithi110.stdout:6/501: unlink d4/fc 0 2024-09-21T04:15:17.381 INFO:tasks.workunit.client.0.smithi110.stdout:6/502: chown d4/d12/d16/d6d/l85 637643 1 2024-09-21T04:15:17.381 INFO:tasks.workunit.client.1.smithi148.stdout:8/483: read dc/d43/f47 [778096,66553] 0 2024-09-21T04:15:17.381 INFO:tasks.workunit.client.1.smithi148.stdout:8/484: chown f8 5291 1 2024-09-21T04:15:17.383 INFO:tasks.workunit.client.0.smithi110.stdout:2/301: rename d2/d1a/d1f/d2a/d41/f49 to d2/d1a/d1f/d3f/d31/f51 0 2024-09-21T04:15:17.385 INFO:tasks.workunit.client.0.smithi110.stdout:7/453: symlink dd/d1c/d39/d4a/d52/l8d 0 2024-09-21T04:15:17.385 INFO:tasks.workunit.client.0.smithi110.stdout:7/454: creat dd/d48/d55/f8e x:0 0 0 2024-09-21T04:15:17.385 INFO:tasks.workunit.client.0.smithi110.stdout:7/455: truncate dd/d48/f4b 359166 0 2024-09-21T04:15:17.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/503: mkdir d4/d12/d16/d66/d7a/d9e 0 2024-09-21T04:15:17.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/504: dread - d4/f8c zero size 2024-09-21T04:15:17.388 INFO:tasks.workunit.client.1.smithi148.stdout:2/560: link d0/d9/d16/d55/l86 d0/d2b/da3/d8f/laa 0 2024-09-21T04:15:17.388 INFO:tasks.workunit.client.1.smithi148.stdout:2/561: fdatasync d0/d9/f29 0 2024-09-21T04:15:17.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/302: creat d2/d50/f52 x:0 0 0 2024-09-21T04:15:17.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/460: mknod d5/d60/cb1 0 2024-09-21T04:15:17.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/461: chown d5/d1e/d37/f64 528151506 1 2024-09-21T04:15:17.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/462: chown d5/d1e/d37/c8b 16655 1 2024-09-21T04:15:17.390 INFO:tasks.workunit.client.1.smithi148.stdout:2/562: dread d0/d2b/f5e [4194304,4194304] 0 2024-09-21T04:15:17.393 INFO:tasks.workunit.client.1.smithi148.stdout:6/463: chown d0/d1a/d23/d27/d62 10049 1 2024-09-21T04:15:17.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/456: rmdir dd/d1c/d7d 0 2024-09-21T04:15:17.396 INFO:tasks.workunit.client.0.smithi110.stdout:6/505: unlink d4/d12/d17/d33/c74 0 2024-09-21T04:15:17.396 INFO:tasks.workunit.client.0.smithi110.stdout:6/506: chown d4/d12/d17/d2e/d38 3395 1 2024-09-21T04:15:17.396 INFO:tasks.workunit.client.1.smithi148.stdout:9/463: read d5/d60/f3c [894903,27073] 0 2024-09-21T04:15:17.398 INFO:tasks.workunit.client.0.smithi110.stdout:2/303: rename d2/d1a/d1f/d3f/f3a to d2/d1a/d1f/d3f/d31/f53 0 2024-09-21T04:15:17.400 INFO:tasks.workunit.client.0.smithi110.stdout:7/457: symlink dd/d4d/l8f 0 2024-09-21T04:15:17.402 INFO:tasks.workunit.client.0.smithi110.stdout:6/507: link d4/d12/d16/d66/d7a/d4b/f8d d4/d12/d16/d66/d7a/d83/f9f 0 2024-09-21T04:15:17.402 INFO:tasks.workunit.client.0.smithi110.stdout:6/508: fdatasync d4/f6c 0 2024-09-21T04:15:17.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/304: dread d2/d1a/d1f/d3f/f24 [0,4194304] 0 2024-09-21T04:15:17.403 INFO:tasks.workunit.client.1.smithi148.stdout:3/427: sync 2024-09-21T04:15:17.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/465: sync 2024-09-21T04:15:17.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/466: chown d5/d6/d7/d11/f38 11 1 2024-09-21T04:15:17.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/422: mkdir db/dd/d21/d89 0 2024-09-21T04:15:17.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/458: creat dd/f90 x:0 0 0 2024-09-21T04:15:17.406 INFO:tasks.workunit.client.1.smithi148.stdout:7/472: creat d5/d14/d25/fa2 x:0 0 0 2024-09-21T04:15:17.407 INFO:tasks.workunit.client.0.smithi110.stdout:4/639: sync 2024-09-21T04:15:17.408 INFO:tasks.workunit.client.1.smithi148.stdout:8/485: creat dc/d13/d76/d42/f90 x:0 0 0 2024-09-21T04:15:17.410 INFO:tasks.workunit.client.1.smithi148.stdout:2/563: creat d0/d2b/d48/fab x:0 0 0 2024-09-21T04:15:17.411 INFO:tasks.workunit.client.1.smithi148.stdout:9/464: symlink d5/d1e/lb2 0 2024-09-21T04:15:17.413 INFO:tasks.workunit.client.1.smithi148.stdout:1/423: write db/dd/d21/f5c [1478921,78206] 0 2024-09-21T04:15:17.414 INFO:tasks.workunit.client.1.smithi148.stdout:3/428: unlink d3/d1f/d4d/d82/d52/c22 0 2024-09-21T04:15:17.416 INFO:tasks.workunit.client.1.smithi148.stdout:7/473: dread d5/d14/d25/d40/d7b/f84 [0,4194304] 0 2024-09-21T04:15:17.416 INFO:tasks.workunit.client.1.smithi148.stdout:2/564: mknod d0/d9/cac 0 2024-09-21T04:15:17.416 INFO:tasks.workunit.client.1.smithi148.stdout:6/464: unlink d0/d1a/d23/d32/l42 0 2024-09-21T04:15:17.417 INFO:tasks.workunit.client.0.smithi110.stdout:6/509: creat d4/d12/d17/d81/fa0 x:0 0 0 2024-09-21T04:15:17.419 INFO:tasks.workunit.client.0.smithi110.stdout:2/305: rename d2/d1a/d1f/d29/f4b to d2/f54 0 2024-09-21T04:15:17.419 INFO:tasks.workunit.client.0.smithi110.stdout:2/306: write d2/d1a/f20 [72903,121482] 0 2024-09-21T04:15:17.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/640: rmdir d17/d3e 39 2024-09-21T04:15:17.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/641: chown d17/d1e/fbf 25718845 1 2024-09-21T04:15:17.420 INFO:tasks.workunit.client.1.smithi148.stdout:9/465: dread d5/d1f/f4b [0,4194304] 0 2024-09-21T04:15:17.422 INFO:tasks.workunit.client.0.smithi110.stdout:6/510: rename d4/d12/d17/d33/d3e/f42 to d4/d12/d16/d66/d7a/d4b/fa1 0 2024-09-21T04:15:17.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/467: getdents d5/d6/d2a/d3f 0 2024-09-21T04:15:17.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/468: truncate d5/d6/d3e/f86 855775 0 2024-09-21T04:15:17.424 INFO:tasks.workunit.client.0.smithi110.stdout:2/307: rmdir d2 39 2024-09-21T04:15:17.424 INFO:tasks.workunit.client.1.smithi148.stdout:0/469: write d5/d6/d2a/d3f/f41 [178672,15284] 0 2024-09-21T04:15:17.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/470: getdents d5/d6/d3e/d1e 0 2024-09-21T04:15:17.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/471: chown d5/d6/d2a/d45/d70/f75 6 1 2024-09-21T04:15:17.425 INFO:tasks.workunit.client.0.smithi110.stdout:2/308: read d2/d1a/d1f/d3f/d31/f53 [775307,67932] 0 2024-09-21T04:15:17.427 INFO:tasks.workunit.client.0.smithi110.stdout:4/642: truncate d17/d1e/f79 1039386 0 2024-09-21T04:15:17.429 INFO:tasks.workunit.client.0.smithi110.stdout:6/511: rmdir d4/d3c 39 2024-09-21T04:15:17.429 INFO:tasks.workunit.client.0.smithi110.stdout:6/512: chown d4/d12/d17/c26 336 1 2024-09-21T04:15:17.429 INFO:tasks.workunit.client.0.smithi110.stdout:6/513: readlink d4/d12/d17/d2e/d40/l3b 0 2024-09-21T04:15:17.431 INFO:tasks.workunit.client.1.smithi148.stdout:6/465: mknod d0/d59/c84 0 2024-09-21T04:15:17.432 INFO:tasks.workunit.client.0.smithi110.stdout:4/643: write d17/d1e/d7b/fb8 [592602,122834] 0 2024-09-21T04:15:17.432 INFO:tasks.workunit.client.0.smithi110.stdout:4/644: chown fa 3 1 2024-09-21T04:15:17.432 INFO:tasks.workunit.client.0.smithi110.stdout:4/645: stat d17/d1e/d21/f5b 0 2024-09-21T04:15:17.435 INFO:tasks.workunit.client.0.smithi110.stdout:6/514: creat d4/d12/d17/d9a/fa2 x:0 0 0 2024-09-21T04:15:17.437 INFO:tasks.workunit.client.1.smithi148.stdout:6/466: mkdir d0/d8/d85 0 2024-09-21T04:15:17.437 INFO:tasks.workunit.client.1.smithi148.stdout:9/466: getdents d5/d2b/d32/d67/d80 0 2024-09-21T04:15:17.448 INFO:tasks.workunit.client.1.smithi148.stdout:8/486: read dc/f14 [2514536,75356] 0 2024-09-21T04:15:17.458 INFO:tasks.workunit.client.0.smithi110.stdout:7/459: dwrite dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:17.475 INFO:tasks.workunit.client.0.smithi110.stdout:8/527: sync 2024-09-21T04:15:17.475 INFO:tasks.workunit.client.0.smithi110.stdout:0/740: sync 2024-09-21T04:15:17.475 INFO:tasks.workunit.client.0.smithi110.stdout:3/397: sync 2024-09-21T04:15:17.475 INFO:tasks.workunit.client.0.smithi110.stdout:5/540: sync 2024-09-21T04:15:17.475 INFO:tasks.workunit.client.0.smithi110.stdout:9/407: sync 2024-09-21T04:15:17.476 INFO:tasks.workunit.client.0.smithi110.stdout:1/772: sync 2024-09-21T04:15:17.476 INFO:tasks.workunit.client.0.smithi110.stdout:1/773: chown d2/d9/l20 186 1 2024-09-21T04:15:17.477 INFO:tasks.workunit.client.0.smithi110.stdout:6/515: rename d4/d12/d17/d33/f64 to d4/d12/d16/d66/d7a/d83/fa3 0 2024-09-21T04:15:17.480 INFO:tasks.workunit.client.1.smithi148.stdout:3/429: rename d3/d5/d6/d12/d76 to d3/d84 0 2024-09-21T04:15:17.487 INFO:tasks.workunit.client.0.smithi110.stdout:7/460: creat dd/d48/d55/d7a/f91 x:0 0 0 2024-09-21T04:15:17.487 INFO:tasks.workunit.client.0.smithi110.stdout:3/398: mknod d0/d12/d2a/d80/c90 0 2024-09-21T04:15:17.487 INFO:tasks.workunit.client.0.smithi110.stdout:7/461: chown dd/d1c/d39/d4a/l79 44260606 1 2024-09-21T04:15:17.487 INFO:tasks.workunit.client.0.smithi110.stdout:0/741: dread d4/d47/d2c/d3a/f51 [0,4194304] 0 2024-09-21T04:15:17.488 INFO:tasks.workunit.client.1.smithi148.stdout:9/467: mkdir d5/d2b/d4e/db3 0 2024-09-21T04:15:17.488 INFO:tasks.workunit.client.1.smithi148.stdout:9/468: truncate d5/d60/d28/d2f/f62 722788 0 2024-09-21T04:15:17.489 INFO:tasks.workunit.client.1.smithi148.stdout:8/487: dwrite dc/d13/d76/d42/f53 [0,4194304] 0 2024-09-21T04:15:17.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/474: rename d5/d14/d25/d40/d7b/f82 to d5/d14/d25/d79/d86/fa3 0 2024-09-21T04:15:17.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/475: write d5/d8/f91 [233667,51884] 0 2024-09-21T04:15:17.491 INFO:tasks.workunit.client.0.smithi110.stdout:8/528: dread d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:17.491 INFO:tasks.workunit.client.0.smithi110.stdout:8/529: chown d0/d7/d13/d4b/f7d 583037595 1 2024-09-21T04:15:17.492 INFO:tasks.workunit.client.0.smithi110.stdout:5/541: creat d1/d49/d4d/d1e/d5a/d64/d7b/fb5 x:0 0 0 2024-09-21T04:15:17.494 INFO:tasks.workunit.client.0.smithi110.stdout:9/408: creat d7/d10/d24/d3c/d11/f8b x:0 0 0 2024-09-21T04:15:17.495 INFO:tasks.workunit.client.1.smithi148.stdout:2/565: write d0/f4 [1098455,35400] 0 2024-09-21T04:15:17.495 INFO:tasks.workunit.client.1.smithi148.stdout:6/467: unlink d0/d1a/d23/dc/d68/d5b/f74 0 2024-09-21T04:15:17.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/309: dwrite d2/f44 [0,4194304] 0 2024-09-21T04:15:17.497 INFO:tasks.workunit.client.0.smithi110.stdout:6/516: rename d4/d12/d17/c26 to d4/d12/d17/d33/ca4 0 2024-09-21T04:15:17.501 INFO:tasks.workunit.client.1.smithi148.stdout:1/424: dwrite db/dd/f13 [0,4194304] 0 2024-09-21T04:15:17.501 INFO:tasks.workunit.client.1.smithi148.stdout:0/472: dwrite d5/d6/d7/f4f [4194304,4194304] 0 2024-09-21T04:15:17.502 INFO:tasks.workunit.client.0.smithi110.stdout:7/462: symlink dd/d1c/d39/d6d/d80/l92 0 2024-09-21T04:15:17.503 INFO:tasks.workunit.client.1.smithi148.stdout:9/469: creat d5/d60/d4c/d5e/fb4 x:0 0 0 2024-09-21T04:15:17.503 INFO:tasks.workunit.client.1.smithi148.stdout:9/470: chown d5/d2b/d32/d67/c86 127 1 2024-09-21T04:15:17.503 INFO:tasks.workunit.client.0.smithi110.stdout:0/742: symlink d4/d1c/d62/d69/ldf 0 2024-09-21T04:15:17.503 INFO:tasks.workunit.client.0.smithi110.stdout:0/743: fsync d4/dd/d10/d29/f5d 0 2024-09-21T04:15:17.504 INFO:tasks.workunit.client.0.smithi110.stdout:1/774: dwrite d2/d9/d49/d5c/d91/fa5 [0,4194304] 0 2024-09-21T04:15:17.505 INFO:tasks.workunit.client.0.smithi110.stdout:1/775: write d2/d9/d49/fb1 [134114,103274] 0 2024-09-21T04:15:17.505 INFO:tasks.workunit.client.0.smithi110.stdout:3/399: dread d0/f1f [0,4194304] 0 2024-09-21T04:15:17.505 INFO:tasks.workunit.client.0.smithi110.stdout:5/542: mkdir d1/d22/db6 0 2024-09-21T04:15:17.506 INFO:tasks.workunit.client.0.smithi110.stdout:9/409: getdents d7/d10/d24/d3c/d1f/d72 0 2024-09-21T04:15:17.507 INFO:tasks.workunit.client.0.smithi110.stdout:9/410: truncate d7/d10/d24/d3c/d11/f8b 665163 0 2024-09-21T04:15:17.513 INFO:tasks.workunit.client.0.smithi110.stdout:7/463: rename dd/d14/d17/l40 to dd/d48/d55/l93 0 2024-09-21T04:15:17.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/411: dread d7/d10/d24/d3c/f20 [0,4194304] 0 2024-09-21T04:15:17.517 INFO:tasks.workunit.client.0.smithi110.stdout:9/412: creat d7/d10/d24/d3c/d11/f8c x:0 0 0 2024-09-21T04:15:17.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/744: mknod d4/dd/d10/d29/d5f/ce0 0 2024-09-21T04:15:17.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/745: chown d4/f57 3593 1 2024-09-21T04:15:17.518 INFO:tasks.workunit.client.0.smithi110.stdout:0/746: chown d4/dd/d10/d29/d5f/c82 35772 1 2024-09-21T04:15:17.534 INFO:tasks.workunit.client.0.smithi110.stdout:3/400: link d0/c78 d0/d51/c91 0 2024-09-21T04:15:17.537 INFO:tasks.workunit.client.0.smithi110.stdout:5/543: getdents d1/d39 0 2024-09-21T04:15:17.540 INFO:tasks.workunit.client.0.smithi110.stdout:3/401: dread d0/d12/d2a/f3e [0,4194304] 0 2024-09-21T04:15:17.541 INFO:tasks.workunit.client.1.smithi148.stdout:0/473: creat d5/d6/d3e/f95 x:0 0 0 2024-09-21T04:15:17.541 INFO:tasks.workunit.client.1.smithi148.stdout:0/474: stat d5/f85 0 2024-09-21T04:15:17.542 INFO:tasks.workunit.client.0.smithi110.stdout:7/464: creat dd/d1c/d39/d56/f94 x:0 0 0 2024-09-21T04:15:17.542 INFO:tasks.workunit.client.0.smithi110.stdout:7/465: chown dd/d4d/f59 6733 1 2024-09-21T04:15:17.543 INFO:tasks.workunit.client.1.smithi148.stdout:8/488: getdents dc/d3e 0 2024-09-21T04:15:17.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/566: getdents d0/d9/d38 0 2024-09-21T04:15:17.547 INFO:tasks.workunit.client.0.smithi110.stdout:9/413: symlink d7/d10/d24/d3c/d1f/d43/l8d 0 2024-09-21T04:15:17.547 INFO:tasks.workunit.client.0.smithi110.stdout:9/414: write d7/d10/d24/d3c/d35/f73 [800638,76757] 0 2024-09-21T04:15:17.547 INFO:tasks.workunit.client.0.smithi110.stdout:0/747: mknod d4/d1c/d62/d69/ce1 0 2024-09-21T04:15:17.550 INFO:tasks.workunit.client.1.smithi148.stdout:2/567: mkdir d0/d9/dad 0 2024-09-21T04:15:17.550 INFO:tasks.workunit.client.1.smithi148.stdout:2/568: stat d0/d9/f6e 0 2024-09-21T04:15:17.551 INFO:tasks.workunit.client.1.smithi148.stdout:1/425: write db/d24/f33 [339043,25729] 0 2024-09-21T04:15:17.551 INFO:tasks.workunit.client.0.smithi110.stdout:2/310: rmdir d2/d1a/d1f/d2a/d41 0 2024-09-21T04:15:17.552 INFO:tasks.workunit.client.1.smithi148.stdout:3/430: dwrite d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:17.552 INFO:tasks.workunit.client.0.smithi110.stdout:3/402: truncate d0/d12/d3c/d5a/f64 1124045 0 2024-09-21T04:15:17.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/475: dread d5/d6/d2a/d45/d70/f7f [0,4194304] 0 2024-09-21T04:15:17.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/482: sync 2024-09-21T04:15:17.557 INFO:tasks.workunit.client.0.smithi110.stdout:0/748: symlink d4/d1c/d62/d69/d9b/le2 0 2024-09-21T04:15:17.557 INFO:tasks.workunit.client.0.smithi110.stdout:0/749: creat d4/dd/fe3 x:0 0 0 2024-09-21T04:15:17.559 INFO:tasks.workunit.client.0.smithi110.stdout:5/544: mknod d1/d7/cb7 0 2024-09-21T04:15:17.559 INFO:tasks.workunit.client.0.smithi110.stdout:5/545: write d1/d7/fa1 [228374,119856] 0 2024-09-21T04:15:17.559 INFO:tasks.workunit.client.0.smithi110.stdout:5/546: creat d1/d49/d6f/fb8 x:0 0 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/471: dwrite d5/d1f/f24 [4194304,4194304] 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/472: fdatasync d5/d1e/d37/d46/f72 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/473: write d5/d60/d28/d2f/d8e/f99 [4507245,440] 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/474: fsync d5/d39/f98 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/475: fsync d5/d1e/d37/d46/fac 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/468: dwrite d0/d8/f24 [0,4194304] 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/469: fdatasync d0/d34/d38/d43/d63/f77 0 2024-09-21T04:15:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/476: dwrite d5/d8/d6f/d7d/d90/d28/f29 [0,4194304] 0 2024-09-21T04:15:17.561 INFO:tasks.workunit.client.0.smithi110.stdout:8/530: dwrite d0/d7/d13/d4b/f67 [0,4194304] 0 2024-09-21T04:15:17.561 INFO:tasks.workunit.client.1.smithi148.stdout:4/483: unlink d4/d12/c81 0 2024-09-21T04:15:17.561 INFO:tasks.workunit.client.1.smithi148.stdout:4/484: write d4/d87/d2d/f76 [148035,35415] 0 2024-09-21T04:15:17.562 INFO:tasks.workunit.client.0.smithi110.stdout:5/547: dread d1/d39/fa4 [0,4194304] 0 2024-09-21T04:15:17.563 INFO:tasks.workunit.client.0.smithi110.stdout:2/311: symlink d2/d1a/d1f/d3f/l55 0 2024-09-21T04:15:17.566 INFO:tasks.workunit.client.0.smithi110.stdout:6/517: dwrite d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:17.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/776: dwrite d2/db7/fcd [0,4194304] 0 2024-09-21T04:15:17.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/777: chown d2/d9/d26/f8b 1719 1 2024-09-21T04:15:17.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/778: chown d2/d9/d23/cc2 22611912 1 2024-09-21T04:15:17.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/470: write d0/d1a/d23/d16/f1f [649064,106714] 0 2024-09-21T04:15:17.569 INFO:tasks.workunit.client.1.smithi148.stdout:9/476: mknod d5/d1f/d81/d9e/da8/cb5 0 2024-09-21T04:15:17.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/403: symlink d0/d12/d3c/d5a/l92 0 2024-09-21T04:15:17.571 INFO:tasks.workunit.client.0.smithi110.stdout:9/415: rmdir d7/d26 39 2024-09-21T04:15:17.574 INFO:tasks.workunit.client.1.smithi148.stdout:0/476: dwrite d5/d6/d7/d11/f71 [0,4194304] 0 2024-09-21T04:15:17.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/477: dread - d5/f85 zero size 2024-09-21T04:15:17.575 INFO:tasks.workunit.client.0.smithi110.stdout:8/531: dread d0/d30/d54/f98 [0,4194304] 0 2024-09-21T04:15:17.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/548: write d1/d7/d18/f74 [456889,47504] 0 2024-09-21T04:15:17.575 INFO:tasks.workunit.client.0.smithi110.stdout:0/750: creat d4/dd/d10/d29/fe4 x:0 0 0 2024-09-21T04:15:17.582 INFO:tasks.workunit.client.0.smithi110.stdout:3/404: mknod d0/d12/d3c/c93 0 2024-09-21T04:15:17.582 INFO:tasks.workunit.client.0.smithi110.stdout:2/312: rename d2/d1a/d1f/d3f/d31 to d2/d50/d56 0 2024-09-21T04:15:17.582 INFO:tasks.workunit.client.0.smithi110.stdout:2/313: fsync d2/d1a/d1f/d3f/f24 0 2024-09-21T04:15:17.583 INFO:tasks.workunit.client.0.smithi110.stdout:3/405: write d0/d12/f82 [899968,114193] 0 2024-09-21T04:15:17.583 INFO:tasks.workunit.client.0.smithi110.stdout:3/406: truncate d0/f77 341699 0 2024-09-21T04:15:17.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/416: mkdir d7/d10/d24/d3c/d14/d5c/d6f/d78/d8e 0 2024-09-21T04:15:17.586 INFO:tasks.workunit.client.0.smithi110.stdout:8/532: creat d0/d7/d13/d1c/d41/fba x:0 0 0 2024-09-21T04:15:17.586 INFO:tasks.workunit.client.0.smithi110.stdout:8/533: write d0/d30/f31 [5118765,39627] 0 2024-09-21T04:15:17.588 INFO:tasks.workunit.client.1.smithi148.stdout:4/485: creat d4/d12/d16/d29/d61/d6c/f88 x:0 0 0 2024-09-21T04:15:17.588 INFO:tasks.workunit.client.1.smithi148.stdout:4/486: truncate d4/d12/f63 1042854 0 2024-09-21T04:15:17.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/549: unlink d1/dc/c66 0 2024-09-21T04:15:17.589 INFO:tasks.workunit.client.0.smithi110.stdout:2/314: mknod d2/d50/d56/c57 0 2024-09-21T04:15:17.590 INFO:tasks.workunit.client.0.smithi110.stdout:2/315: stat d2/l23 0 2024-09-21T04:15:17.591 INFO:tasks.workunit.client.1.smithi148.stdout:6/471: rename d0/d1a/d23/d16/d4f/f54 to d0/d1a/d23/dc/d68/f86 0 2024-09-21T04:15:17.594 INFO:tasks.workunit.client.1.smithi148.stdout:2/569: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:15:17.594 INFO:tasks.workunit.client.1.smithi148.stdout:2/570: chown d0/d9/d16/d55/d71 450461 1 2024-09-21T04:15:17.594 INFO:tasks.workunit.client.1.smithi148.stdout:2/571: write d0/f14 [1013892,28456] 0 2024-09-21T04:15:17.594 INFO:tasks.workunit.client.1.smithi148.stdout:9/477: mknod d5/d2b/d32/cb6 0 2024-09-21T04:15:17.595 INFO:tasks.workunit.client.0.smithi110.stdout:3/407: mknod d0/d12/d2a/c94 0 2024-09-21T04:15:17.595 INFO:tasks.workunit.client.0.smithi110.stdout:3/408: write d0/d22/f72 [953638,35469] 0 2024-09-21T04:15:17.598 INFO:tasks.workunit.client.0.smithi110.stdout:9/417: mkdir d7/d10/d88/d8f 0 2024-09-21T04:15:17.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/478: mkdir d5/d6/d2a/d96 0 2024-09-21T04:15:17.601 INFO:tasks.workunit.client.0.smithi110.stdout:5/550: creat d1/d22/fb9 x:0 0 0 2024-09-21T04:15:17.602 INFO:tasks.workunit.client.0.smithi110.stdout:5/551: dread d1/d7/d58/f77 [0,4194304] 0 2024-09-21T04:15:17.604 INFO:tasks.workunit.client.0.smithi110.stdout:2/316: rename d2/d1a/d1f/d3f/f43 to d2/d1a/d1f/d3f/f58 0 2024-09-21T04:15:17.605 INFO:tasks.workunit.client.1.smithi148.stdout:5/439: sync 2024-09-21T04:15:17.606 INFO:tasks.workunit.client.1.smithi148.stdout:5/440: write d3/d14/f44 [1373076,79453] 0 2024-09-21T04:15:17.606 INFO:tasks.workunit.client.1.smithi148.stdout:5/441: fdatasync d3/d14/d1a/f4f 0 2024-09-21T04:15:17.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.604+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 25) v1 ==== 51991+0+0 (secure 0 0 0) 0x7f8410011b40 con 0x7f841c1085c0 2024-09-21T04:15:17.608 INFO:tasks.workunit.client.0.smithi110.stdout:1/779: rename d2/d9/d49/d5c/le2 to d2/d9/d23/lf7 0 2024-09-21T04:15:17.609 INFO:tasks.workunit.client.1.smithi148.stdout:7/477: rmdir d5/d8/d6f 39 2024-09-21T04:15:17.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/418: creat d7/d26/f90 x:0 0 0 2024-09-21T04:15:17.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/419: stat d7/d10/d24/d3c/d1f/d68/f7c 0 2024-09-21T04:15:17.611 INFO:tasks.workunit.client.1.smithi148.stdout:4/487: rmdir d4/d12/d16/d29 39 2024-09-21T04:15:17.612 INFO:tasks.workunit.client.0.smithi110.stdout:5/552: read d1/d7/d58/f82 [1544563,80934] 0 2024-09-21T04:15:17.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/553: write d1/d7/f13 [71307,112547] 0 2024-09-21T04:15:17.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/554: stat d1/d49/d4d/d1e/d5a/d64 0 2024-09-21T04:15:17.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/555: write d1/f8a [1654399,112212] 0 2024-09-21T04:15:17.614 INFO:tasks.workunit.client.0.smithi110.stdout:7/466: dwrite dd/f1a [0,4194304] 0 2024-09-21T04:15:17.616 INFO:tasks.workunit.client.0.smithi110.stdout:1/780: truncate d2/db7/d86/f89 3295106 0 2024-09-21T04:15:17.617 INFO:tasks.workunit.client.1.smithi148.stdout:7/478: write f4 [4623285,49869] 0 2024-09-21T04:15:17.617 INFO:tasks.workunit.client.1.smithi148.stdout:7/479: write d5/d8/d6f/f9d [412425,127042] 0 2024-09-21T04:15:17.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/572: rename d0/d9/dad to d0/d2b/da3/d82/dae 0 2024-09-21T04:15:17.618 INFO:tasks.workunit.client.0.smithi110.stdout:2/317: dread d2/f16 [0,4194304] 0 2024-09-21T04:15:17.619 INFO:tasks.workunit.client.0.smithi110.stdout:9/420: stat d7/d10/d24/d3c/d1f/d72/d59/c5f 0 2024-09-21T04:15:17.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/479: mknod d5/d6/d2a/c97 0 2024-09-21T04:15:17.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/556: symlink d1/d22/lba 0 2024-09-21T04:15:17.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/442: creat d3/d34/d78/fa1 x:0 0 0 2024-09-21T04:15:17.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/488: creat d4/d87/f89 x:0 0 0 2024-09-21T04:15:17.628 INFO:tasks.workunit.client.1.smithi148.stdout:6/472: rmdir d0/d1a/d23/d16/d36 39 2024-09-21T04:15:17.628 INFO:tasks.workunit.client.1.smithi148.stdout:6/473: chown d0/d8/c7e 1 1 2024-09-21T04:15:17.629 INFO:tasks.workunit.client.0.smithi110.stdout:0/751: rename d4/d47/d2c/d3a/d84 to d4/dd/d60/de5 0 2024-09-21T04:15:17.629 INFO:tasks.workunit.client.0.smithi110.stdout:0/752: stat d4/d47/l63 0 2024-09-21T04:15:17.630 INFO:tasks.workunit.client.1.smithi148.stdout:9/478: read d5/d60/d28/f30 [1409191,125104] 0 2024-09-21T04:15:17.632 INFO:tasks.workunit.client.1.smithi148.stdout:2/573: write d0/d9/dc/d13/d25/f2a [163089,59923] 0 2024-09-21T04:15:17.633 INFO:tasks.workunit.client.1.smithi148.stdout:2/574: chown d0/d9/d16/d55/l86 27227226 1 2024-09-21T04:15:17.634 INFO:tasks.workunit.client.0.smithi110.stdout:1/781: dread d2/d9/d49/d7a/f7c [0,4194304] 0 2024-09-21T04:15:17.636 INFO:tasks.workunit.client.1.smithi148.stdout:7/480: mknod d5/d8/d6f/d7d/d90/d9b/ca4 0 2024-09-21T04:15:17.636 INFO:tasks.workunit.client.1.smithi148.stdout:4/489: write d4/d66/d7f/d2a/f36 [1379894,49731] 0 2024-09-21T04:15:17.636 INFO:tasks.workunit.client.1.smithi148.stdout:1/426: dwrite db/dd/f23 [0,4194304] 0 2024-09-21T04:15:17.636 INFO:tasks.workunit.client.1.smithi148.stdout:1/427: write db/d24/d2a/f45 [49863,18715] 0 2024-09-21T04:15:17.637 INFO:tasks.workunit.client.0.smithi110.stdout:6/518: dwrite d4/d12/d16/f7e [4194304,4194304] 0 2024-09-21T04:15:17.637 INFO:tasks.workunit.client.0.smithi110.stdout:6/519: fdatasync d4/d12/d16/d66/d7a/d4b/fa1 0 2024-09-21T04:15:17.641 INFO:tasks.workunit.client.1.smithi148.stdout:0/480: creat d5/d6/d2a/d45/f98 x:0 0 0 2024-09-21T04:15:17.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/474: write d0/f21 [779380,81439] 0 2024-09-21T04:15:17.647 INFO:tasks.workunit.client.0.smithi110.stdout:5/557: mknod d1/d22/d6e/cbb 0 2024-09-21T04:15:17.647 INFO:tasks.workunit.client.0.smithi110.stdout:8/534: rename d0/d30/d34/d3e/d60/c79 to d0/d7/d23/d82/cbb 0 2024-09-21T04:15:17.647 INFO:tasks.workunit.client.0.smithi110.stdout:8/535: dread - d0/d30/d34/fa5 zero size 2024-09-21T04:15:17.648 INFO:tasks.workunit.client.0.smithi110.stdout:0/753: creat d4/d1c/fe6 x:0 0 0 2024-09-21T04:15:17.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/481: rmdir d5/d8/d6f/d7d/d90 39 2024-09-21T04:15:17.651 INFO:tasks.workunit.client.1.smithi148.stdout:4/490: creat d4/d12/d16/d26/f8a x:0 0 0 2024-09-21T04:15:17.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/782: symlink d2/df3/lf8 0 2024-09-21T04:15:17.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/783: fsync d2/d9/d49/fbe 0 2024-09-21T04:15:17.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/784: getdents d2/d9/d49/d7a/dee/de1 0 2024-09-21T04:15:17.654 INFO:tasks.workunit.client.1.smithi148.stdout:1/428: mknod db/d24/d4f/c8a 0 2024-09-21T04:15:17.655 INFO:tasks.workunit.client.0.smithi110.stdout:3/409: dwrite d0/d12/d16/d1a/f86 [0,4194304] 0 2024-09-21T04:15:17.656 INFO:tasks.workunit.client.1.smithi148.stdout:6/475: unlink d0/d1a/d23/d27/f2d 0 2024-09-21T04:15:17.658 INFO:tasks.workunit.client.1.smithi148.stdout:7/482: unlink f4 0 2024-09-21T04:15:17.659 INFO:tasks.workunit.client.0.smithi110.stdout:6/520: symlink d4/d3c/la5 0 2024-09-21T04:15:17.661 INFO:tasks.workunit.client.1.smithi148.stdout:4/491: link d4/d12/d16/d29/c47 d4/d66/d7f/c8b 0 2024-09-21T04:15:17.663 INFO:tasks.workunit.client.1.smithi148.stdout:1/429: creat db/d24/f8b x:0 0 0 2024-09-21T04:15:17.666 INFO:tasks.workunit.client.0.smithi110.stdout:5/558: dread d1/d49/d4d/d1e/d5a/f67 [0,4194304] 0 2024-09-21T04:15:17.667 INFO:tasks.workunit.client.1.smithi148.stdout:7/483: symlink d5/d14/d25/d79/la5 0 2024-09-21T04:15:17.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/492: rmdir d4/d12/d16 39 2024-09-21T04:15:17.669 INFO:tasks.workunit.client.0.smithi110.stdout:1/785: dread d2/d9/d49/fbe [0,4194304] 0 2024-09-21T04:15:17.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/484: stat d5/d8/d64/l66 0 2024-09-21T04:15:17.673 INFO:tasks.workunit.client.0.smithi110.stdout:7/467: dwrite dd/d1c/f38 [0,4194304] 0 2024-09-21T04:15:17.673 INFO:tasks.workunit.client.0.smithi110.stdout:7/468: chown dd/d4d/f71 1 1 2024-09-21T04:15:17.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/443: dwrite d3/f8 [0,4194304] 0 2024-09-21T04:15:17.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/444: creat d3/d14/d1f/d26/d2e/d3d/fa2 x:0 0 0 2024-09-21T04:15:17.674 INFO:tasks.workunit.client.0.smithi110.stdout:1/786: dread d2/d9/d49/d7a/f7c [0,4194304] 0 2024-09-21T04:15:17.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/430: dread db/d24/d2a/f45 [0,4194304] 0 2024-09-21T04:15:17.675 INFO:tasks.workunit.client.1.smithi148.stdout:4/493: creat d4/d12/d78/f8c x:0 0 0 2024-09-21T04:15:17.685 INFO:tasks.workunit.client.0.smithi110.stdout:9/421: rename d7/c9 to d7/d10/c91 0 2024-09-21T04:15:17.686 INFO:tasks.workunit.client.1.smithi148.stdout:2/575: dwrite d0/f1d [4194304,4194304] 0 2024-09-21T04:15:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:17 smithi148 ceph-mon[34957]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:17 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3767295695' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:15:17.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/479: dwrite d5/d1e/d37/f64 [0,4194304] 0 2024-09-21T04:15:17.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/480: dread - d5/d2b/d32/f58 zero size 2024-09-21T04:15:17.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/481: dread - d5/d60/d4c/d5e/fb4 zero size 2024-09-21T04:15:17.692 INFO:tasks.workunit.client.1.smithi148.stdout:9/482: fsync d5/d60/d28/d38/f8f 0 2024-09-21T04:15:17.693 INFO:tasks.workunit.client.1.smithi148.stdout:5/445: mknod d3/d34/d78/ca3 0 2024-09-21T04:15:17.693 INFO:tasks.workunit.client.0.smithi110.stdout:9/422: dread d7/d10/d24/d3c/d11/f2e [0,4194304] 0 2024-09-21T04:15:17.694 INFO:tasks.workunit.client.0.smithi110.stdout:9/423: chown d7/d10/d24/d3c/d1f/d72/d59/l65 62850936 1 2024-09-21T04:15:17.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/431: creat db/dd/f8c x:0 0 0 2024-09-21T04:15:17.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/432: creat db/d46/f8d x:0 0 0 2024-09-21T04:15:17.697 INFO:tasks.workunit.client.0.smithi110.stdout:6/521: creat d4/d12/d17/d33/d76/fa6 x:0 0 0 2024-09-21T04:15:17.699 INFO:tasks.workunit.client.1.smithi148.stdout:0/481: dwrite d5/d6/d7/d11/f17 [0,4194304] 0 2024-09-21T04:15:17.699 INFO:tasks.workunit.client.1.smithi148.stdout:0/482: write d5/d6/f63 [342232,14306] 0 2024-09-21T04:15:17.699 INFO:tasks.workunit.client.1.smithi148.stdout:4/494: symlink d4/d12/d16/d29/d61/l8d 0 2024-09-21T04:15:17.699 INFO:tasks.workunit.client.1.smithi148.stdout:4/495: stat d4/d12/d16/f4f 0 2024-09-21T04:15:17.700 INFO:tasks.workunit.client.0.smithi110.stdout:5/559: mknod d1/d22/d6e/cbc 0 2024-09-21T04:15:17.701 INFO:tasks.workunit.client.0.smithi110.stdout:8/536: dwrite d0/d7/d13/d1c/f1d [4194304,4194304] 0 2024-09-21T04:15:17.706 INFO:tasks.workunit.client.1.smithi148.stdout:6/476: dwrite d0/d1a/d23/f12 [0,4194304] 0 2024-09-21T04:15:17.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/496: read d4/d12/d16/d29/f3d [4192683,17125] 0 2024-09-21T04:15:17.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/497: truncate d4/d12/d78/f8c 33959 0 2024-09-21T04:15:17.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/498: fsync d4/d66/d7f/f2b 0 2024-09-21T04:15:17.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/499: readlink d4/d12/d16/d29/d61/l8d 0 2024-09-21T04:15:17.710 INFO:tasks.workunit.client.0.smithi110.stdout:7/469: creat dd/d1c/d39/f95 x:0 0 0 2024-09-21T04:15:17.713 INFO:tasks.workunit.client.1.smithi148.stdout:7/485: dwrite d5/d8/f91 [0,4194304] 0 2024-09-21T04:15:17.713 INFO:tasks.workunit.client.1.smithi148.stdout:7/486: read d5/d14/d25/d40/d7b/f84 [3282615,51429] 0 2024-09-21T04:15:17.715 INFO:tasks.workunit.client.0.smithi110.stdout:0/754: rename d4/d1c/d62/d91/l9c to d4/dd/d10/d29/d5f/dc3/le7 0 2024-09-21T04:15:17.717 INFO:tasks.workunit.client.0.smithi110.stdout:9/424: creat d7/d10/d24/d3c/d1f/d43/f92 x:0 0 0 2024-09-21T04:15:17.720 INFO:tasks.workunit.client.0.smithi110.stdout:1/787: dwrite d2/d9/d49/f99 [0,4194304] 0 2024-09-21T04:15:17.721 INFO:tasks.workunit.client.1.smithi148.stdout:5/446: dwrite d3/d14/d1a/f4f [0,4194304] 0 2024-09-21T04:15:17.722 INFO:tasks.workunit.client.0.smithi110.stdout:5/560: dwrite d1/f35 [0,4194304] 0 2024-09-21T04:15:17.722 INFO:tasks.workunit.client.0.smithi110.stdout:3/410: rmdir d0/d12/d3c/d5a 39 2024-09-21T04:15:17.723 INFO:tasks.workunit.client.0.smithi110.stdout:3/411: read d0/f77 [214985,63167] 0 2024-09-21T04:15:17.724 INFO:tasks.workunit.client.1.smithi148.stdout:2/576: mkdir d0/d9/dc/d13/daf 0 2024-09-21T04:15:17.724 INFO:tasks.workunit.client.1.smithi148.stdout:2/577: chown d0/d2b/da3/d8f/d9f 1 1 2024-09-21T04:15:17.724 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:17 smithi110 ceph-mon[27723]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:17.724 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:17 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3767295695' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:15:17.728 INFO:tasks.workunit.client.0.smithi110.stdout:6/522: symlink d4/d12/d17/d33/d76/la7 0 2024-09-21T04:15:17.728 INFO:tasks.workunit.client.0.smithi110.stdout:6/523: getdents d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99 0 2024-09-21T04:15:17.729 INFO:tasks.workunit.client.0.smithi110.stdout:8/537: truncate d0/d7/f20 143298 0 2024-09-21T04:15:17.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.728+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f841c1110d0 con 0x7f841c1085c0 2024-09-21T04:15:17.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.729+0000 7f8418ff9640 1 -- 172.21.15.110:0/163999997 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f8410011960 con 0x7f841c1085c0 2024-09-21T04:15:17.732 INFO:tasks.workunit.client.1.smithi148.stdout:5/447: write d3/d34/f8d [3044709,4324] 0 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:15:17.732 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:15:17.733 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:17.735 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.0.smithi110.stdout:7/470: symlink dd/d48/d55/d7a/l96 0 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.0.smithi110.stdout:5/561: write d1/d49/d4d/f46 [849020,27705] 0 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.0.smithi110.stdout:5/562: creat d1/d49/d6f/fbd x:0 0 0 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.0.smithi110.stdout:5/563: write d1/d49/d4d/f24 [4141929,82179] 0 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.0.smithi110.stdout:0/755: symlink d4/dd/d10/d8a/db5/le8 0 2024-09-21T04:15:17.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/483: rmdir d5/d1f/d81/d9e/da8 39 2024-09-21T04:15:17.737 INFO:tasks.workunit.client.0.smithi110.stdout:0/756: truncate d4/d47/f40 1580723 0 2024-09-21T04:15:17.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.733+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 msgr2=0x7f8404042010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.733+0000 7f8421ef1640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 0x7f8404042010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f840c000c00 tx=0x7f840c005db0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.733+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 msgr2=0x7f841c10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.733+0000 7f8421ef1640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f841000b420 tx=0x7f841000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 shutdown_connections 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f840403fb50 0x7f8404042010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f841c1085c0 0x7f841c10d9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 --2- 172.21.15.110:0/163999997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f841c071760 0x7f841c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 >> 172.21.15.110:0/163999997 conn(0x7f841c06cc80 msgr2=0x7f841c10b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 shutdown_connections 2024-09-21T04:15:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.734+0000 7f8421ef1640 1 -- 172.21.15.110:0/163999997 wait complete. 2024-09-21T04:15:17.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/448: dread d3/f58 [0,4194304] 0 2024-09-21T04:15:17.742 INFO:tasks.workunit.client.0.smithi110.stdout:9/425: link d7/d10/d24/d3c/d1f/c46 d7/d10/d24/d3c/d14/d5c/d6f/d78/d7d/c93 0 2024-09-21T04:15:17.742 INFO:tasks.workunit.client.0.smithi110.stdout:9/426: chown d7/d10/d24/d3c/d1f/d43/f6e 143 1 2024-09-21T04:15:17.743 INFO:tasks.workunit.client.0.smithi110.stdout:7/471: dread dd/d1c/d39/f3b [0,4194304] 0 2024-09-21T04:15:17.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/433: creat db/d24/d2e/d71/f8e x:0 0 0 2024-09-21T04:15:17.745 INFO:tasks.workunit.client.1.smithi148.stdout:9/484: dread d5/d60/f3f [0,4194304] 0 2024-09-21T04:15:17.745 INFO:tasks.workunit.client.0.smithi110.stdout:3/412: dwrite d0/d12/d3c/f5e [4194304,4194304] 0 2024-09-21T04:15:17.749 INFO:tasks.workunit.client.0.smithi110.stdout:6/524: dread d4/fb [0,4194304] 0 2024-09-21T04:15:17.751 INFO:tasks.workunit.client.0.smithi110.stdout:1/788: rename d2/d9/d26/d3d/f6f to d2/d9/d49/d4f/ff9 0 2024-09-21T04:15:17.755 INFO:tasks.workunit.client.1.smithi148.stdout:0/483: rmdir d5/d6/d7/d11/d48 39 2024-09-21T04:15:17.761 INFO:tasks.workunit.client.1.smithi148.stdout:6/477: mknod d0/d34/d38/d43/d63/c87 0 2024-09-21T04:15:17.764 INFO:tasks.workunit.client.1.smithi148.stdout:4/500: creat d4/d12/d16/d29/d61/f8e x:0 0 0 2024-09-21T04:15:17.764 INFO:tasks.workunit.client.1.smithi148.stdout:4/501: write d4/d3a/f53 [117765,56096] 0 2024-09-21T04:15:17.764 INFO:tasks.workunit.client.1.smithi148.stdout:4/502: dread - d4/d12/d75/f7a zero size 2024-09-21T04:15:17.764 INFO:tasks.workunit.client.1.smithi148.stdout:4/503: stat d4/d12/d16/l7d 0 2024-09-21T04:15:17.765 INFO:tasks.workunit.client.0.smithi110.stdout:8/538: creat d0/d7/d13/fbc x:0 0 0 2024-09-21T04:15:17.770 INFO:tasks.workunit.client.1.smithi148.stdout:6/478: write d0/d8/d18/f25 [3336971,57443] 0 2024-09-21T04:15:17.773 INFO:tasks.workunit.client.0.smithi110.stdout:0/757: creat d4/dd/d10/d29/d5f/dc3/fe9 x:0 0 0 2024-09-21T04:15:17.774 INFO:tasks.workunit.client.0.smithi110.stdout:5/564: dwrite d1/d7/f4e [0,4194304] 0 2024-09-21T04:15:17.774 INFO:tasks.workunit.client.1.smithi148.stdout:6/479: dread d0/d8/f49 [0,4194304] 0 2024-09-21T04:15:17.778 INFO:tasks.workunit.client.0.smithi110.stdout:9/427: creat d7/d10/d24/d3c/f94 x:0 0 0 2024-09-21T04:15:17.778 INFO:tasks.workunit.client.0.smithi110.stdout:9/428: truncate d7/d10/d24/d3c/d1f/f41 662911 0 2024-09-21T04:15:17.778 INFO:tasks.workunit.client.0.smithi110.stdout:9/429: dread d7/d10/d24/d3c/d1f/f41 [0,4194304] 0 2024-09-21T04:15:17.778 INFO:tasks.workunit.client.0.smithi110.stdout:9/430: getdents d7/d10/d24/d3c/d14 0 2024-09-21T04:15:17.779 INFO:tasks.workunit.client.1.smithi148.stdout:1/434: dwrite db/d46/f5f [0,4194304] 0 2024-09-21T04:15:17.779 INFO:tasks.workunit.client.0.smithi110.stdout:7/472: dwrite dd/d14/d17/f54 [0,4194304] 0 2024-09-21T04:15:17.779 INFO:tasks.workunit.client.0.smithi110.stdout:7/473: fsync dd/d48/d55/d7a/f6a 0 2024-09-21T04:15:17.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/485: dwrite d5/d60/d28/d38/f47 [0,4194304] 0 2024-09-21T04:15:17.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/486: truncate d5/d2b/d4e/fa3 491429 0 2024-09-21T04:15:17.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/487: chown d5/d1e/d37/d46/f7e 852044 1 2024-09-21T04:15:17.784 INFO:tasks.workunit.client.0.smithi110.stdout:3/413: unlink d0/l21 0 2024-09-21T04:15:17.789 INFO:tasks.workunit.client.0.smithi110.stdout:6/525: symlink d4/d12/la8 0 2024-09-21T04:15:17.789 INFO:tasks.workunit.client.0.smithi110.stdout:6/526: dread - d4/d12/d17/d81/fa0 zero size 2024-09-21T04:15:17.795 INFO:tasks.workunit.client.0.smithi110.stdout:8/539: mknod d0/d30/d34/d3e/d32/cbd 0 2024-09-21T04:15:17.795 INFO:tasks.workunit.client.0.smithi110.stdout:1/789: rename d2/db7/lc3 to d2/d9/d26/lfa 0 2024-09-21T04:15:17.808 INFO:tasks.workunit.client.0.smithi110.stdout:7/474: dwrite dd/d14/d17/f85 [0,4194304] 0 2024-09-21T04:15:17.808 INFO:tasks.workunit.client.1.smithi148.stdout:3/431: sync 2024-09-21T04:15:17.808 INFO:tasks.workunit.client.1.smithi148.stdout:8/489: sync 2024-09-21T04:15:17.808 INFO:tasks.workunit.client.0.smithi110.stdout:9/431: dwrite d7/d10/d24/d3c/f5a [0,4194304] 0 2024-09-21T04:15:17.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/484: getdents d5/d6/d7 0 2024-09-21T04:15:17.811 INFO:tasks.workunit.client.0.smithi110.stdout:3/414: mkdir d0/d12/d16/d1a/d95 0 2024-09-21T04:15:17.812 INFO:tasks.workunit.client.1.smithi148.stdout:4/504: rmdir d4 39 2024-09-21T04:15:17.816 INFO:tasks.workunit.client.1.smithi148.stdout:6/480: mknod d0/d1a/d23/d16/d4f/c88 0 2024-09-21T04:15:17.817 INFO:tasks.workunit.client.0.smithi110.stdout:8/540: mkdir d0/d7/d23/d52/d8b/dbe 0 2024-09-21T04:15:17.817 INFO:tasks.workunit.client.0.smithi110.stdout:8/541: chown d0/d7/d13/fb1 39499031 1 2024-09-21T04:15:17.817 INFO:tasks.workunit.client.0.smithi110.stdout:8/542: read - d0/d7/d6f/f93 zero size 2024-09-21T04:15:17.817 INFO:tasks.workunit.client.0.smithi110.stdout:8/543: read - d0/d30/d54/d6b/f81 zero size 2024-09-21T04:15:17.818 INFO:tasks.workunit.client.0.smithi110.stdout:0/758: dwrite d4/d47/d2c/d3a/f9f [0,4194304] 0 2024-09-21T04:15:17.818 INFO:tasks.workunit.client.0.smithi110.stdout:0/759: readlink d4/dd/lbe 0 2024-09-21T04:15:17.819 INFO:tasks.workunit.client.0.smithi110.stdout:1/790: getdents d2/d9/d23/d3b 0 2024-09-21T04:15:17.820 INFO:tasks.workunit.client.0.smithi110.stdout:5/565: getdents d1/d49/d4d/d1e/d5a/d64/d7b 0 2024-09-21T04:15:17.821 INFO:tasks.workunit.client.0.smithi110.stdout:7/475: write dd/f1a [3745294,50285] 0 2024-09-21T04:15:17.823 INFO:tasks.workunit.client.1.smithi148.stdout:1/435: mkdir db/d46/d8f 0 2024-09-21T04:15:17.825 INFO:tasks.workunit.client.0.smithi110.stdout:7/476: dread dd/d14/f20 [0,4194304] 0 2024-09-21T04:15:17.826 INFO:tasks.workunit.client.1.smithi148.stdout:9/488: rename d5/d2b/d4e/c56 to d5/d1e/d37/d50/cb7 0 2024-09-21T04:15:17.827 INFO:tasks.workunit.client.1.smithi148.stdout:6/481: dread d0/d8/d18/f4d [0,4194304] 0 2024-09-21T04:15:17.831 INFO:tasks.workunit.client.0.smithi110.stdout:3/415: creat d0/d12/d16/f96 x:0 0 0 2024-09-21T04:15:17.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/527: rmdir d4/d12/d16/d66 39 2024-09-21T04:15:17.837 INFO:tasks.workunit.client.0.smithi110.stdout:7/477: dread dd/f2e [0,4194304] 0 2024-09-21T04:15:17.837 INFO:tasks.workunit.client.0.smithi110.stdout:7/478: readlink dd/d1c/d39/d6d/d80/l92 0 2024-09-21T04:15:17.840 INFO:tasks.workunit.client.0.smithi110.stdout:8/544: mkdir d0/d30/d54/d6b/dbf 0 2024-09-21T04:15:17.840 INFO:tasks.workunit.client.0.smithi110.stdout:8/545: write d0/d7/d23/f33 [818025,128905] 0 2024-09-21T04:15:17.840 INFO:tasks.workunit.client.0.smithi110.stdout:6/528: dread d4/f7 [0,4194304] 0 2024-09-21T04:15:17.840 INFO:tasks.workunit.client.0.smithi110.stdout:6/529: stat d4/d12/d17/f9d 0 2024-09-21T04:15:17.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/489: dwrite d5/d1f/d81/f8c [0,4194304] 0 2024-09-21T04:15:17.841 INFO:tasks.workunit.client.1.smithi148.stdout:5/449: rmdir d3/d34 39 2024-09-21T04:15:17.841 INFO:tasks.workunit.client.1.smithi148.stdout:8/490: stat dc/d43/c64 0 2024-09-21T04:15:17.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/485: mkdir d5/d6/d2a/d2b/d99 0 2024-09-21T04:15:17.845 INFO:tasks.workunit.client.1.smithi148.stdout:4/505: creat d4/d66/f8f x:0 0 0 2024-09-21T04:15:17.848 INFO:tasks.workunit.client.1.smithi148.stdout:3/432: rename d3/d5/d6/d12/f66 to d3/f85 0 2024-09-21T04:15:17.848 INFO:tasks.workunit.client.1.smithi148.stdout:3/433: dread - d3/d5/d6/d12/d43/d54/f71 zero size 2024-09-21T04:15:17.849 INFO:tasks.workunit.client.1.smithi148.stdout:6/482: chown d0/d1a/d23/l31 344835522 1 2024-09-21T04:15:17.850 INFO:tasks.workunit.client.1.smithi148.stdout:7/487: link d5/d8/d6f/d7d/d90/l69 d5/d8/d64/la6 0 2024-09-21T04:15:17.853 INFO:tasks.workunit.client.1.smithi148.stdout:9/490: creat d5/fb8 x:0 0 0 2024-09-21T04:15:17.853 INFO:tasks.workunit.client.1.smithi148.stdout:8/491: dread dc/f68 [0,4194304] 0 2024-09-21T04:15:17.853 INFO:tasks.workunit.client.1.smithi148.stdout:0/486: write d5/d6/d7/f43 [2067770,36815] 0 2024-09-21T04:15:17.854 INFO:tasks.workunit.client.1.smithi148.stdout:5/450: mknod d3/d34/ca4 0 2024-09-21T04:15:17.856 INFO:tasks.workunit.client.0.smithi110.stdout:0/760: truncate d4/d47/fde 5171405 0 2024-09-21T04:15:17.864 INFO:tasks.workunit.client.0.smithi110.stdout:7/479: dwrite dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:17.864 INFO:tasks.workunit.client.0.smithi110.stdout:7/480: chown dd/d1c/d39/d4a/d52/c69 54 1 2024-09-21T04:15:17.864 INFO:tasks.workunit.client.0.smithi110.stdout:7/481: chown dd/f1a 0 1 2024-09-21T04:15:17.868 INFO:tasks.workunit.client.1.smithi148.stdout:3/434: dread d3/d1f/d4d/d82/d23/d28/d46/f5b [0,4194304] 0 2024-09-21T04:15:17.872 INFO:tasks.workunit.client.1.smithi148.stdout:4/506: mknod d4/d66/d7f/d2a/c90 0 2024-09-21T04:15:17.878 INFO:tasks.workunit.client.0.smithi110.stdout:1/791: mkdir d2/df3/dfb 0 2024-09-21T04:15:17.878 INFO:tasks.workunit.client.0.smithi110.stdout:1/792: stat d2/db7/l60 0 2024-09-21T04:15:17.878 INFO:tasks.workunit.client.0.smithi110.stdout:1/793: chown d2/d9/d49/d7a/dee/de1 123618519 1 2024-09-21T04:15:17.881 INFO:tasks.workunit.client.1.smithi148.stdout:8/492: write dc/d13/d3c/f5f [714537,109043] 0 2024-09-21T04:15:17.881 INFO:tasks.workunit.client.1.smithi148.stdout:1/436: rename db/dd/f83 to db/d46/d4c/d82/f90 0 2024-09-21T04:15:17.881 INFO:tasks.workunit.client.1.smithi148.stdout:1/437: dread - db/d24/d4f/f72 zero size 2024-09-21T04:15:17.881 INFO:tasks.workunit.client.1.smithi148.stdout:1/438: fdatasync db/d24/f8b 0 2024-09-21T04:15:17.884 INFO:tasks.workunit.client.0.smithi110.stdout:5/566: creat d1/d7/d58/d72/fbe x:0 0 0 2024-09-21T04:15:17.884 INFO:tasks.workunit.client.1.smithi148.stdout:7/488: creat d5/fa7 x:0 0 0 2024-09-21T04:15:17.886 INFO:tasks.workunit.client.0.smithi110.stdout:9/432: rmdir d7/d10/d24/d3c/d1f/d68 39 2024-09-21T04:15:17.890 INFO:tasks.workunit.client.0.smithi110.stdout:3/416: rename d0/d12/d16/f4a to d0/d12/d3c/d5a/d70/f97 0 2024-09-21T04:15:17.890 INFO:tasks.workunit.client.0.smithi110.stdout:9/433: dread d7/d10/d24/d3c/d1f/f23 [0,4194304] 0 2024-09-21T04:15:17.892 INFO:tasks.workunit.client.0.smithi110.stdout:1/794: dread d2/d9/d23/f82 [0,4194304] 0 2024-09-21T04:15:17.892 INFO:tasks.workunit.client.0.smithi110.stdout:1/795: fsync d2/d9/d26/f2d 0 2024-09-21T04:15:17.893 INFO:tasks.workunit.client.0.smithi110.stdout:7/482: dwrite dd/d1c/f34 [0,4194304] 0 2024-09-21T04:15:17.894 INFO:tasks.workunit.client.1.smithi148.stdout:9/491: mknod d5/d39/cb9 0 2024-09-21T04:15:17.895 INFO:tasks.workunit.client.0.smithi110.stdout:8/546: creat d0/d7/d6f/d86/fc0 x:0 0 0 2024-09-21T04:15:17.895 INFO:tasks.workunit.client.0.smithi110.stdout:8/547: chown d0/d7/f2a 120128547 1 2024-09-21T04:15:17.896 INFO:tasks.workunit.client.1.smithi148.stdout:1/439: read db/d46/f57 [1086560,55489] 0 2024-09-21T04:15:17.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.893+0000 7fd85ffff640 1 -- 172.21.15.110:0/4250074034 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864036610 con 0x7fd8680fe860 2024-09-21T04:15:17.897 INFO:tasks.workunit.client.0.smithi110.stdout:0/761: dwrite d4/d47/d2c/fce [0,4194304] 0 2024-09-21T04:15:17.897 INFO:tasks.workunit.client.1.smithi148.stdout:0/487: dwrite d5/d6/d2a/d45/d70/f7f [0,4194304] 0 2024-09-21T04:15:17.897 INFO:tasks.workunit.client.1.smithi148.stdout:0/488: write d5/d6/d2a/d2b/f35 [3915411,69083] 0 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 -- 172.21.15.110:0/4250074034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680fe860 msgr2=0x7fd8680fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/4250074034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680fe860 0x7fd8680fec60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd864010860 tx=0x7fd86402e360 comp rx=0 tx=0).stop 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 -- 172.21.15.110:0/4250074034 shutdown_connections 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/4250074034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680ffa60 0x7fd8680ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/4250074034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680fe860 0x7fd8680fec60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 -- 172.21.15.110:0/4250074034 >> 172.21.15.110:0/4250074034 conn(0x7fd8680fa010 msgr2=0x7fd8680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 -- 172.21.15.110:0/4250074034 shutdown_connections 2024-09-21T04:15:17.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.895+0000 7fd86e7d7640 1 -- 172.21.15.110:0/4250074034 wait complete. 2024-09-21T04:15:17.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/483: dwrite d0/d8/f29 [0,4194304] 0 2024-09-21T04:15:17.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.897+0000 7fd86e7d7640 1 Processor -- start 2024-09-21T04:15:17.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.897+0000 7fd86e7d7640 1 -- start start 2024-09-21T04:15:17.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.897+0000 7fd86e7d7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.901 INFO:tasks.workunit.client.0.smithi110.stdout:6/530: link d4/l1a d4/d12/d16/d66/d7a/d83/la9 0 2024-09-21T04:15:17.901 INFO:tasks.workunit.client.0.smithi110.stdout:6/531: fdatasync d4/f6 0 2024-09-21T04:15:17.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.898+0000 7fd86e7d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680ffa60 0x7fd868193740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.902 INFO:tasks.workunit.client.1.smithi148.stdout:5/451: symlink d3/d34/d3c/d5d/d75/la5 0 2024-09-21T04:15:17.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.898+0000 7fd86e7d7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868193d10 con 0x7fd8680fe860 2024-09-21T04:15:17.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.898+0000 7fd86e7d7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868193e80 con 0x7fd8680ffa60 2024-09-21T04:15:17.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.898+0000 7fd86d7d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.899+0000 7fd86d7d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48066/0 (socket says 172.21.15.110:48066) 2024-09-21T04:15:17.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.899+0000 7fd86d7d5640 1 -- 172.21.15.110:0/3797764758 learned_addr learned my addr 172.21.15.110:0/3797764758 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:17.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.899+0000 7fd86cfd4640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680ffa60 0x7fd868193740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.900+0000 7fd86d7d5640 1 -- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680ffa60 msgr2=0x7fd868193740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:17.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.900+0000 7fd86d7d5640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680ffa60 0x7fd868193740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:17.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.900+0000 7fd86d7d5640 1 -- 172.21.15.110:0/3797764758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864010510 con 0x7fd8680fe860 2024-09-21T04:15:17.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.901+0000 7fd86d7d5640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8640095c0 tx=0x7fd8640095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:17.904 INFO:tasks.workunit.client.1.smithi148.stdout:3/435: mknod d3/d5/d6/d12/d43/d54/c86 0 2024-09-21T04:15:17.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/507: truncate d4/d87/f57 723413 0 2024-09-21T04:15:17.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.903+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864036610 con 0x7fd8680fe860 2024-09-21T04:15:17.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.904+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd864004500 con 0x7fd8680fe860 2024-09-21T04:15:17.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.904+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8681988c0 con 0x7fd8680fe860 2024-09-21T04:15:17.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.904+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd868198e30 con 0x7fd8680fe860 2024-09-21T04:15:17.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.904+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86403f3f0 con 0x7fd8680fe860 2024-09-21T04:15:17.907 INFO:tasks.workunit.client.1.smithi148.stdout:8/493: link dc/d3e/c63 dc/d13/c91 0 2024-09-21T04:15:17.908 INFO:tasks.workunit.client.0.smithi110.stdout:8/548: write d0/d30/d54/f59 [3520174,14533] 0 2024-09-21T04:15:17.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/440: read db/d24/f25 [3872175,81415] 0 2024-09-21T04:15:17.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/441: fsync db/d46/f5f 0 2024-09-21T04:15:17.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/442: truncate db/dd/f13 9078608 0 2024-09-21T04:15:17.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.906+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 51991+0+0 (secure 0 0 0) 0x7fd86403d040 con 0x7fd8680fe860 2024-09-21T04:15:17.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.906+0000 7fd85e7fc640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 0x7fd8540420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:17.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.907+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd864079e00 con 0x7fd8680fe860 2024-09-21T04:15:17.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.907+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c005280 con 0x7fd8680fe860 2024-09-21T04:15:17.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.910+0000 7fd86cfd4640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 0x7fd8540420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:17.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.911+0000 7fd86cfd4640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 0x7fd8540420b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd868194720 tx=0x7fd85800b390 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:17.914 INFO:tasks.workunit.client.1.smithi148.stdout:6/484: dread d0/d8/f29 [0,4194304] 0 2024-09-21T04:15:17.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:17.913+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fd864043080 con 0x7fd8680fe860 2024-09-21T04:15:17.975 INFO:tasks.workunit.client.1.smithi148.stdout:7/489: truncate d5/d8/d6f/d7d/d90/d28/f29 4109716 0 2024-09-21T04:15:17.975 INFO:tasks.workunit.client.1.smithi148.stdout:7/490: write d5/d8/d6f/d7d/d90/f10 [1074899,114255] 0 2024-09-21T04:15:17.975 INFO:tasks.workunit.client.0.smithi110.stdout:2/318: sync 2024-09-21T04:15:17.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/319: chown d2/d50/d56 2876 1 2024-09-21T04:15:17.976 INFO:tasks.workunit.client.0.smithi110.stdout:4/646: sync 2024-09-21T04:15:17.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/492: mkdir d5/d39/dba 0 2024-09-21T04:15:17.978 INFO:tasks.workunit.client.0.smithi110.stdout:3/417: mknod d0/d12/d16/d1a/d95/c98 0 2024-09-21T04:15:17.979 INFO:tasks.workunit.client.1.smithi148.stdout:3/436: dwrite d3/d84/f7e [0,4194304] 0 2024-09-21T04:15:17.981 INFO:tasks.workunit.client.0.smithi110.stdout:1/796: rename d2/d9/d23/d4e/d8e/cd8 to d2/d9/d49/d7a/cfc 0 2024-09-21T04:15:17.985 INFO:tasks.workunit.client.0.smithi110.stdout:7/483: dwrite dd/d14/f2a [4194304,4194304] 0 2024-09-21T04:15:17.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/452: symlink d3/d14/d41/la6 0 2024-09-21T04:15:17.993 INFO:tasks.workunit.client.0.smithi110.stdout:0/762: symlink d4/dd/d53/d96/lea 0 2024-09-21T04:15:17.994 INFO:tasks.workunit.client.1.smithi148.stdout:8/494: mknod dc/c92 0 2024-09-21T04:15:17.994 INFO:tasks.workunit.client.0.smithi110.stdout:7/484: dread dd/d14/f41 [0,4194304] 0 2024-09-21T04:15:17.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/453: dread d3/d14/d1f/f23 [0,4194304] 0 2024-09-21T04:15:17.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/454: fdatasync d3/d34/f47 0 2024-09-21T04:15:17.996 INFO:tasks.workunit.client.1.smithi148.stdout:4/508: dwrite d4/d87/f89 [0,4194304] 0 2024-09-21T04:15:17.997 INFO:tasks.workunit.client.1.smithi148.stdout:0/489: dwrite d5/d6/d2a/d2b/f61 [0,4194304] 0 2024-09-21T04:15:17.998 INFO:tasks.workunit.client.0.smithi110.stdout:5/567: rmdir d1/d7/d58/d72 39 2024-09-21T04:15:17.999 INFO:tasks.workunit.client.0.smithi110.stdout:5/568: chown d1/d49/d4d/d1e/f2f 8419485 1 2024-09-21T04:15:18.001 INFO:tasks.workunit.client.1.smithi148.stdout:6/485: mkdir d0/d1a/d23/dc/d68/d5b/d89 0 2024-09-21T04:15:18.001 INFO:tasks.workunit.client.1.smithi148.stdout:6/486: readlink d0/d1a/d23/dc/l56 0 2024-09-21T04:15:18.003 INFO:tasks.workunit.client.1.smithi148.stdout:7/491: mknod d5/d14/d25/d40/d8b/ca8 0 2024-09-21T04:15:18.005 INFO:tasks.workunit.client.0.smithi110.stdout:8/549: mkdir d0/d7/d23/dc1 0 2024-09-21T04:15:18.008 INFO:tasks.workunit.client.1.smithi148.stdout:2/578: sync 2024-09-21T04:15:18.011 INFO:tasks.workunit.client.0.smithi110.stdout:2/320: link d2/d1a/d1f/d2a/l11 d2/d1a/d1f/d3f/l59 0 2024-09-21T04:15:18.014 INFO:tasks.workunit.client.0.smithi110.stdout:4/647: unlink d17/d1e/d7b/l83 0 2024-09-21T04:15:18.014 INFO:tasks.workunit.client.0.smithi110.stdout:4/648: stat d17/d34/d3d/d20/d43/d51/d52/d98 0 2024-09-21T04:15:18.015 INFO:tasks.workunit.client.1.smithi148.stdout:5/455: dwrite d3/d24/f2f [0,4194304] 0 2024-09-21T04:15:18.015 INFO:tasks.workunit.client.1.smithi148.stdout:5/456: chown d3/f36 2 1 2024-09-21T04:15:18.016 INFO:tasks.workunit.client.1.smithi148.stdout:3/437: rename d3/d1f/d4d/d82/d23/c7d to d3/d1f/c87 0 2024-09-21T04:15:18.016 INFO:tasks.workunit.client.1.smithi148.stdout:3/438: fsync d3/d5/f41 0 2024-09-21T04:15:18.016 INFO:tasks.workunit.client.1.smithi148.stdout:3/439: chown d3/d5/d6/d12/d15/c3c 46 1 2024-09-21T04:15:18.017 INFO:tasks.workunit.client.0.smithi110.stdout:3/418: creat d0/d12/f99 x:0 0 0 2024-09-21T04:15:18.020 INFO:tasks.workunit.client.0.smithi110.stdout:7/485: dwrite dd/f7e [0,4194304] 0 2024-09-21T04:15:18.020 INFO:tasks.workunit.client.1.smithi148.stdout:8/495: unlink dc/d13/f1c 0 2024-09-21T04:15:18.020 INFO:tasks.workunit.client.1.smithi148.stdout:8/496: creat dc/d13/d3c/f93 x:0 0 0 2024-09-21T04:15:18.024 INFO:tasks.workunit.client.1.smithi148.stdout:5/457: dread d3/d14/d1f/d26/d2e/d56/f54 [0,4194304] 0 2024-09-21T04:15:18.024 INFO:tasks.workunit.client.1.smithi148.stdout:5/458: chown d3/d24/f76 25591986 1 2024-09-21T04:15:18.024 INFO:tasks.workunit.client.1.smithi148.stdout:5/459: truncate d3/d14/d1f/d26/d2e/d3d/f61 1462002 0 2024-09-21T04:15:18.024 INFO:tasks.workunit.client.1.smithi148.stdout:5/460: chown d3/d37/c3e 28 1 2024-09-21T04:15:18.027 INFO:tasks.workunit.client.0.smithi110.stdout:0/763: creat d4/dd/d53/feb x:0 0 0 2024-09-21T04:15:18.027 INFO:tasks.workunit.client.0.smithi110.stdout:0/764: chown d4/dd/d10 165 1 2024-09-21T04:15:18.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/486: write dd/d1c/d39/f5a [8310025,39906] 0 2024-09-21T04:15:18.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/487: creat dd/d48/d55/d7a/f97 x:0 0 0 2024-09-21T04:15:18.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/488: write dd/f90 [182265,94824] 0 2024-09-21T04:15:18.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/509: mkdir d4/d87/d91 0 2024-09-21T04:15:18.037 INFO:tasks.workunit.client.0.smithi110.stdout:2/321: unlink d2/d1a/d1f/d2a/f21 0 2024-09-21T04:15:18.037 INFO:tasks.workunit.client.0.smithi110.stdout:2/322: chown d2/d1a/d1f/d3f/f4a 0 1 2024-09-21T04:15:18.040 INFO:tasks.workunit.client.0.smithi110.stdout:4/649: link d17/d34/d3d/d57/l85 d17/d3e/dda/lde 0 2024-09-21T04:15:18.040 INFO:tasks.workunit.client.0.smithi110.stdout:4/650: stat d17/d34/fb2 0 2024-09-21T04:15:18.041 INFO:tasks.workunit.client.1.smithi148.stdout:1/443: rmdir db/d24/d4f 39 2024-09-21T04:15:18.041 INFO:tasks.workunit.client.1.smithi148.stdout:1/444: fdatasync db/d24/d2e/d71/f8e 0 2024-09-21T04:15:18.042 INFO:tasks.workunit.client.1.smithi148.stdout:1/445: read db/d24/d4f/f7d [652094,27469] 0 2024-09-21T04:15:18.043 INFO:tasks.workunit.client.0.smithi110.stdout:3/419: mknod d0/d12/d3c/c9a 0 2024-09-21T04:15:18.044 INFO:tasks.workunit.client.1.smithi148.stdout:6/487: write d0/d1a/d23/f14 [2116296,110804] 0 2024-09-21T04:15:18.045 INFO:tasks.workunit.client.0.smithi110.stdout:3/420: write d0/d22/f72 [841916,7715] 0 2024-09-21T04:15:18.046 INFO:tasks.workunit.client.0.smithi110.stdout:1/797: symlink d2/d9/d49/lfd 0 2024-09-21T04:15:18.047 INFO:tasks.workunit.client.1.smithi148.stdout:7/492: symlink d5/la9 0 2024-09-21T04:15:18.048 INFO:tasks.workunit.client.1.smithi148.stdout:7/493: dread - d5/d14/d25/d40/d8b/f99 zero size 2024-09-21T04:15:18.049 INFO:tasks.workunit.client.1.smithi148.stdout:2/579: creat d0/d2b/da3/d82/fb0 x:0 0 0 2024-09-21T04:15:18.052 INFO:tasks.workunit.client.0.smithi110.stdout:0/765: link d4/d47/fde d4/dd/d10/d29/fec 0 2024-09-21T04:15:18.056 INFO:tasks.workunit.client.1.smithi148.stdout:0/490: rename d5/d6/d7/d22/f73 to d5/f9a 0 2024-09-21T04:15:18.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/510: dwrite d4/d66/d7f/f4a [0,4194304] 0 2024-09-21T04:15:18.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/511: stat d4/d12/d16/d29/d61 0 2024-09-21T04:15:18.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/512: chown d4/d12/d16/d29/f33 49849 1 2024-09-21T04:15:18.058 INFO:tasks.workunit.client.1.smithi148.stdout:3/440: getdents d3/d1f/d4d/d82/d23/d28/d46 0 2024-09-21T04:15:18.059 INFO:tasks.workunit.client.0.smithi110.stdout:7/489: mkdir dd/d48/d55/d98 0 2024-09-21T04:15:18.063 INFO:tasks.workunit.client.0.smithi110.stdout:5/569: getdents d1/d49/d4d/d1e/d5a/d64 0 2024-09-21T04:15:18.063 INFO:tasks.workunit.client.0.smithi110.stdout:5/570: fsync d1/d39/f63 0 2024-09-21T04:15:18.065 INFO:tasks.workunit.client.1.smithi148.stdout:5/461: mkdir d3/d14/d1f/d26/d2e/d56/da7 0 2024-09-21T04:15:18.069 INFO:tasks.workunit.client.1.smithi148.stdout:1/446: mknod db/dd/d21/c91 0 2024-09-21T04:15:18.069 INFO:tasks.workunit.client.0.smithi110.stdout:8/550: getdents d0/d7/d13/d4b 0 2024-09-21T04:15:18.070 INFO:tasks.workunit.client.0.smithi110.stdout:8/551: dread - d0/d30/d34/d3e/d5d/f9f zero size 2024-09-21T04:15:18.071 INFO:tasks.workunit.client.0.smithi110.stdout:2/323: mknod d2/d1a/d1f/d29/c5a 0 2024-09-21T04:15:18.074 INFO:tasks.workunit.client.0.smithi110.stdout:1/798: dread d2/d9/d23/d4e/f61 [0,4194304] 0 2024-09-21T04:15:18.074 INFO:tasks.workunit.client.0.smithi110.stdout:1/799: dread - d2/d9/d49/d7a/da8/fd3 zero size 2024-09-21T04:15:18.074 INFO:tasks.workunit.client.0.smithi110.stdout:1/800: chown d2/d9/d49/lfd 6038000 1 2024-09-21T04:15:18.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/494: mkdir d5/daa 0 2024-09-21T04:15:18.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/495: chown d5/d8/d6f/d7d/d90/d28/c2c 126990 1 2024-09-21T04:15:18.076 INFO:tasks.workunit.client.0.smithi110.stdout:1/801: write d2/d9/d23/d3b/d9c/fa4 [210164,49716] 0 2024-09-21T04:15:18.076 INFO:tasks.workunit.client.0.smithi110.stdout:6/532: rename d4/de to d4/d78/daa 0 2024-09-21T04:15:18.076 INFO:tasks.workunit.client.0.smithi110.stdout:1/802: fdatasync d2/db7/d86/f95 0 2024-09-21T04:15:18.076 INFO:tasks.workunit.client.1.smithi148.stdout:2/580: mkdir d0/d2f/db1 0 2024-09-21T04:15:18.079 INFO:tasks.workunit.client.0.smithi110.stdout:7/490: rmdir dd/d1c/d39/d4a/d52 39 2024-09-21T04:15:18.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/497: rename f8 to dc/d13/d6a/f94 0 2024-09-21T04:15:18.080 INFO:tasks.workunit.client.0.smithi110.stdout:2/324: symlink d2/d1a/d1f/l5b 0 2024-09-21T04:15:18.081 INFO:tasks.workunit.client.0.smithi110.stdout:7/491: truncate dd/d1c/d39/f3b 3159444 0 2024-09-21T04:15:18.081 INFO:tasks.workunit.client.0.smithi110.stdout:7/492: readlink dd/d48/d55/d7a/l67 0 2024-09-21T04:15:18.081 INFO:tasks.workunit.client.0.smithi110.stdout:7/493: stat dd/d1c/d39/d4a/d52/f60 0 2024-09-21T04:15:18.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/494: chown dd/d14/c21 2 1 2024-09-21T04:15:18.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/495: dread - dd/d1c/d39/d6d/f7c zero size 2024-09-21T04:15:18.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/496: chown dd/d4d/f5e 64393 1 2024-09-21T04:15:18.083 INFO:tasks.workunit.client.0.smithi110.stdout:2/325: mknod d2/c5c 0 2024-09-21T04:15:18.086 INFO:tasks.workunit.client.1.smithi148.stdout:4/513: symlink d4/d3a/l92 0 2024-09-21T04:15:18.087 INFO:tasks.workunit.client.1.smithi148.stdout:3/441: link d3/d5/d6/d12/d43/l62 d3/d1f/d4d/d82/d52/l88 0 2024-09-21T04:15:18.087 INFO:tasks.workunit.client.1.smithi148.stdout:3/442: truncate d3/d5/f1e 4196648 0 2024-09-21T04:15:18.087 INFO:tasks.workunit.client.1.smithi148.stdout:3/443: read d3/d1f/d4d/d82/d52/f4a [636529,19676] 0 2024-09-21T04:15:18.087 INFO:tasks.workunit.client.0.smithi110.stdout:2/326: rename d2/d1a/d1f/d3f/f46 to d2/d1a/d1f/f5d 0 2024-09-21T04:15:18.087 INFO:tasks.workunit.client.0.smithi110.stdout:2/327: dread - d2/d50/f52 zero size 2024-09-21T04:15:18.093 INFO:tasks.workunit.client.1.smithi148.stdout:1/447: creat db/f92 x:0 0 0 2024-09-21T04:15:18.094 INFO:tasks.workunit.client.1.smithi148.stdout:1/448: read db/d24/d2a/f45 [145740,120207] 0 2024-09-21T04:15:18.094 INFO:tasks.workunit.client.1.smithi148.stdout:1/449: creat db/dd/d21/f93 x:0 0 0 2024-09-21T04:15:18.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/651: dwrite d17/f3c [0,4194304] 0 2024-09-21T04:15:18.096 INFO:tasks.workunit.client.0.smithi110.stdout:2/328: mknod d2/d1a/c5e 0 2024-09-21T04:15:18.097 INFO:tasks.workunit.client.0.smithi110.stdout:8/552: dread d0/d7/d13/d4b/f1f [0,4194304] 0 2024-09-21T04:15:18.098 INFO:tasks.workunit.client.1.smithi148.stdout:6/488: unlink d0/d1a/d23/d53/l73 0 2024-09-21T04:15:18.099 INFO:tasks.workunit.client.1.smithi148.stdout:6/489: chown d0/f20 901 1 2024-09-21T04:15:18.099 INFO:tasks.workunit.client.0.smithi110.stdout:1/803: dread d2/f94 [0,4194304] 0 2024-09-21T04:15:18.100 INFO:tasks.workunit.client.0.smithi110.stdout:1/804: write d2/d9/d26/d3d/fd9 [395190,93965] 0 2024-09-21T04:15:18.102 INFO:tasks.workunit.client.1.smithi148.stdout:7/496: mkdir d5/d14/d25/d40/dab 0 2024-09-21T04:15:18.102 INFO:tasks.workunit.client.1.smithi148.stdout:7/497: write d5/d14/d25/f4c [71271,47629] 0 2024-09-21T04:15:18.102 INFO:tasks.workunit.client.1.smithi148.stdout:6/490: dread d0/d1a/d23/d16/f39 [0,4194304] 0 2024-09-21T04:15:18.102 INFO:tasks.workunit.client.1.smithi148.stdout:7/498: write d5/d14/d25/d79/d86/fa3 [1019165,117612] 0 2024-09-21T04:15:18.103 INFO:tasks.workunit.client.1.smithi148.stdout:6/491: chown d0/d34/d38/d43/d63/c87 5233084 1 2024-09-21T04:15:18.103 INFO:tasks.workunit.client.0.smithi110.stdout:8/553: link d0/d7/d23/d82/ca4 d0/d30/d34/d3e/d60/cc2 0 2024-09-21T04:15:18.104 INFO:tasks.workunit.client.0.smithi110.stdout:2/329: write d2/f16 [3820133,107243] 0 2024-09-21T04:15:18.105 INFO:tasks.workunit.client.0.smithi110.stdout:2/330: truncate d2/d1a/d1f/d3f/f2b 2471830 0 2024-09-21T04:15:18.105 INFO:tasks.workunit.client.0.smithi110.stdout:2/331: stat d2/d1a/d1f/d3f/f58 0 2024-09-21T04:15:18.105 INFO:tasks.workunit.client.0.smithi110.stdout:2/332: write d2/d1a/d1f/f5d [768013,73908] 0 2024-09-21T04:15:18.105 INFO:tasks.workunit.client.1.smithi148.stdout:2/581: mkdir d0/d9/dc/d13/daf/db2 0 2024-09-21T04:15:18.105 INFO:tasks.workunit.client.1.smithi148.stdout:2/582: chown d0/d2b/da3/da1/f2e 51 1 2024-09-21T04:15:18.106 INFO:tasks.workunit.client.0.smithi110.stdout:1/805: rename d2/db7/d86/ldf to d2/d9/d23/d4e/d57/lfe 0 2024-09-21T04:15:18.108 INFO:tasks.workunit.client.1.smithi148.stdout:4/514: link d4/d87/d2d/l4e d4/d12/d16/d29/d61/l93 0 2024-09-21T04:15:18.108 INFO:tasks.workunit.client.1.smithi148.stdout:4/515: fdatasync d4/d87/f32 0 2024-09-21T04:15:18.108 INFO:tasks.workunit.client.1.smithi148.stdout:4/516: fsync d4/f11 0 2024-09-21T04:15:18.108 INFO:tasks.workunit.client.1.smithi148.stdout:4/517: readlink d4/d12/d16/l4d 0 2024-09-21T04:15:18.109 INFO:tasks.workunit.client.0.smithi110.stdout:2/333: truncate d2/d50/d56/f53 945970 0 2024-09-21T04:15:18.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/444: rename l2 to d3/d5/d6/l89 0 2024-09-21T04:15:18.115 INFO:tasks.workunit.client.1.smithi148.stdout:5/462: getdents d3/d14/d1f/d26/d2e/d3d 0 2024-09-21T04:15:18.115 INFO:tasks.workunit.client.1.smithi148.stdout:5/463: chown d3/d14/d1f/d26/d2e/d56/f17 53626 1 2024-09-21T04:15:18.116 INFO:tasks.workunit.client.0.smithi110.stdout:1/806: write d2/d9/f1a [44689,120339] 0 2024-09-21T04:15:18.120 INFO:tasks.workunit.client.0.smithi110.stdout:3/421: dwrite d0/d12/d16/d1a/f3b [0,4194304] 0 2024-09-21T04:15:18.122 INFO:tasks.workunit.client.1.smithi148.stdout:1/450: chown db/dd/c1c 124 1 2024-09-21T04:15:18.122 INFO:tasks.workunit.client.1.smithi148.stdout:1/451: dread - db/d46/f8d zero size 2024-09-21T04:15:18.123 INFO:tasks.workunit.client.0.smithi110.stdout:4/652: dread d17/f2a [0,4194304] 0 2024-09-21T04:15:18.123 INFO:tasks.workunit.client.0.smithi110.stdout:4/653: fsync d17/d34/fb0 0 2024-09-21T04:15:18.123 INFO:tasks.workunit.client.0.smithi110.stdout:4/654: truncate d17/d1e/d7b/dc5/fdd 358450 0 2024-09-21T04:15:18.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/766: dwrite d4/dd/d10/d29/d45/f5b [0,4194304] 0 2024-09-21T04:15:18.123 INFO:tasks.workunit.client.0.smithi110.stdout:1/807: link d2/d9/d49/d4f/cb5 d2/d9/d23/d3b/cff 0 2024-09-21T04:15:18.126 INFO:tasks.workunit.client.1.smithi148.stdout:6/492: getdents d0/d1a/d23 0 2024-09-21T04:15:18.129 INFO:tasks.workunit.client.1.smithi148.stdout:7/499: creat d5/d8/d6f/d7d/fac x:0 0 0 2024-09-21T04:15:18.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/445: symlink d3/d1f/d4d/d82/d52/l8a 0 2024-09-21T04:15:18.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/446: stat d3/d1f/d4d/d82/d23/l6f 0 2024-09-21T04:15:18.131 INFO:tasks.workunit.client.1.smithi148.stdout:6/493: write d0/d8/d18/f4d [1256454,6854] 0 2024-09-21T04:15:18.131 INFO:tasks.workunit.client.1.smithi148.stdout:6/494: read - d0/d34/d38/d43/d63/f77 zero size 2024-09-21T04:15:18.131 INFO:tasks.workunit.client.1.smithi148.stdout:6/495: creat d0/d34/d38/d43/f8a x:0 0 0 2024-09-21T04:15:18.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/422: creat d0/d22/f9b x:0 0 0 2024-09-21T04:15:18.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/423: write d0/d22/f72 [771576,12838] 0 2024-09-21T04:15:18.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/452: chown db/dd/c14 28 1 2024-09-21T04:15:18.135 INFO:tasks.workunit.client.0.smithi110.stdout:4/655: mkdir d17/d34/d3d/d20/d43/d51/d7a/d96/ddf 0 2024-09-21T04:15:18.136 INFO:tasks.workunit.client.1.smithi148.stdout:7/500: mkdir d5/d14/d25/d40/dab/dad 0 2024-09-21T04:15:18.138 INFO:tasks.workunit.client.0.smithi110.stdout:5/571: dwrite d1/f4c [0,4194304] 0 2024-09-21T04:15:18.138 INFO:tasks.workunit.client.0.smithi110.stdout:5/572: chown d1/l75 16130 1 2024-09-21T04:15:18.138 INFO:tasks.workunit.client.0.smithi110.stdout:5/573: chown d1/d7/d18/f74 114 1 2024-09-21T04:15:18.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/496: mknod d0/d1a/c8b 0 2024-09-21T04:15:18.143 INFO:tasks.workunit.client.0.smithi110.stdout:1/808: truncate d2/d9/d23/f64 3824324 0 2024-09-21T04:15:18.143 INFO:tasks.workunit.client.0.smithi110.stdout:1/809: write d2/d9/d23/d4e/d57/dc9/fe8 [41354,75457] 0 2024-09-21T04:15:18.143 INFO:tasks.workunit.client.0.smithi110.stdout:1/810: readlink d2/d9/d23/d3b/l74 0 2024-09-21T04:15:18.144 INFO:tasks.workunit.client.1.smithi148.stdout:1/453: mkdir db/d24/d2e/d71/d94 0 2024-09-21T04:15:18.145 INFO:tasks.workunit.client.1.smithi148.stdout:6/497: getdents d0/d1a/d23/d16/d36 0 2024-09-21T04:15:18.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/498: creat d0/d1a/d23/d16/d4f/f8c x:0 0 0 2024-09-21T04:15:18.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/499: readlink d0/d34/d38/l70 0 2024-09-21T04:15:18.148 INFO:tasks.workunit.client.1.smithi148.stdout:6/500: mknod d0/d34/c8d 0 2024-09-21T04:15:18.152 INFO:tasks.workunit.client.1.smithi148.stdout:3/447: dread d3/d1f/d4d/d82/f1d [0,4194304] 0 2024-09-21T04:15:18.152 INFO:tasks.workunit.client.1.smithi148.stdout:3/448: write d3/d5/f1e [4908418,644] 0 2024-09-21T04:15:18.152 INFO:tasks.workunit.client.1.smithi148.stdout:3/449: truncate d3/d1f/f3e 1274595 0 2024-09-21T04:15:18.152 INFO:tasks.workunit.client.1.smithi148.stdout:6/501: rename d0/d1a/d23/d16/f5d to d0/d1a/d23/dc/f8e 0 2024-09-21T04:15:18.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/491: dwrite d5/f9a [0,4194304] 0 2024-09-21T04:15:18.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/492: write d5/d6/d7/d22/f6c [542141,44946] 0 2024-09-21T04:15:18.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/493: dread - d5/d6/d2a/d45/d70/f75 zero size 2024-09-21T04:15:18.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/494: creat d5/d6/d2a/d45/f9b x:0 0 0 2024-09-21T04:15:18.154 INFO:tasks.workunit.client.1.smithi148.stdout:6/502: dread d0/d8/f49 [0,4194304] 0 2024-09-21T04:15:18.154 INFO:tasks.workunit.client.1.smithi148.stdout:3/450: mkdir d3/d1f/d4d/d8b 0 2024-09-21T04:15:18.154 INFO:tasks.workunit.client.0.smithi110.stdout:6/533: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:18.154 INFO:tasks.workunit.client.0.smithi110.stdout:6/534: stat d4/d12/d16/d66/d7a/d4b/f51 0 2024-09-21T04:15:18.155 INFO:tasks.workunit.client.0.smithi110.stdout:6/535: dread - d4/d12/d17/f9d zero size 2024-09-21T04:15:18.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/454: write db/f58 [1437465,10972] 0 2024-09-21T04:15:18.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/455: dread - db/d46/f6a zero size 2024-09-21T04:15:18.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/456: chown db/ff 221693 1 2024-09-21T04:15:18.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/457: write db/d46/f66 [740256,24886] 0 2024-09-21T04:15:18.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/458: fsync db/f64 0 2024-09-21T04:15:18.160 INFO:tasks.workunit.client.1.smithi148.stdout:0/495: rmdir d5/d6/d2a/d96 0 2024-09-21T04:15:18.160 INFO:tasks.workunit.client.1.smithi148.stdout:0/496: truncate d5/d6/d7/f83 24805 0 2024-09-21T04:15:18.161 INFO:tasks.workunit.client.1.smithi148.stdout:0/497: dread - d5/d6/d7/d11/d48/f6b zero size 2024-09-21T04:15:18.162 INFO:tasks.workunit.client.1.smithi148.stdout:0/498: dread d5/d6/d3e/f64 [4194304,4194304] 0 2024-09-21T04:15:18.163 INFO:tasks.workunit.client.1.smithi148.stdout:8/498: dwrite dc/d13/d76/d42/f4a [0,4194304] 0 2024-09-21T04:15:18.164 INFO:tasks.workunit.client.1.smithi148.stdout:8/499: write dc/d13/d3c/f5f [2621687,130704] 0 2024-09-21T04:15:18.164 INFO:tasks.workunit.client.0.smithi110.stdout:7/497: dwrite dd/d4d/f68 [0,4194304] 0 2024-09-21T04:15:18.164 INFO:tasks.workunit.client.0.smithi110.stdout:7/498: stat dd/d1c/d39/d4a 0 2024-09-21T04:15:18.166 INFO:tasks.workunit.client.0.smithi110.stdout:2/334: rename d2/d1a/d1f/d3f to d2/d1a/d1f/d5f 0 2024-09-21T04:15:18.166 INFO:tasks.workunit.client.0.smithi110.stdout:2/335: fdatasync d2/f44 0 2024-09-21T04:15:18.167 INFO:tasks.workunit.client.0.smithi110.stdout:8/554: dwrite d0/d30/d54/fb0 [0,4194304] 0 2024-09-21T04:15:18.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/424: mkdir d0/d12/d2a/d80/d9c 0 2024-09-21T04:15:18.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/425: write d0/d12/f23 [1912853,31801] 0 2024-09-21T04:15:18.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/426: write d0/f6 [4271917,77236] 0 2024-09-21T04:15:18.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/427: chown d0/d51/l6d 13416343 1 2024-09-21T04:15:18.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/428: creat d0/d12/f9d x:0 0 0 2024-09-21T04:15:18.169 INFO:tasks.workunit.client.0.smithi110.stdout:3/429: chown d0/d12/d2a/l47 1912 1 2024-09-21T04:15:18.170 INFO:tasks.workunit.client.1.smithi148.stdout:0/499: link d5/d6/d2a/d45/c92 d5/d6/d2a/d3f/d8b/c9c 0 2024-09-21T04:15:18.173 INFO:tasks.workunit.client.0.smithi110.stdout:4/656: truncate d17/d34/d3d/d20/d43/d51/d52/d98/f99 3004649 0 2024-09-21T04:15:18.173 INFO:tasks.workunit.client.0.smithi110.stdout:4/657: chown d17/d1e/d21/l76 74092 1 2024-09-21T04:15:18.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/583: dwrite d0/ff [0,4194304] 0 2024-09-21T04:15:18.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/584: chown d0/d2b/da3/d95 19394 1 2024-09-21T04:15:18.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/518: dwrite d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:18.174 INFO:tasks.workunit.client.1.smithi148.stdout:5/464: dwrite d3/f1b [0,4194304] 0 2024-09-21T04:15:18.174 INFO:tasks.workunit.client.1.smithi148.stdout:7/501: dread d5/d14/d25/f4c [0,4194304] 0 2024-09-21T04:15:18.176 INFO:tasks.workunit.client.0.smithi110.stdout:6/536: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:18.176 INFO:tasks.workunit.client.1.smithi148.stdout:6/503: dwrite d0/d34/d38/d43/f7b [0,4194304] 0 2024-09-21T04:15:18.184 INFO:tasks.workunit.client.0.smithi110.stdout:5/574: link d1/d49/d4d/d1e/c5b d1/d7/d58/d72/cbf 0 2024-09-21T04:15:18.185 INFO:tasks.workunit.client.1.smithi148.stdout:1/459: dwrite db/d24/d2e/f63 [4194304,4194304] 0 2024-09-21T04:15:18.186 INFO:tasks.workunit.client.0.smithi110.stdout:0/767: rename d4/dd/lbe to d4/led 0 2024-09-21T04:15:18.187 INFO:tasks.workunit.client.0.smithi110.stdout:0/768: chown d4/dd/d10/d29/fec 7 1 2024-09-21T04:15:18.188 INFO:tasks.workunit.client.0.smithi110.stdout:1/811: mkdir d2/d9/d23/d4e/d8e/d100 0 2024-09-21T04:15:18.196 INFO:tasks.workunit.client.0.smithi110.stdout:7/499: symlink dd/d14/d17/l99 0 2024-09-21T04:15:18.196 INFO:tasks.workunit.client.0.smithi110.stdout:7/500: dread - dd/d1c/d39/d6d/f7c zero size 2024-09-21T04:15:18.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/336: mkdir d2/d1a/d60 0 2024-09-21T04:15:18.203 INFO:tasks.workunit.client.0.smithi110.stdout:8/555: creat d0/d30/d54/d6b/fc3 x:0 0 0 2024-09-21T04:15:18.207 INFO:tasks.workunit.client.1.smithi148.stdout:8/500: creat dc/d13/d76/d5d/d8f/f95 x:0 0 0 2024-09-21T04:15:18.208 INFO:tasks.workunit.client.0.smithi110.stdout:6/537: dwrite d4/d12/d16/d66/d7a/d83/f9f [0,4194304] 0 2024-09-21T04:15:18.208 INFO:tasks.workunit.client.1.smithi148.stdout:1/460: dwrite db/f15 [0,4194304] 0 2024-09-21T04:15:18.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/451: rmdir d3/d1f/d4d/d82/d21 39 2024-09-21T04:15:18.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/452: fdatasync d3/d5/d6/d12/d15/d73/f7c 0 2024-09-21T04:15:18.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/453: readlink d3/d1f/d4d/d82/d52/l8a 0 2024-09-21T04:15:18.212 INFO:tasks.workunit.client.0.smithi110.stdout:3/430: symlink d0/d12/d3c/l9e 0 2024-09-21T04:15:18.213 INFO:tasks.workunit.client.0.smithi110.stdout:3/431: dread d0/d12/d3c/d5a/d70/f84 [0,4194304] 0 2024-09-21T04:15:18.213 INFO:tasks.workunit.client.0.smithi110.stdout:3/432: stat d0/d12/d3c/d5a/d70/d81 0 2024-09-21T04:15:18.214 INFO:tasks.workunit.client.1.smithi148.stdout:0/500: link d5/d6/d2a/d2b/c5d d5/d6/d2a/d2b/d84/c9d 0 2024-09-21T04:15:18.214 INFO:tasks.workunit.client.1.smithi148.stdout:0/501: truncate d5/d6/d7/d11/f38 1926611 0 2024-09-21T04:15:18.215 INFO:tasks.workunit.client.0.smithi110.stdout:4/658: symlink d17/d34/d3d/d57/db9/le0 0 2024-09-21T04:15:18.219 INFO:tasks.workunit.client.0.smithi110.stdout:5/575: rmdir d1/d7/d18/da9 0 2024-09-21T04:15:18.219 INFO:tasks.workunit.client.1.smithi148.stdout:2/585: symlink d0/d2b/lb3 0 2024-09-21T04:15:18.221 INFO:tasks.workunit.client.0.smithi110.stdout:0/769: mkdir d4/dd/d10/dee 0 2024-09-21T04:15:18.221 INFO:tasks.workunit.client.0.smithi110.stdout:0/770: truncate d4/dd/d53/feb 20754 0 2024-09-21T04:15:18.230 INFO:tasks.workunit.client.0.smithi110.stdout:1/812: rmdir d2/d9/d23/d4e/d8e/d100 0 2024-09-21T04:15:18.231 INFO:tasks.workunit.client.0.smithi110.stdout:1/813: chown d2/d9/d23/f75 2 1 2024-09-21T04:15:18.232 INFO:tasks.workunit.client.1.smithi148.stdout:8/501: dwrite dc/f46 [4194304,4194304] 0 2024-09-21T04:15:18.232 INFO:tasks.workunit.client.1.smithi148.stdout:4/519: mkdir d4/d66/d94 0 2024-09-21T04:15:18.234 INFO:tasks.workunit.client.0.smithi110.stdout:7/501: unlink dd/f2e 0 2024-09-21T04:15:18.236 INFO:tasks.workunit.client.1.smithi148.stdout:0/502: dwrite d5/d6/d7/d11/d48/f7d [0,4194304] 0 2024-09-21T04:15:18.237 INFO:tasks.workunit.client.0.smithi110.stdout:2/337: creat d2/d1a/d1f/d2a/f61 x:0 0 0 2024-09-21T04:15:18.239 INFO:tasks.workunit.client.1.smithi148.stdout:5/465: unlink d3/d14/d1f/d26/d2e/d56/l22 0 2024-09-21T04:15:18.240 INFO:tasks.workunit.client.0.smithi110.stdout:8/556: link d0/d30/d34/d3e/d32/f7c d0/d7/d23/d82/d92/fc4 0 2024-09-21T04:15:18.243 INFO:tasks.workunit.client.1.smithi148.stdout:7/502: truncate d5/d8/d6f/d7d/d90/d28/f87 1413281 0 2024-09-21T04:15:18.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.240+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd84c000bf0 con 0x7fd85403fbf0 2024-09-21T04:15:18.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.242+0000 7fd85e7fc640 1 -- 172.21.15.110:0/3797764758 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd84c000bf0 con 0x7fd85403fbf0 2024-09-21T04:15:18.244 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "1/23 daemons upgraded", 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:15:18.245 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:15:18.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 msgr2=0x7fd8540420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 0x7fd8540420b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd868194720 tx=0x7fd85800b390 comp rx=0 tx=0).stop 2024-09-21T04:15:18.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 msgr2=0x7fd868193200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8640095c0 tx=0x7fd8640095f0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.248 INFO:tasks.workunit.client.0.smithi110.stdout:5/576: dwrite d1/d39/f79 [0,4194304] 0 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 shutdown_connections 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd85403fbf0 0x7fd8540420b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680ffa60 0x7fd868193740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 --2- 172.21.15.110:0/3797764758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8680fe860 0x7fd868193200 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8640095c0 tx=0x7fd8640095f0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 >> 172.21.15.110:0/3797764758 conn(0x7fd8680fa010 msgr2=0x7fd868102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:18.249 INFO:tasks.workunit.client.0.smithi110.stdout:0/771: dwrite d4/d1c/d62/d69/d9b/fad [0,4194304] 0 2024-09-21T04:15:18.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 shutdown_connections 2024-09-21T04:15:18.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.245+0000 7fd86e7d7640 1 -- 172.21.15.110:0/3797764758 wait complete. 2024-09-21T04:15:18.250 INFO:tasks.workunit.client.0.smithi110.stdout:6/538: mkdir d4/d12/d17/d33/dab 0 2024-09-21T04:15:18.251 INFO:tasks.workunit.client.1.smithi148.stdout:6/504: link d0/d8/d18/c6d d0/d1a/d23/dc/d68/c8f 0 2024-09-21T04:15:18.252 INFO:tasks.workunit.client.1.smithi148.stdout:6/505: chown d0/d1a/d23/dc/d68/d5b/d89 36315 1 2024-09-21T04:15:18.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/466: dread d3/d24/f2f [0,4194304] 0 2024-09-21T04:15:18.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/467: fsync d3/f36 0 2024-09-21T04:15:18.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/468: chown d3/d34/d3c/d5d/f93 2356521 1 2024-09-21T04:15:18.254 INFO:tasks.workunit.client.1.smithi148.stdout:2/586: dwrite d0/d9/d38/f41 [0,4194304] 0 2024-09-21T04:15:18.254 INFO:tasks.workunit.client.1.smithi148.stdout:2/587: dread - d0/d9/d38/f6d zero size 2024-09-21T04:15:18.255 INFO:tasks.workunit.client.1.smithi148.stdout:9/493: sync 2024-09-21T04:15:18.255 INFO:tasks.workunit.client.1.smithi148.stdout:9/494: creat d5/d60/d28/fbb x:0 0 0 2024-09-21T04:15:18.261 INFO:tasks.workunit.client.1.smithi148.stdout:1/461: truncate db/d46/f48 3682557 0 2024-09-21T04:15:18.261 INFO:tasks.workunit.client.1.smithi148.stdout:1/462: chown db/dd/d21/l31 7 1 2024-09-21T04:15:18.264 INFO:tasks.workunit.client.1.smithi148.stdout:3/454: symlink d3/d5/d5c/l8c 0 2024-09-21T04:15:18.267 INFO:tasks.workunit.client.1.smithi148.stdout:8/502: unlink dc/fe 0 2024-09-21T04:15:18.269 INFO:tasks.workunit.client.0.smithi110.stdout:4/659: mkdir d17/d1e/d21/de1 0 2024-09-21T04:15:18.269 INFO:tasks.workunit.client.0.smithi110.stdout:4/660: getdents d17/d3e/dd3 0 2024-09-21T04:15:18.269 INFO:tasks.workunit.client.1.smithi148.stdout:8/503: dread dc/d13/f24 [0,4194304] 0 2024-09-21T04:15:18.269 INFO:tasks.workunit.client.1.smithi148.stdout:5/469: dwrite d3/d37/f7f [0,4194304] 0 2024-09-21T04:15:18.270 INFO:tasks.workunit.client.1.smithi148.stdout:4/520: unlink d4/d12/d75/d85/c79 0 2024-09-21T04:15:18.274 INFO:tasks.workunit.client.0.smithi110.stdout:1/814: truncate d2/d9/f1f 248418 0 2024-09-21T04:15:18.275 INFO:tasks.workunit.client.1.smithi148.stdout:0/503: symlink d5/d6/d2a/d45/d70/l9e 0 2024-09-21T04:15:18.275 INFO:tasks.workunit.client.1.smithi148.stdout:0/504: creat d5/d6/d7/d11/d60/f9f x:0 0 0 2024-09-21T04:15:18.277 INFO:tasks.workunit.client.1.smithi148.stdout:7/503: fsync d5/d8/f91 0 2024-09-21T04:15:18.278 INFO:tasks.workunit.client.0.smithi110.stdout:7/502: mknod dd/d48/c9a 0 2024-09-21T04:15:18.278 INFO:tasks.workunit.client.0.smithi110.stdout:4/661: dread d17/d34/d3d/d20/f26 [0,4194304] 0 2024-09-21T04:15:18.278 INFO:tasks.workunit.client.0.smithi110.stdout:4/662: stat d17/d34/d3d/d20/fa8 0 2024-09-21T04:15:18.280 INFO:tasks.workunit.client.0.smithi110.stdout:2/338: mkdir d2/d1a/d1f/d29/d19/d62 0 2024-09-21T04:15:18.280 INFO:tasks.workunit.client.0.smithi110.stdout:2/339: chown d2/d1a/f20 5 1 2024-09-21T04:15:18.283 INFO:tasks.workunit.client.1.smithi148.stdout:2/588: getdents d0/d2b 0 2024-09-21T04:15:18.283 INFO:tasks.workunit.client.1.smithi148.stdout:2/589: stat d0/d9/d59/f65 0 2024-09-21T04:15:18.286 INFO:tasks.workunit.client.0.smithi110.stdout:5/577: symlink d1/d22/d51/lc0 0 2024-09-21T04:15:18.286 INFO:tasks.workunit.client.0.smithi110.stdout:5/578: truncate d1/d49/fab 201153 0 2024-09-21T04:15:18.287 INFO:tasks.workunit.client.1.smithi148.stdout:6/506: dwrite d0/d1a/d23/d16/d4f/f8c [0,4194304] 0 2024-09-21T04:15:18.288 INFO:tasks.workunit.client.1.smithi148.stdout:9/495: creat d5/d60/d69/fbc x:0 0 0 2024-09-21T04:15:18.290 INFO:tasks.workunit.client.1.smithi148.stdout:1/463: creat db/d24/d4f/f95 x:0 0 0 2024-09-21T04:15:18.290 INFO:tasks.workunit.client.0.smithi110.stdout:0/772: unlink d4/f57 0 2024-09-21T04:15:18.292 INFO:tasks.workunit.client.1.smithi148.stdout:3/455: mknod d3/c8d 0 2024-09-21T04:15:18.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/470: dwrite d3/d14/d1a/f9a [0,4194304] 0 2024-09-21T04:15:18.294 INFO:tasks.workunit.client.0.smithi110.stdout:6/539: rename d4/d12/d17/d9a/fa2 to d4/d12/d16/d66/fac 0 2024-09-21T04:15:18.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/504: unlink dc/d13/d39/d6e/f89 0 2024-09-21T04:15:18.298 INFO:tasks.workunit.client.0.smithi110.stdout:3/433: mknod d0/d22/d29/d76/c9f 0 2024-09-21T04:15:18.302 INFO:tasks.workunit.client.0.smithi110.stdout:1/815: truncate d2/d9/d26/d3d/fd9 492954 0 2024-09-21T04:15:18.302 INFO:tasks.workunit.client.0.smithi110.stdout:1/816: stat d2/db7/d86 0 2024-09-21T04:15:18.303 INFO:tasks.workunit.client.1.smithi148.stdout:3/456: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:18.304 INFO:tasks.workunit.client.0.smithi110.stdout:4/663: creat d17/d34/d3d/d20/d43/fe2 x:0 0 0 2024-09-21T04:15:18.305 INFO:tasks.workunit.client.0.smithi110.stdout:3/434: write d0/f7 [2599561,76000] 0 2024-09-21T04:15:18.305 INFO:tasks.workunit.client.0.smithi110.stdout:3/435: stat d0/d12/d16/d1a/c71 0 2024-09-21T04:15:18.307 INFO:tasks.workunit.client.0.smithi110.stdout:1/817: dread d2/d9/d49/d7a/f7c [0,4194304] 0 2024-09-21T04:15:18.307 INFO:tasks.workunit.client.0.smithi110.stdout:1/818: fdatasync d2/d9/d49/d5c/d91/fde 0 2024-09-21T04:15:18.307 INFO:tasks.workunit.client.1.smithi148.stdout:4/521: unlink d4/d3a/f5f 0 2024-09-21T04:15:18.308 INFO:tasks.workunit.client.1.smithi148.stdout:3/457: dread d3/f55 [0,4194304] 0 2024-09-21T04:15:18.308 INFO:tasks.workunit.client.1.smithi148.stdout:3/458: fdatasync d3/d1f/d4d/d82/d23/d28/d65/f67 0 2024-09-21T04:15:18.309 INFO:tasks.workunit.client.0.smithi110.stdout:2/340: mknod d2/d1a/d1f/c63 0 2024-09-21T04:15:18.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/590: dwrite d0/f68 [0,4194304] 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/591: write d0/d2b/f7d [1707592,42071] 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/592: truncate d0/d9/d38/f4f 515517 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/593: stat d0/d2f/l3e 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/594: readlink d0/d9/d16/d55/d71/l7a 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.0.smithi110.stdout:0/773: creat d4/d1c/d62/fef x:0 0 0 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.0.smithi110.stdout:0/774: chown d4/dd/d53/d66 344701 1 2024-09-21T04:15:18.316 INFO:tasks.workunit.client.0.smithi110.stdout:0/775: truncate d4/dd/d60/f7e 4765656 0 2024-09-21T04:15:18.318 INFO:tasks.workunit.client.0.smithi110.stdout:6/540: unlink d4/fb 0 2024-09-21T04:15:18.319 INFO:tasks.workunit.client.0.smithi110.stdout:6/541: dread - d4/d12/d17/d81/fa0 zero size 2024-09-21T04:15:18.319 INFO:tasks.workunit.client.0.smithi110.stdout:6/542: write d4/d12/d16/d66/f93 [28824,88681] 0 2024-09-21T04:15:18.320 INFO:tasks.workunit.client.1.smithi148.stdout:7/504: symlink d5/d14/lae 0 2024-09-21T04:15:18.322 INFO:tasks.workunit.client.1.smithi148.stdout:0/505: dwrite d5/d6/d7/f4b [0,4194304] 0 2024-09-21T04:15:18.322 INFO:tasks.workunit.client.1.smithi148.stdout:0/506: readlink d5/d6/d7/d18/l6a 0 2024-09-21T04:15:18.323 INFO:tasks.workunit.client.0.smithi110.stdout:4/664: rename d17/d34/d3d/d20/d43/d51/d7a/f7f to d17/d34/d3d/d20/d43/fe3 0 2024-09-21T04:15:18.326 INFO:tasks.workunit.client.0.smithi110.stdout:0/776: symlink d4/dd/d60/d72/lf0 0 2024-09-21T04:15:18.327 INFO:tasks.workunit.client.1.smithi148.stdout:3/459: read d3/d5/f35 [3076085,39607] 0 2024-09-21T04:15:18.330 INFO:tasks.workunit.client.0.smithi110.stdout:6/543: mkdir d4/d12/d17/d2e/d40/dad 0 2024-09-21T04:15:18.330 INFO:tasks.workunit.client.0.smithi110.stdout:6/544: creat d4/d12/d17/d33/d76/fae x:0 0 0 2024-09-21T04:15:18.332 INFO:tasks.workunit.client.0.smithi110.stdout:7/503: dwrite dd/d1c/d39/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:18.332 INFO:tasks.workunit.client.0.smithi110.stdout:6/545: dread d4/d12/f1d [4194304,4194304] 0 2024-09-21T04:15:18.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/507: write d5/d6/f62 [3433338,106759] 0 2024-09-21T04:15:18.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/508: write d5/d6/d7/d22/f6c [1260527,80181] 0 2024-09-21T04:15:18.336 INFO:tasks.workunit.client.0.smithi110.stdout:2/341: rename d2/l47 to d2/d1a/d1f/d29/l64 0 2024-09-21T04:15:18.339 INFO:tasks.workunit.client.1.smithi148.stdout:9/496: rename d5/d60/d28/d2f/d8e to d5/d60/d4c/dbd 0 2024-09-21T04:15:18.340 INFO:tasks.workunit.client.0.smithi110.stdout:0/777: rmdir d4/dd/d10/d8a/db5 39 2024-09-21T04:15:18.341 INFO:tasks.workunit.client.0.smithi110.stdout:9/434: sync 2024-09-21T04:15:18.344 INFO:tasks.workunit.client.1.smithi148.stdout:1/464: unlink db/dd/d21/l67 0 2024-09-21T04:15:18.344 INFO:tasks.workunit.client.1.smithi148.stdout:9/497: dread d5/d60/d28/f30 [0,4194304] 0 2024-09-21T04:15:18.346 INFO:tasks.workunit.client.0.smithi110.stdout:6/546: write d4/f3d [1130466,31333] 0 2024-09-21T04:15:18.351 INFO:tasks.workunit.client.0.smithi110.stdout:7/504: mkdir dd/d48/d55/d7a/d9b 0 2024-09-21T04:15:18.351 INFO:tasks.workunit.client.0.smithi110.stdout:7/505: write dd/d1c/d39/d6d/f7c [69302,26795] 0 2024-09-21T04:15:18.351 INFO:tasks.workunit.client.0.smithi110.stdout:1/819: dwrite d2/d9/d26/f31 [0,4194304] 0 2024-09-21T04:15:18.351 INFO:tasks.workunit.client.1.smithi148.stdout:4/522: dwrite d4/f6 [8388608,4194304] 0 2024-09-21T04:15:18.351 INFO:tasks.workunit.client.1.smithi148.stdout:4/523: chown d4/d66/d7f/c8b 5 1 2024-09-21T04:15:18.354 INFO:tasks.workunit.client.0.smithi110.stdout:1/820: write d2/d9/d49/d4f/d53/f85 [1257902,43239] 0 2024-09-21T04:15:18.355 INFO:tasks.workunit.client.1.smithi148.stdout:2/595: dwrite d0/ff [0,4194304] 0 2024-09-21T04:15:18.356 INFO:tasks.workunit.client.1.smithi148.stdout:8/505: creat dc/d43/f96 x:0 0 0 2024-09-21T04:15:18.356 INFO:tasks.workunit.client.0.smithi110.stdout:4/665: dwrite d17/d34/d3d/d20/d43/d51/fb5 [0,4194304] 0 2024-09-21T04:15:18.356 INFO:tasks.workunit.client.0.smithi110.stdout:4/666: fdatasync d17/f39 0 2024-09-21T04:15:18.358 INFO:tasks.workunit.client.0.smithi110.stdout:0/778: creat d4/d1c/d62/d69/d9b/ff1 x:0 0 0 2024-09-21T04:15:18.358 INFO:tasks.workunit.client.0.smithi110.stdout:2/342: dwrite d2/f44 [0,4194304] 0 2024-09-21T04:15:18.360 INFO:tasks.workunit.client.0.smithi110.stdout:1/821: read d2/d9/d49/fbe [4115592,103363] 0 2024-09-21T04:15:18.360 INFO:tasks.workunit.client.0.smithi110.stdout:1/822: write d2/d9/d26/d3d/fc1 [519616,122047] 0 2024-09-21T04:15:18.360 INFO:tasks.workunit.client.0.smithi110.stdout:1/823: dread - d2/d9/d26/d3d/fc5 zero size 2024-09-21T04:15:18.361 INFO:tasks.workunit.client.0.smithi110.stdout:9/435: rmdir d7/d10/d24/d3c/d4f 39 2024-09-21T04:15:18.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/547: unlink d4/d12/d16/d66/d7a/d4b/f8d 0 2024-09-21T04:15:18.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/548: readlink d4/d12/l2c 0 2024-09-21T04:15:18.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/549: write d4/f6 [1076349,91947] 0 2024-09-21T04:15:18.369 INFO:tasks.workunit.client.1.smithi148.stdout:7/505: creat d5/d8/d6f/d7d/d90/faf x:0 0 0 2024-09-21T04:15:18.371 INFO:tasks.workunit.client.0.smithi110.stdout:6/550: dread d4/d12/d17/d2e/d38/f4f [0,4194304] 0 2024-09-21T04:15:18.374 INFO:tasks.workunit.client.0.smithi110.stdout:4/667: creat d17/d1e/d77/dcf/fe4 x:0 0 0 2024-09-21T04:15:18.376 INFO:tasks.workunit.client.1.smithi148.stdout:6/507: rmdir d0/d34/d38/d57/d58 39 2024-09-21T04:15:18.378 INFO:tasks.workunit.client.1.smithi148.stdout:3/460: rename d3/d5/d6/d12/d43/f83 to d3/d1f/f8e 0 2024-09-21T04:15:18.379 INFO:tasks.workunit.client.0.smithi110.stdout:0/779: mknod d4/dd/d60/d72/cf2 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:8/506: dwrite dc/d3e/f70 [0,4194304] 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:8/507: getdents dc/d13/d76/d7c 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/465: link db/dd/d21/c91 db/d46/d4c/c96 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/466: dread - db/d24/d2e/d59/f7e zero size 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/467: creat db/d46/f97 x:0 0 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.1.smithi148.stdout:1/468: truncate db/d24/d4f/f95 933279 0 2024-09-21T04:15:18.381 INFO:tasks.workunit.client.0.smithi110.stdout:1/824: dwrite d2/d9/d49/d5c/d91/fa5 [4194304,4194304] 0 2024-09-21T04:15:18.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/524: creat d4/d66/d7f/d2a/f95 x:0 0 0 2024-09-21T04:15:18.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/525: write d4/d87/f57 [865431,19256] 0 2024-09-21T04:15:18.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/526: fdatasync d4/d12/d16/f19 0 2024-09-21T04:15:18.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/527: stat d4/d12/f41 0 2024-09-21T04:15:18.385 INFO:tasks.workunit.client.0.smithi110.stdout:2/343: mkdir d2/d1a/d1f/d65 0 2024-09-21T04:15:18.385 INFO:tasks.workunit.client.0.smithi110.stdout:2/344: read d2/d1a/d1f/d5f/f4a [215492,114729] 0 2024-09-21T04:15:18.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/471: creat d3/d34/d3c/fa8 x:0 0 0 2024-09-21T04:15:18.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/436: symlink d7/d10/d24/d3c/d14/d5c/d6f/l95 0 2024-09-21T04:15:18.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/506: symlink d5/d14/lb0 0 2024-09-21T04:15:18.388 INFO:tasks.workunit.client.0.smithi110.stdout:4/668: symlink d17/d3e/le5 0 2024-09-21T04:15:18.390 INFO:tasks.workunit.client.1.smithi148.stdout:6/508: creat d0/d1a/d23/d53/f90 x:0 0 0 2024-09-21T04:15:18.390 INFO:tasks.workunit.client.1.smithi148.stdout:6/509: chown d0/d34/d38/d43/d63/f7c 26344837 1 2024-09-21T04:15:18.392 INFO:tasks.workunit.client.1.smithi148.stdout:0/509: rename d5/d6/d2a/d3f/d6d/l8c to d5/d6/d2a/la0 0 2024-09-21T04:15:18.392 INFO:tasks.workunit.client.1.smithi148.stdout:0/510: write d5/d6/d2a/f79 [178326,129637] 0 2024-09-21T04:15:18.393 INFO:tasks.workunit.client.1.smithi148.stdout:0/511: chown d5/d6/f15 5 1 2024-09-21T04:15:18.393 INFO:tasks.workunit.client.1.smithi148.stdout:0/512: write d5/d6/f88 [366785,19811] 0 2024-09-21T04:15:18.394 INFO:tasks.workunit.client.0.smithi110.stdout:1/825: mkdir d2/d9/d49/d5c/d91/d101 0 2024-09-21T04:15:18.396 INFO:tasks.workunit.client.0.smithi110.stdout:4/669: dread d17/d1e/f33 [0,4194304] 0 2024-09-21T04:15:18.396 INFO:tasks.workunit.client.0.smithi110.stdout:4/670: fdatasync d17/f39 0 2024-09-21T04:15:18.397 INFO:tasks.workunit.client.0.smithi110.stdout:6/551: dwrite d4/f8c [0,4194304] 0 2024-09-21T04:15:18.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/780: write d4/d47/fde [1765225,109859] 0 2024-09-21T04:15:18.398 INFO:tasks.workunit.client.1.smithi148.stdout:3/461: creat d3/d1f/d4d/d82/d23/d28/d65/f8f x:0 0 0 2024-09-21T04:15:18.398 INFO:tasks.workunit.client.1.smithi148.stdout:3/462: fsync d3/f49 0 2024-09-21T04:15:18.398 INFO:tasks.workunit.client.1.smithi148.stdout:3/463: chown d3/d1f/d4d/d82/d21 121469 1 2024-09-21T04:15:18.401 INFO:tasks.workunit.client.0.smithi110.stdout:2/345: rename d2/d1a/d1f/d2a/c4e to d2/d1a/d1f/d29/d19/d62/c66 0 2024-09-21T04:15:18.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 -- 172.21.15.110:0/1294156477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74000760b0 msgr2=0x7f7400076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 --2- 172.21.15.110:0/1294156477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74000760b0 0x7f7400076510 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73f00097e0 tx=0x7f73f002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 -- 172.21.15.110:0/1294156477 shutdown_connections 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 --2- 172.21.15.110:0/1294156477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74000760b0 0x7f7400076510 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 --2- 172.21.15.110:0/1294156477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7400074e60 0x7f7400075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.400+0000 7f7404f80640 1 -- 172.21.15.110:0/1294156477 >> 172.21.15.110:0/1294156477 conn(0x7f74000fd550 msgr2=0x7f74000ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:18.404 INFO:tasks.workunit.client.0.smithi110.stdout:4/671: dread d17/d34/d3d/fa6 [0,4194304] 0 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.401+0000 7f7404f80640 1 -- 172.21.15.110:0/1294156477 shutdown_connections 2024-09-21T04:15:18.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.401+0000 7f7404f80640 1 -- 172.21.15.110:0/1294156477 wait complete. 2024-09-21T04:15:18.405 INFO:tasks.workunit.client.0.smithi110.stdout:9/437: mkdir d7/d96 0 2024-09-21T04:15:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 Processor -- start 2024-09-21T04:15:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 -- start start 2024-09-21T04:15:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74000760b0 0x7f740019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740019c740 con 0x7f7400074e60 2024-09-21T04:15:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f7404f80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740019c8b0 con 0x7f74000760b0 2024-09-21T04:15:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f73fe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:18.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f73fe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48082/0 (socket says 172.21.15.110:48082) 2024-09-21T04:15:18.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.403+0000 7f73fe59c640 1 -- 172.21.15.110:0/2568174832 learned_addr learned my addr 172.21.15.110:0/2568174832 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:18.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.404+0000 7f73fe59c640 1 -- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74000760b0 msgr2=0x7f740019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.404+0000 7f73fe59c640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74000760b0 0x7f740019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.404+0000 7f73fe59c640 1 -- 172.21.15.110:0/2568174832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009490 con 0x7f7400074e60 2024-09-21T04:15:18.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.404+0000 7f73fe59c640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e970 tx=0x7f73f400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:18.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.405+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f400cc90 con 0x7f7400074e60 2024-09-21T04:15:18.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.405+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7400069ff0 con 0x7f7400074e60 2024-09-21T04:15:18.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.405+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740006a540 con 0x7f7400074e60 2024-09-21T04:15:18.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.406+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73f400cdf0 con 0x7f7400074e60 2024-09-21T04:15:18.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.407+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f4004e40 con 0x7f7400074e60 2024-09-21T04:15:18.410 INFO:tasks.workunit.client.0.smithi110.stdout:1/826: symlink d2/d9/d49/d7a/dee/de1/l102 0 2024-09-21T04:15:18.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.408+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e0005250 con 0x7f7400074e60 2024-09-21T04:15:18.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.409+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 51991+0+0 (secure 0 0 0) 0x7f73f4021440 con 0x7f7400074e60 2024-09-21T04:15:18.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.409+0000 7f73ef7fe640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 0x7f73e8042060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:18.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.409+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f73f4014070 con 0x7f7400074e60 2024-09-21T04:15:18.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.410+0000 7f73fdd9b640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 0x7f73e8042060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:18.413 INFO:tasks.workunit.client.1.smithi148.stdout:8/508: creat dc/d13/d28/d62/f97 x:0 0 0 2024-09-21T04:15:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.413+0000 7f73fdd9b640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 0x7f73e8042060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73f00308d0 tx=0x7f73f0030520 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:18.417 INFO:tasks.workunit.client.0.smithi110.stdout:6/552: symlink d4/d12/d16/d66/d7a/d4b/d7f/laf 0 2024-09-21T04:15:18.418 INFO:tasks.workunit.client.1.smithi148.stdout:4/528: creat d4/d66/d94/f96 x:0 0 0 2024-09-21T04:15:18.419 INFO:tasks.workunit.client.0.smithi110.stdout:2/346: mknod d2/d1a/d1f/d65/c67 0 2024-09-21T04:15:18.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.417+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f73f4021970 con 0x7f7400074e60 2024-09-21T04:15:18.424 INFO:tasks.workunit.client.1.smithi148.stdout:5/472: dread d3/d14/d1f/d26/f94 [0,4194304] 0 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:1/827: dread d2/f45 [0,4194304] 0 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:0/781: dwrite d4/d1c/d62/f90 [0,4194304] 0 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:9/438: dwrite d7/d10/d24/d3c/d1f/f41 [0,4194304] 0 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:9/439: chown d7/d10/d24/d3c/d4f 236 1 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:4/672: dwrite d17/d3e/f40 [4194304,4194304] 0 2024-09-21T04:15:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:4/673: fdatasync d17/d34/d3d/f41 0 2024-09-21T04:15:18.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/507: mknod d5/d14/d25/d40/dab/cb1 0 2024-09-21T04:15:18.438 INFO:tasks.workunit.client.1.smithi148.stdout:6/510: unlink d0/d59/f7d 0 2024-09-21T04:15:18.438 INFO:tasks.workunit.client.1.smithi148.stdout:6/511: read - d0/d1a/d23/d27/d62/f67 zero size 2024-09-21T04:15:18.439 INFO:tasks.workunit.client.0.smithi110.stdout:4/674: dwrite d17/d34/d3d/fa3 [4194304,4194304] 0 2024-09-21T04:15:18.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/498: rename d5/d60/d28/d38 to d5/d2b/dbe 0 2024-09-21T04:15:18.441 INFO:tasks.workunit.client.0.smithi110.stdout:6/553: truncate d4/d12/d17/d33/d3e/f53 5120334 0 2024-09-21T04:15:18.444 INFO:tasks.workunit.client.0.smithi110.stdout:2/347: creat d2/d1a/d1f/d65/f68 x:0 0 0 2024-09-21T04:15:18.446 INFO:tasks.workunit.client.1.smithi148.stdout:3/464: symlink d3/d1f/d4d/d82/d23/d28/d65/l90 0 2024-09-21T04:15:18.446 INFO:tasks.workunit.client.1.smithi148.stdout:3/465: write d3/d5/d6/d12/d15/d73/f7c [309208,70363] 0 2024-09-21T04:15:18.446 INFO:tasks.workunit.client.1.smithi148.stdout:3/466: fdatasync d3/d5/d6/d12/d43/d54/f71 0 2024-09-21T04:15:18.446 INFO:tasks.workunit.client.0.smithi110.stdout:1/828: mkdir d2/d9/d49/d5c/d91/d101/d103 0 2024-09-21T04:15:18.467 INFO:tasks.workunit.client.0.smithi110.stdout:0/782: truncate d4/d47/f23 1391535 0 2024-09-21T04:15:18.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/499: dwrite d5/d39/f6f [0,4194304] 0 2024-09-21T04:15:18.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/500: truncate d5/d1e/d37/fb0 187122 0 2024-09-21T04:15:18.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/501: chown d5/d2b/d32/f82 470470 1 2024-09-21T04:15:18.474 INFO:tasks.workunit.client.0.smithi110.stdout:9/440: rename d7/d10/d24/d3c/d14/d5c/d6f/d78/d7d/c89 to d7/d10/d24/d3c/d1f/d72/c97 0 2024-09-21T04:15:18.474 INFO:tasks.workunit.client.0.smithi110.stdout:9/441: chown d7/d10/d24/d3c/d1f/d43/f6e 61566306 1 2024-09-21T04:15:18.477 INFO:tasks.workunit.client.0.smithi110.stdout:4/675: creat d17/d3e/dbd/fe6 x:0 0 0 2024-09-21T04:15:18.478 INFO:tasks.workunit.client.0.smithi110.stdout:4/676: fdatasync d17/d34/fb0 0 2024-09-21T04:15:18.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/554: mkdir d4/d3c/db0 0 2024-09-21T04:15:18.481 INFO:tasks.workunit.client.0.smithi110.stdout:2/348: mknod d2/d1a/c69 0 2024-09-21T04:15:18.481 INFO:tasks.workunit.client.0.smithi110.stdout:2/349: write d2/d1a/d1f/d5f/f58 [147932,87500] 0 2024-09-21T04:15:18.491 INFO:tasks.workunit.client.1.smithi148.stdout:4/529: unlink d4/d66/d7f/l7b 0 2024-09-21T04:15:18.493 INFO:tasks.workunit.client.1.smithi148.stdout:6/512: mknod d0/d34/d38/c91 0 2024-09-21T04:15:18.495 INFO:tasks.workunit.client.1.smithi148.stdout:2/596: rename d0/d9/dc/l8c to d0/d9/d16/d9e/lb4 0 2024-09-21T04:15:18.496 INFO:tasks.workunit.client.1.smithi148.stdout:9/502: mkdir d5/d60/d28/d2f/dbf 0 2024-09-21T04:15:18.504 INFO:tasks.workunit.client.1.smithi148.stdout:4/530: read d4/f10 [638730,106402] 0 2024-09-21T04:15:18.505 INFO:tasks.workunit.client.0.smithi110.stdout:9/442: mknod d7/d10/d24/d3c/d14/d31/c98 0 2024-09-21T04:15:18.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/597: read d0/f68 [577244,112245] 0 2024-09-21T04:15:18.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/598: creat d0/d9/dc/d13/d50/fb5 x:0 0 0 2024-09-21T04:15:18.509 INFO:tasks.workunit.client.0.smithi110.stdout:6/555: mknod d4/d86/cb1 0 2024-09-21T04:15:18.511 INFO:tasks.workunit.client.0.smithi110.stdout:0/783: rmdir d4/dd/d10/d29/d5f/dc3 39 2024-09-21T04:15:18.515 INFO:tasks.workunit.client.0.smithi110.stdout:2/350: dwrite d2/d1a/d1f/d5f/f24 [0,4194304] 0 2024-09-21T04:15:18.516 INFO:tasks.workunit.client.1.smithi148.stdout:9/503: dwrite d5/fe [0,4194304] 0 2024-09-21T04:15:18.516 INFO:tasks.workunit.client.1.smithi148.stdout:9/504: truncate d5/d1e/d37/d46/f6d 928320 0 2024-09-21T04:15:18.516 INFO:tasks.workunit.client.1.smithi148.stdout:9/505: dread - d5/d60/d28/f51 zero size 2024-09-21T04:15:18.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/677: rmdir d17/d34/d3d/d20/dca 39 2024-09-21T04:15:18.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/678: chown d17/d34/d3d/d20/d43/d51/d7a 30669 1 2024-09-21T04:15:18.519 INFO:tasks.workunit.client.1.smithi148.stdout:2/599: dread d0/d2b/f5e [0,4194304] 0 2024-09-21T04:15:18.519 INFO:tasks.workunit.client.0.smithi110.stdout:2/351: dread d2/f54 [0,4194304] 0 2024-09-21T04:15:18.523 INFO:tasks.workunit.client.1.smithi148.stdout:8/509: rmdir dc 39 2024-09-21T04:15:18.523 INFO:tasks.workunit.client.1.smithi148.stdout:8/510: truncate dc/d13/d3c/f93 852902 0 2024-09-21T04:15:18.526 INFO:tasks.workunit.client.1.smithi148.stdout:6/513: mkdir d0/d1a/d23/d16/d36/d92 0 2024-09-21T04:15:18.528 INFO:tasks.workunit.client.1.smithi148.stdout:1/469: rename db/d24/d2e/d59/l84 to db/d46/l98 0 2024-09-21T04:15:18.528 INFO:tasks.workunit.client.0.smithi110.stdout:2/352: symlink d2/d1a/d1f/d29/d19/l6a 0 2024-09-21T04:15:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:2/353: fsync d2/d1a/d1f/d5f/f24 0 2024-09-21T04:15:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:2/354: stat d2/d1a/d1f/c37 0 2024-09-21T04:15:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:0/784: dwrite d4/dd/d10/fa1 [0,4194304] 0 2024-09-21T04:15:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:0/785: fdatasync d4/dd/d10/d29/d5f/dc3/fe9 0 2024-09-21T04:15:18.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/531: link d4/f51 d4/d66/d7f/f97 0 2024-09-21T04:15:18.530 INFO:tasks.workunit.client.0.smithi110.stdout:9/443: rename d7/d10/d24 to d7/d96/d99 0 2024-09-21T04:15:18.531 INFO:tasks.workunit.client.0.smithi110.stdout:2/355: rmdir d2/d1a/d1f/d29/d19/d62 39 2024-09-21T04:15:18.535 INFO:tasks.workunit.client.1.smithi148.stdout:2/600: mknod d0/d2b/da3/d82/cb6 0 2024-09-21T04:15:18.536 INFO:tasks.workunit.client.0.smithi110.stdout:0/786: creat d4/dd/d60/ff3 x:0 0 0 2024-09-21T04:15:18.536 INFO:tasks.workunit.client.0.smithi110.stdout:0/787: truncate d4/d8b/fda 996393 0 2024-09-21T04:15:18.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/511: mkdir dc/d13/d3c/d81/d98 0 2024-09-21T04:15:18.537 INFO:tasks.workunit.client.1.smithi148.stdout:8/512: truncate dc/d43/d4c/d4f/f8e 259701 0 2024-09-21T04:15:18.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/679: dwrite d17/d34/d3d/d20/dca/fd1 [0,4194304] 0 2024-09-21T04:15:18.540 INFO:tasks.workunit.client.0.smithi110.stdout:6/556: rename d4/d12/d16/d66/f67 to d4/d12/d17/d9a/fb2 0 2024-09-21T04:15:18.540 INFO:tasks.workunit.client.0.smithi110.stdout:6/557: write d4/d12/d17/f8e [616005,61715] 0 2024-09-21T04:15:18.541 INFO:tasks.workunit.client.1.smithi148.stdout:6/514: mkdir d0/d1a/d23/d32/d93 0 2024-09-21T04:15:18.542 INFO:tasks.workunit.client.1.smithi148.stdout:6/515: fdatasync d0/d1a/d23/f14 0 2024-09-21T04:15:18.542 INFO:tasks.workunit.client.1.smithi148.stdout:6/516: read d0/f20 [290869,54713] 0 2024-09-21T04:15:18.544 INFO:tasks.workunit.client.0.smithi110.stdout:9/444: creat d7/d96/f9a x:0 0 0 2024-09-21T04:15:18.547 INFO:tasks.workunit.client.1.smithi148.stdout:7/508: rename d5/d8/d64/l66 to d5/lb2 0 2024-09-21T04:15:18.548 INFO:tasks.workunit.client.0.smithi110.stdout:2/356: mknod d2/d1a/d1f/c6b 0 2024-09-21T04:15:18.549 INFO:tasks.workunit.client.1.smithi148.stdout:1/470: symlink db/d46/d4c/l99 0 2024-09-21T04:15:18.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/788: mkdir d4/dd/d53/df4 0 2024-09-21T04:15:18.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/532: creat d4/f98 x:0 0 0 2024-09-21T04:15:18.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/533: chown d4/d66/d7f/f2e 188023 1 2024-09-21T04:15:18.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/534: chown d4/d12/d16/f4f 261517165 1 2024-09-21T04:15:18.555 INFO:tasks.workunit.client.1.smithi148.stdout:9/506: getdents d5/d1f 0 2024-09-21T04:15:18.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/517: truncate d0/f21 6043135 0 2024-09-21T04:15:18.557 INFO:tasks.workunit.client.1.smithi148.stdout:6/518: dread - d0/d1a/d23/d27/d62/f67 zero size 2024-09-21T04:15:18.558 INFO:tasks.workunit.client.0.smithi110.stdout:4/680: link d17/d34/d3d/d57/l8e d17/d34/d3d/le7 0 2024-09-21T04:15:18.561 INFO:tasks.workunit.client.1.smithi148.stdout:3/467: rename d3/d5/d6/d12/f75 to d3/d1f/d4d/d82/d74/f91 0 2024-09-21T04:15:18.561 INFO:tasks.workunit.client.1.smithi148.stdout:2/601: dwrite d0/d2b/d48/f58 [0,4194304] 0 2024-09-21T04:15:18.562 INFO:tasks.workunit.client.0.smithi110.stdout:6/558: creat d4/d12/d16/d66/d7a/d63/fb3 x:0 0 0 2024-09-21T04:15:18.563 INFO:tasks.workunit.client.0.smithi110.stdout:9/445: creat d7/d96/d99/f9b x:0 0 0 2024-09-21T04:15:18.565 INFO:tasks.workunit.client.1.smithi148.stdout:4/535: creat d4/d12/d75/d85/f99 x:0 0 0 2024-09-21T04:15:18.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/519: creat d0/d1a/d23/d27/d80/f94 x:0 0 0 2024-09-21T04:15:18.572 INFO:tasks.workunit.client.0.smithi110.stdout:4/681: unlink d17/d34/d3d/d20/d43/d51/d52/c75 0 2024-09-21T04:15:18.577 INFO:tasks.workunit.client.0.smithi110.stdout:9/446: creat d7/d96/d99/d3c/d35/f9c x:0 0 0 2024-09-21T04:15:18.580 INFO:tasks.workunit.client.0.smithi110.stdout:6/559: getdents d4/d12/d17 0 2024-09-21T04:15:18.580 INFO:tasks.workunit.client.1.smithi148.stdout:8/513: dwrite dc/d3e/f70 [0,4194304] 0 2024-09-21T04:15:18.580 INFO:tasks.workunit.client.1.smithi148.stdout:8/514: chown dc/d13/d76/d42/f90 151 1 2024-09-21T04:15:18.580 INFO:tasks.workunit.client.1.smithi148.stdout:8/515: chown dc/d13/l20 9690498 1 2024-09-21T04:15:18.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/468: symlink d3/d5/l92 0 2024-09-21T04:15:18.586 INFO:tasks.workunit.client.1.smithi148.stdout:2/602: stat d0/d2b/da3/d8f/laa 0 2024-09-21T04:15:18.586 INFO:tasks.workunit.client.1.smithi148.stdout:2/603: chown d0/d2b/da3/d8f/d9f 0 1 2024-09-21T04:15:18.586 INFO:tasks.workunit.client.0.smithi110.stdout:9/447: mknod d7/d96/d99/d3c/d1f/d72/d59/c9d 0 2024-09-21T04:15:18.587 INFO:tasks.workunit.client.0.smithi110.stdout:6/560: creat d4/d78/daa/fb4 x:0 0 0 2024-09-21T04:15:18.587 INFO:tasks.workunit.client.1.smithi148.stdout:7/509: truncate d5/d8/ff 3639045 0 2024-09-21T04:15:18.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/469: write d3/f49 [970043,13534] 0 2024-09-21T04:15:18.588 INFO:tasks.workunit.client.0.smithi110.stdout:9/448: creat d7/d96/d99/d3c/d1f/d25/f9e x:0 0 0 2024-09-21T04:15:18.589 INFO:tasks.workunit.client.0.smithi110.stdout:6/561: symlink d4/d78/daa/lb5 0 2024-09-21T04:15:18.592 INFO:tasks.workunit.client.0.smithi110.stdout:6/562: getdents d4/d86/d8f 0 2024-09-21T04:15:18.595 INFO:tasks.workunit.client.1.smithi148.stdout:2/604: dread d0/d2b/d48/f5c [0,4194304] 0 2024-09-21T04:15:18.598 INFO:tasks.workunit.client.0.smithi110.stdout:6/563: dread d4/d86/d8f/f9b [0,4194304] 0 2024-09-21T04:15:18.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/507: dwrite d5/d2b/dbe/f9a [0,4194304] 0 2024-09-21T04:15:18.599 INFO:tasks.workunit.client.0.smithi110.stdout:4/682: dwrite d17/d34/d3d/d20/fa8 [0,4194304] 0 2024-09-21T04:15:18.599 INFO:tasks.workunit.client.0.smithi110.stdout:4/683: stat d17/d1e/d21/fbe 0 2024-09-21T04:15:18.599 INFO:tasks.workunit.client.0.smithi110.stdout:0/789: dwrite d4/d1c/d62/f36 [0,4194304] 0 2024-09-21T04:15:18.602 INFO:tasks.workunit.client.1.smithi148.stdout:6/520: dwrite d0/d1a/d23/d16/d36/f7a [0,4194304] 0 2024-09-21T04:15:18.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/536: dwrite d4/d12/d16/f19 [0,4194304] 0 2024-09-21T04:15:18.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/537: truncate d4/d12/d16/d29/d61/d6c/f88 969209 0 2024-09-21T04:15:18.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/538: chown d4/d66/d7f/f4a 11189 1 2024-09-21T04:15:18.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/510: link d5/d14/d25/d40/dab/cb1 d5/daa/cb3 0 2024-09-21T04:15:18.603 INFO:tasks.workunit.client.0.smithi110.stdout:6/564: mkdir d4/d12/d17/d81/db6 0 2024-09-21T04:15:18.603 INFO:tasks.workunit.client.0.smithi110.stdout:6/565: chown d4/d12/d17/l35 44875683 1 2024-09-21T04:15:18.603 INFO:tasks.workunit.client.0.smithi110.stdout:6/566: read - d4/d12/d16/d66/fac zero size 2024-09-21T04:15:18.604 INFO:tasks.workunit.client.1.smithi148.stdout:3/470: mknod d3/d5/c93 0 2024-09-21T04:15:18.604 INFO:tasks.workunit.client.1.smithi148.stdout:3/471: write d3/d1f/f8e [801668,57131] 0 2024-09-21T04:15:18.605 INFO:tasks.workunit.client.1.smithi148.stdout:1/471: rename db/d24/d4f to db/d24/d2e/d71/d94/d9a 0 2024-09-21T04:15:18.611 INFO:tasks.workunit.client.0.smithi110.stdout:4/684: unlink d17/d3e/d6b/caa 0 2024-09-21T04:15:18.615 INFO:tasks.workunit.client.0.smithi110.stdout:0/790: creat d4/dd/d10/d29/d45/ff5 x:0 0 0 2024-09-21T04:15:18.618 INFO:tasks.workunit.client.1.smithi148.stdout:3/472: dwrite d3/d5/f7b [0,4194304] 0 2024-09-21T04:15:18.620 INFO:tasks.workunit.client.0.smithi110.stdout:6/567: mkdir d4/d12/d17/d33/d76/db7 0 2024-09-21T04:15:18.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/539: mkdir d4/d66/d7f/d9a 0 2024-09-21T04:15:18.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/540: chown d4/d66/d7f/d2a/c90 151693057 1 2024-09-21T04:15:18.629 INFO:tasks.workunit.client.0.smithi110.stdout:6/568: write d4/d12/d17/d9a/fb2 [4100169,105963] 0 2024-09-21T04:15:18.631 INFO:tasks.workunit.client.1.smithi148.stdout:8/516: rename dc/d43/f96 to dc/d13/d3c/d81/d98/f99 0 2024-09-21T04:15:18.632 INFO:tasks.workunit.client.1.smithi148.stdout:0/513: sync 2024-09-21T04:15:18.633 INFO:tasks.workunit.client.1.smithi148.stdout:0/514: chown d5/d6/d2a/d3f/f4c 633589964 1 2024-09-21T04:15:18.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/508: rmdir d5/d60/d69 39 2024-09-21T04:15:18.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/509: dread - d5/d1e/f90 zero size 2024-09-21T04:15:18.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/510: readlink d5/d39/l61 0 2024-09-21T04:15:18.637 INFO:tasks.workunit.client.1.smithi148.stdout:9/511: write d5/d60/d28/f6a [492505,57] 0 2024-09-21T04:15:18.638 INFO:tasks.workunit.client.0.smithi110.stdout:0/791: dwrite d4/dd/d10/d29/d5f/dc3/fe9 [0,4194304] 0 2024-09-21T04:15:18.638 INFO:tasks.workunit.client.0.smithi110.stdout:4/685: dwrite d17/d1e/f48 [0,4194304] 0 2024-09-21T04:15:18.638 INFO:tasks.workunit.client.0.smithi110.stdout:4/686: stat d17/d34/d3d/d20/d43/d51/ca0 0 2024-09-21T04:15:18.641 INFO:tasks.workunit.client.1.smithi148.stdout:3/473: symlink d3/d1f/l94 0 2024-09-21T04:15:18.641 INFO:tasks.workunit.client.1.smithi148.stdout:3/474: write d3/d5/d6/d12/d43/d54/f6a [5006308,124675] 0 2024-09-21T04:15:18.641 INFO:tasks.workunit.client.0.smithi110.stdout:4/687: dread d17/d1e/d64/f8f [0,4194304] 0 2024-09-21T04:15:18.642 INFO:tasks.workunit.client.1.smithi148.stdout:4/541: symlink d4/d87/l9b 0 2024-09-21T04:15:18.646 INFO:tasks.workunit.client.1.smithi148.stdout:7/511: getdents d5/d8/d64 0 2024-09-21T04:15:18.646 INFO:tasks.workunit.client.0.smithi110.stdout:6/569: dwrite d4/d12/d17/d2e/f47 [0,4194304] 0 2024-09-21T04:15:18.647 INFO:tasks.workunit.client.0.smithi110.stdout:6/570: dread - d4/d12/d16/d66/fac zero size 2024-09-21T04:15:18.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/473: sync 2024-09-21T04:15:18.648 INFO:tasks.workunit.client.0.smithi110.stdout:0/792: rename d4/d47/d2c/l44 to d4/dd/d60/lf6 0 2024-09-21T04:15:18.652 INFO:tasks.workunit.client.1.smithi148.stdout:8/517: link dc/c11 dc/d13/d3c/c9a 0 2024-09-21T04:15:18.652 INFO:tasks.workunit.client.1.smithi148.stdout:8/518: write dc/d13/d3c/f49 [1452224,20314] 0 2024-09-21T04:15:18.654 INFO:tasks.workunit.client.1.smithi148.stdout:3/475: write d3/d5/d6/f17 [4440456,93961] 0 2024-09-21T04:15:18.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/515: creat d5/d6/d2a/d3f/d8b/fa1 x:0 0 0 2024-09-21T04:15:18.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/516: chown d5/d6/d3e/f49 1793 1 2024-09-21T04:15:18.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/517: chown d5/d6/d7/d18/f32 2039262 1 2024-09-21T04:15:18.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/518: write d5/d6/d2a/d45/f6e [660602,60748] 0 2024-09-21T04:15:18.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/519: truncate d5/d82/f87 607297 0 2024-09-21T04:15:18.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/520: dread - d5/d6/d7/d22/f81 zero size 2024-09-21T04:15:18.659 INFO:tasks.workunit.client.0.smithi110.stdout:6/571: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:18.659 INFO:tasks.workunit.client.0.smithi110.stdout:6/572: stat d4/d12/d16/d66/d7a/d63 0 2024-09-21T04:15:18.659 INFO:tasks.workunit.client.0.smithi110.stdout:6/573: write d4/d12/d17/f8e [1281970,3855] 0 2024-09-21T04:15:18.661 INFO:tasks.workunit.client.1.smithi148.stdout:9/512: link d5/d60/d28/c5c d5/d2b/d4e/db3/cc0 0 2024-09-21T04:15:18.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/513: creat d5/d39/fc1 x:0 0 0 2024-09-21T04:15:18.662 INFO:tasks.workunit.client.0.smithi110.stdout:6/574: dread d4/d12/d16/f56 [0,4194304] 0 2024-09-21T04:15:18.662 INFO:tasks.workunit.client.0.smithi110.stdout:6/575: truncate d4/f13 6246084 0 2024-09-21T04:15:18.665 INFO:tasks.workunit.client.1.smithi148.stdout:4/542: mkdir d4/d12/d75/d9c 0 2024-09-21T04:15:18.666 INFO:tasks.workunit.client.0.smithi110.stdout:4/688: dwrite d17/f1a [0,4194304] 0 2024-09-21T04:15:18.666 INFO:tasks.workunit.client.0.smithi110.stdout:0/793: dread d4/d47/d2c/d3a/f9f [0,4194304] 0 2024-09-21T04:15:18.667 INFO:tasks.workunit.client.0.smithi110.stdout:6/576: mknod d4/d12/d17/d2e/cb8 0 2024-09-21T04:15:18.668 INFO:tasks.workunit.client.0.smithi110.stdout:6/577: chown d4/d78/daa/l19 26315204 1 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:8/519: creat dc/d13/f9b x:0 0 0 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/521: getdents d5/d82 0 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/522: readlink d5/d6/d2a/d2b/d84/l91 0 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/523: dread - d5/d6/f5a zero size 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/524: write d5/d6/d2a/d3f/f41 [2167407,15797] 0 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/525: dread - d5/d6/d7/d22/f8a zero size 2024-09-21T04:15:18.673 INFO:tasks.workunit.client.0.smithi110.stdout:6/578: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:18.674 INFO:tasks.workunit.client.0.smithi110.stdout:6/579: creat d4/d12/d17/d2e/fb9 x:0 0 0 2024-09-21T04:15:18.675 INFO:tasks.workunit.client.0.smithi110.stdout:4/689: rmdir d17/d34/d3d/d20/d43 39 2024-09-21T04:15:18.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/514: mkdir d5/d60/d28/d2f/dbf/dc2 0 2024-09-21T04:15:18.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/515: chown d5/d2b/d32/d67/l74 44081 1 2024-09-21T04:15:18.677 INFO:tasks.workunit.client.0.smithi110.stdout:0/794: getdents d4 0 2024-09-21T04:15:18.678 INFO:tasks.workunit.client.0.smithi110.stdout:0/795: dread - d4/dd/d53/fc5 zero size 2024-09-21T04:15:18.678 INFO:tasks.workunit.client.1.smithi148.stdout:4/543: write d4/d12/d16/f19 [3267855,60409] 0 2024-09-21T04:15:18.678 INFO:tasks.workunit.client.1.smithi148.stdout:4/544: fdatasync d4/d66/d94/f96 0 2024-09-21T04:15:18.678 INFO:tasks.workunit.client.1.smithi148.stdout:4/545: write d4/d12/d16/f1f [2608155,52448] 0 2024-09-21T04:15:18.680 INFO:tasks.workunit.client.1.smithi148.stdout:3/476: dwrite d3/d1f/d4d/d82/d23/d28/d65/f67 [8388608,4194304] 0 2024-09-21T04:15:18.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/477: write d3/f55 [950496,90252] 0 2024-09-21T04:15:18.681 INFO:tasks.workunit.client.1.smithi148.stdout:5/474: dwrite d3/d14/d1f/d26/d2e/f8f [0,4194304] 0 2024-09-21T04:15:18.681 INFO:tasks.workunit.client.1.smithi148.stdout:7/512: link d5/d8/f16 d5/d14/d25/d40/dab/dad/fb4 0 2024-09-21T04:15:18.685 INFO:tasks.workunit.client.1.smithi148.stdout:8/520: creat dc/d13/d3c/f9c x:0 0 0 2024-09-21T04:15:18.686 INFO:tasks.workunit.client.1.smithi148.stdout:8/521: write dc/d3e/f70 [4198495,41351] 0 2024-09-21T04:15:18.686 INFO:tasks.workunit.client.1.smithi148.stdout:8/522: fsync dc/f17 0 2024-09-21T04:15:18.686 INFO:tasks.workunit.client.1.smithi148.stdout:8/523: dread - dc/d13/d76/d42/f71 zero size 2024-09-21T04:15:18.686 INFO:tasks.workunit.client.0.smithi110.stdout:6/580: getdents d4/d86 0 2024-09-21T04:15:18.689 INFO:tasks.workunit.client.1.smithi148.stdout:4/546: write d4/d12/d16/d29/f5a [3800835,95640] 0 2024-09-21T04:15:18.689 INFO:tasks.workunit.client.1.smithi148.stdout:4/547: dread - d4/d12/d75/d85/f99 zero size 2024-09-21T04:15:18.691 INFO:tasks.workunit.client.0.smithi110.stdout:4/690: creat d17/d34/d3d/d20/dca/fe8 x:0 0 0 2024-09-21T04:15:18.693 INFO:tasks.workunit.client.1.smithi148.stdout:0/526: rename d5/d6/d7/f4b to d5/d6/d7/d22/fa2 0 2024-09-21T04:15:18.694 INFO:tasks.workunit.client.0.smithi110.stdout:6/581: mkdir d4/d12/d17/d9a/dba 0 2024-09-21T04:15:18.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/516: creat d5/d60/d28/d73/fc3 x:0 0 0 2024-09-21T04:15:18.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/517: write d5/d2b/d32/f58 [461171,87215] 0 2024-09-21T04:15:18.697 INFO:tasks.workunit.client.0.smithi110.stdout:4/691: creat d17/fe9 x:0 0 0 2024-09-21T04:15:18.699 INFO:tasks.workunit.client.0.smithi110.stdout:4/692: rename d17/f58 to d17/d34/d3d/d20/d43/d51/d52/fea 0 2024-09-21T04:15:18.700 INFO:tasks.workunit.client.1.smithi148.stdout:3/478: creat d3/d84/f95 x:0 0 0 2024-09-21T04:15:18.702 INFO:tasks.workunit.client.0.smithi110.stdout:4/693: truncate f4 2771106 0 2024-09-21T04:15:18.702 INFO:tasks.workunit.client.0.smithi110.stdout:4/694: dread - d17/d1e/d21/fd4 zero size 2024-09-21T04:15:18.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/513: dwrite d5/d8/d6f/d7d/f7f [0,4194304] 0 2024-09-21T04:15:18.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/514: creat d5/d8/d6f/d7d/d90/fb5 x:0 0 0 2024-09-21T04:15:18.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/515: chown d5/d8/d6f/d7d/l61 1409216273 1 2024-09-21T04:15:18.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/796: dwrite d4/dd/f1b [0,4194304] 0 2024-09-21T04:15:18.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/797: chown d4/dd/f32 11614 1 2024-09-21T04:15:18.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/798: write d4/dd/d60/ff3 [727688,57306] 0 2024-09-21T04:15:18.706 INFO:tasks.workunit.client.1.smithi148.stdout:5/475: creat d3/d34/d3c/d5d/d75/fa9 x:0 0 0 2024-09-21T04:15:18.706 INFO:tasks.workunit.client.0.smithi110.stdout:6/582: dread d4/f3d [0,4194304] 0 2024-09-21T04:15:18.708 INFO:tasks.workunit.client.0.smithi110.stdout:6/583: dread d4/d12/d16/d66/d7a/d4b/f97 [0,4194304] 0 2024-09-21T04:15:18.711 INFO:tasks.workunit.client.0.smithi110.stdout:6/584: link d4/d78/daa/l19 d4/d12/d16/d66/lbb 0 2024-09-21T04:15:18.712 INFO:tasks.workunit.client.0.smithi110.stdout:4/695: dread d17/d34/d3d/d57/f5a [0,4194304] 0 2024-09-21T04:15:18.716 INFO:tasks.workunit.client.1.smithi148.stdout:4/548: mkdir d4/d12/d16/d29/d61/d6c/d9d 0 2024-09-21T04:15:18.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/696: symlink d17/d34/d3d/d20/leb 0 2024-09-21T04:15:18.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/697: stat d17/d34/d3d/f4a 0 2024-09-21T04:15:18.719 INFO:tasks.workunit.client.1.smithi148.stdout:8/524: rename dc/d13/d76/d5d/d8f to dc/d13/d3c/d81/d9d 0 2024-09-21T04:15:18.719 INFO:tasks.workunit.client.1.smithi148.stdout:8/525: fdatasync dc/d13/d39/d6e/f6f 0 2024-09-21T04:15:18.721 INFO:tasks.workunit.client.1.smithi148.stdout:9/518: link d5/d60/d4c/l9b d5/d1f/lc4 0 2024-09-21T04:15:18.721 INFO:tasks.workunit.client.0.smithi110.stdout:0/799: dwrite d4/dd/d60/f7e [0,4194304] 0 2024-09-21T04:15:18.723 INFO:tasks.workunit.client.1.smithi148.stdout:7/516: getdents d5/d8/d6f/d7d/d90/d9b 0 2024-09-21T04:15:18.737 INFO:tasks.workunit.client.0.smithi110.stdout:0/800: mknod d4/dd/d60/d72/cf7 0 2024-09-21T04:15:18.739 INFO:tasks.workunit.client.1.smithi148.stdout:4/549: stat d4/d12/d16/d29/c47 0 2024-09-21T04:15:18.739 INFO:tasks.workunit.client.1.smithi148.stdout:4/550: creat d4/d12/d16/d26/f9e x:0 0 0 2024-09-21T04:15:18.743 INFO:tasks.workunit.client.0.smithi110.stdout:4/698: dwrite d17/d34/d3d/d20/f88 [4194304,4194304] 0 2024-09-21T04:15:18.744 INFO:tasks.workunit.client.1.smithi148.stdout:0/527: rename d5/d6/d3e/d1e to d5/d6/d2a/d3f/d6d/da3 0 2024-09-21T04:15:18.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/517: dwrite d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:18.745 INFO:tasks.workunit.client.1.smithi148.stdout:8/526: mknod dc/d13/d76/d5d/c9e 0 2024-09-21T04:15:18.745 INFO:tasks.workunit.client.1.smithi148.stdout:8/527: dread - dc/d43/d4c/f8c zero size 2024-09-21T04:15:18.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/519: truncate d5/d1e/d37/f3b 6690726 0 2024-09-21T04:15:18.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/520: fsync d5/d60/f3c 0 2024-09-21T04:15:18.751 INFO:tasks.workunit.client.0.smithi110.stdout:0/801: dwrite d4/dd/d53/d66/fc1 [0,4194304] 0 2024-09-21T04:15:18.756 INFO:tasks.workunit.client.0.smithi110.stdout:4/699: link d17/d3e/l3f d17/d3e/d6b/lec 0 2024-09-21T04:15:18.757 INFO:tasks.workunit.client.1.smithi148.stdout:4/551: mknod d4/d12/d16/d29/d61/d6c/c9f 0 2024-09-21T04:15:18.757 INFO:tasks.workunit.client.1.smithi148.stdout:4/552: dread - d4/d12/d16/d26/f8a zero size 2024-09-21T04:15:18.757 INFO:tasks.workunit.client.0.smithi110.stdout:4/700: read d17/d34/d3d/fa6 [202636,95129] 0 2024-09-21T04:15:18.757 INFO:tasks.workunit.client.0.smithi110.stdout:4/701: chown d17/d34/d3d/d20/d43/d51/d7a/d96/fd8 1891130 1 2024-09-21T04:15:18.761 INFO:tasks.workunit.client.0.smithi110.stdout:0/802: mkdir d4/dd/d60/d72/df8 0 2024-09-21T04:15:18.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/528: dwrite d5/d6/d2a/d45/f50 [0,4194304] 0 2024-09-21T04:15:18.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/529: write d5/d6/d7/d22/f46 [6820767,76236] 0 2024-09-21T04:15:18.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/530: truncate d5/d6/d7/ff 4248878 0 2024-09-21T04:15:18.764 INFO:tasks.workunit.client.1.smithi148.stdout:5/476: rename d3/d14/d1f/d26/d2e/l7d to d3/d24/laa 0 2024-09-21T04:15:18.764 INFO:tasks.workunit.client.1.smithi148.stdout:5/477: getdents d3/d14/d1f/d26/d2e/d56/da7 0 2024-09-21T04:15:18.765 INFO:tasks.workunit.client.0.smithi110.stdout:0/803: symlink d4/dd/d10/d29/lf9 0 2024-09-21T04:15:18.766 INFO:tasks.workunit.client.1.smithi148.stdout:7/518: mknod d5/d8/d6f/cb6 0 2024-09-21T04:15:18.768 INFO:tasks.workunit.client.1.smithi148.stdout:8/528: unlink dc/d43/d4c/d4f/f79 0 2024-09-21T04:15:18.769 INFO:tasks.workunit.client.0.smithi110.stdout:0/804: creat d4/d1c/d62/d69/ffa x:0 0 0 2024-09-21T04:15:18.771 INFO:tasks.workunit.client.1.smithi148.stdout:9/521: unlink d5/d1e/d37/d50/l66 0 2024-09-21T04:15:18.773 INFO:tasks.workunit.client.1.smithi148.stdout:4/553: creat d4/d12/d16/d29/d61/d7e/fa0 x:0 0 0 2024-09-21T04:15:18.776 INFO:tasks.workunit.client.1.smithi148.stdout:7/519: dread d5/d8/d6f/d7d/d90/f5d [4194304,4194304] 0 2024-09-21T04:15:18.776 INFO:tasks.workunit.client.1.smithi148.stdout:7/520: chown d5/d14/c3f 15 1 2024-09-21T04:15:18.778 INFO:tasks.workunit.client.1.smithi148.stdout:5/478: creat d3/d14/d1f/d26/d2e/d56/da7/fab x:0 0 0 2024-09-21T04:15:18.779 INFO:tasks.workunit.client.0.smithi110.stdout:0/805: write d4/dd/d10/d29/d45/fca [2862017,64660] 0 2024-09-21T04:15:18.780 INFO:tasks.workunit.client.0.smithi110.stdout:0/806: chown d4/dd/d10/d29/d5f/fd8 0 1 2024-09-21T04:15:18.780 INFO:tasks.workunit.client.0.smithi110.stdout:0/807: chown d4/d8b/fb4 14426 1 2024-09-21T04:15:18.780 INFO:tasks.workunit.client.0.smithi110.stdout:0/808: readlink d4/dd/d10/d29/d5f/dc3/le7 0 2024-09-21T04:15:18.780 INFO:tasks.workunit.client.0.smithi110.stdout:0/809: write d4/fbc [452046,33198] 0 2024-09-21T04:15:18.780 INFO:tasks.workunit.client.1.smithi148.stdout:8/529: creat dc/d13/d76/d5d/d78/f9f x:0 0 0 2024-09-21T04:15:18.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/554: mknod d4/d12/d75/ca1 0 2024-09-21T04:15:18.783 INFO:tasks.workunit.client.1.smithi148.stdout:4/555: chown d4/d66/d7f/d2a 3455148 1 2024-09-21T04:15:18.786 INFO:tasks.workunit.client.1.smithi148.stdout:7/521: symlink d5/d8/d6f/d7d/d90/lb7 0 2024-09-21T04:15:18.788 INFO:tasks.workunit.client.0.smithi110.stdout:0/810: unlink d4/d1c/ld1 0 2024-09-21T04:15:18.788 INFO:tasks.workunit.client.0.smithi110.stdout:0/811: creat d4/dd/d53/ffb x:0 0 0 2024-09-21T04:15:18.789 INFO:tasks.workunit.client.1.smithi148.stdout:0/531: getdents d5/d6/d2a/d45 0 2024-09-21T04:15:18.792 INFO:tasks.workunit.client.0.smithi110.stdout:0/812: write d4/d47/f23 [632797,85386] 0 2024-09-21T04:15:18.792 INFO:tasks.workunit.client.1.smithi148.stdout:5/479: symlink d3/d14/d1f/d26/d2e/d56/lac 0 2024-09-21T04:15:18.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/522: rename d5/fc to d5/d1f/d81/d9e/da2/fc5 0 2024-09-21T04:15:18.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/523: getdents d5/d60/d28/d2f/dbf/dc2 0 2024-09-21T04:15:18.796 INFO:tasks.workunit.client.1.smithi148.stdout:4/556: dread d4/d66/d7f/d2a/f36 [0,4194304] 0 2024-09-21T04:15:18.800 INFO:tasks.workunit.client.1.smithi148.stdout:0/532: rmdir d5/d6/d2a/d45/d70 39 2024-09-21T04:15:18.803 INFO:tasks.workunit.client.1.smithi148.stdout:8/530: dwrite dc/d13/d76/f57 [0,4194304] 0 2024-09-21T04:15:18.803 INFO:tasks.workunit.client.1.smithi148.stdout:9/524: rename d5/d60/d4c/f6b to d5/d1e/d37/fc6 0 2024-09-21T04:15:18.804 INFO:tasks.workunit.client.1.smithi148.stdout:9/525: dread d5/d1e/d37/f8d [0,4194304] 0 2024-09-21T04:15:18.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/526: read d5/d1e/d37/d46/f6d [556310,86625] 0 2024-09-21T04:15:18.805 INFO:tasks.workunit.client.1.smithi148.stdout:0/533: unlink d5/d6/d2a/d3f/d8b/fa1 0 2024-09-21T04:15:18.805 INFO:tasks.workunit.client.1.smithi148.stdout:4/557: read d4/d12/d16/d29/f5a [8136327,4530] 0 2024-09-21T04:15:18.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/480: truncate d3/f1b 795702 0 2024-09-21T04:15:18.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/481: readlink d3/d24/l55 0 2024-09-21T04:15:18.813 INFO:tasks.workunit.client.1.smithi148.stdout:8/531: creat dc/d3e/fa0 x:0 0 0 2024-09-21T04:15:18.814 INFO:tasks.workunit.client.1.smithi148.stdout:4/558: write d4/d12/f5b [203694,115879] 0 2024-09-21T04:15:18.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/534: symlink d5/d6/d2a/d2b/d84/la4 0 2024-09-21T04:15:18.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/535: chown d5/d6/f88 2 1 2024-09-21T04:15:18.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/536: chown d5/d6/d7/d11/d48 4293853 1 2024-09-21T04:15:18.817 INFO:tasks.workunit.client.1.smithi148.stdout:5/482: mkdir d3/d24/dad 0 2024-09-21T04:15:18.817 INFO:tasks.workunit.client.1.smithi148.stdout:5/483: chown d3/d14/d1a/f4d 30886794 1 2024-09-21T04:15:18.817 INFO:tasks.workunit.client.1.smithi148.stdout:5/484: readlink d3/d14/d1f/d26/d2e/d3d/l9e 0 2024-09-21T04:15:18.817 INFO:tasks.workunit.client.1.smithi148.stdout:5/485: write d3/d14/d1a/f57 [957597,107560] 0 2024-09-21T04:15:18.820 INFO:tasks.workunit.client.1.smithi148.stdout:8/532: mkdir dc/d13/d3c/d81/d85/da1 0 2024-09-21T04:15:18.820 INFO:tasks.workunit.client.1.smithi148.stdout:8/533: write dc/d13/d3c/f93 [582864,123597] 0 2024-09-21T04:15:18.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/559: read d4/d66/d7f/f4a [1445469,64637] 0 2024-09-21T04:15:18.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/527: getdents d5 0 2024-09-21T04:15:18.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/528: creat d5/d60/d28/fc7 x:0 0 0 2024-09-21T04:15:18.824 INFO:tasks.workunit.client.1.smithi148.stdout:0/537: chown d5/d6/d2a/d45/d70/f75 426 1 2024-09-21T04:15:18.830 INFO:tasks.workunit.client.1.smithi148.stdout:8/534: rename dc/c32 to dc/d13/d76/d42/ca2 0 2024-09-21T04:15:18.831 INFO:tasks.workunit.client.1.smithi148.stdout:4/560: truncate d4/fe 1558101 0 2024-09-21T04:15:18.833 INFO:tasks.workunit.client.1.smithi148.stdout:9/529: dread d5/f23 [4194304,4194304] 0 2024-09-21T04:15:18.833 INFO:tasks.workunit.client.1.smithi148.stdout:9/530: write d5/f16 [1631406,36211] 0 2024-09-21T04:15:18.835 INFO:tasks.workunit.client.1.smithi148.stdout:5/486: link d3/d24/f43 d3/d4a/fae 0 2024-09-21T04:15:18.838 INFO:tasks.workunit.client.1.smithi148.stdout:8/535: truncate dc/d13/d76/d42/f53 351262 0 2024-09-21T04:15:18.839 INFO:tasks.workunit.client.1.smithi148.stdout:7/522: dwrite d5/d8/ff [0,4194304] 0 2024-09-21T04:15:18.842 INFO:tasks.workunit.client.1.smithi148.stdout:8/536: symlink dc/d13/d76/d5d/la3 0 2024-09-21T04:15:18.845 INFO:tasks.workunit.client.1.smithi148.stdout:7/523: mkdir d5/d14/d25/d40/dab/dad/db8 0 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:3/479: sync 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:2/605: sync 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:6/521: sync 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/472: sync 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/473: chown db/d24/d2e/d59/l6e 117670 1 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/474: fsync db/d46/d4c/d82/f90 0 2024-09-21T04:15:18.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/475: dread - db/d46/d4c/d82/f88 zero size 2024-09-21T04:15:18.851 INFO:tasks.workunit.client.1.smithi148.stdout:0/538: rename d5/d6/d3e to d5/d6/d2a/d2b/d84/da5 0 2024-09-21T04:15:18.852 INFO:tasks.workunit.client.1.smithi148.stdout:0/539: dread - d5/d6/d7/d11/d60/f9f zero size 2024-09-21T04:15:18.852 INFO:tasks.workunit.client.1.smithi148.stdout:7/524: mkdir d5/d14/d25/d40/db9 0 2024-09-21T04:15:18.854 INFO:tasks.workunit.client.1.smithi148.stdout:2/606: link d0/d2b/f7d d0/d2b/da3/d8f/fb7 0 2024-09-21T04:15:18.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/537: dread dc/d13/d76/f40 [0,4194304] 0 2024-09-21T04:15:18.857 INFO:tasks.workunit.client.1.smithi148.stdout:4/561: rename d4/d66/d7f/d2a/f73 to d4/d12/d16/d29/d61/d6c/d9d/fa2 0 2024-09-21T04:15:18.859 INFO:tasks.workunit.client.1.smithi148.stdout:2/607: mknod d0/d9/dc/d13/d50/cb8 0 2024-09-21T04:15:18.859 INFO:tasks.workunit.client.1.smithi148.stdout:2/608: truncate d0/d2f/f4a 5227954 0 2024-09-21T04:15:18.859 INFO:tasks.workunit.client.1.smithi148.stdout:2/609: stat d0/d2f/l3e 0 2024-09-21T04:15:18.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/531: dwrite d5/d2b/d32/f58 [0,4194304] 0 2024-09-21T04:15:18.860 INFO:tasks.workunit.client.1.smithi148.stdout:3/480: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:18.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/532: write d5/d1e/d37/d50/f55 [337107,3852] 0 2024-09-21T04:15:18.861 INFO:tasks.workunit.client.1.smithi148.stdout:1/476: write db/dd/f13 [6574367,32788] 0 2024-09-21T04:15:18.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/538: creat dc/d13/d3c/d81/d98/fa4 x:0 0 0 2024-09-21T04:15:18.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/539: chown dc/d43/c64 44603737 1 2024-09-21T04:15:18.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/540: chown dc/d13/d76/d42/l7e 910226191 1 2024-09-21T04:15:18.864 INFO:tasks.workunit.client.1.smithi148.stdout:1/477: dread db/d46/d4c/f5b [0,4194304] 0 2024-09-21T04:15:18.865 INFO:tasks.workunit.client.1.smithi148.stdout:5/487: dwrite d3/d14/d1f/f42 [0,4194304] 0 2024-09-21T04:15:18.867 INFO:tasks.workunit.client.1.smithi148.stdout:6/522: rename d0/d34/d38/d43/d63/f77 to d0/d1a/d23/d27/f95 0 2024-09-21T04:15:18.867 INFO:tasks.workunit.client.1.smithi148.stdout:6/523: write d0/d8/d18/f25 [3472646,67790] 0 2024-09-21T04:15:18.869 INFO:tasks.workunit.client.1.smithi148.stdout:7/525: dwrite d5/d14/d25/fa2 [0,4194304] 0 2024-09-21T04:15:18.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.870+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f73e00050c0 con 0x7f7400074e60 2024-09-21T04:15:18.873 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:18 smithi110 ceph-mon[27723]: pgmap v6: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:18.873 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:18 smithi110 ceph-mon[27723]: mgrmap e25: smithi148.zjrahi(active, since 6s) 2024-09-21T04:15:18.873 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:18 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/163999997' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:15:18.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.871+0000 7f73ef7fe640 1 -- 172.21.15.110:0/2568174832 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f73f4016330 con 0x7f7400074e60 2024-09-21T04:15:18.874 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:15:18.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 msgr2=0x7f73e8042060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 0x7f73e8042060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73f00308d0 tx=0x7f73f0030520 comp rx=0 tx=0).stop 2024-09-21T04:15:18.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 msgr2=0x7f740019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e970 tx=0x7f73f400ee30 comp rx=0 tx=0).stop 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 shutdown_connections 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f73e803fba0 0x7f73e8042060 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74000760b0 0x7f740019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 --2- 172.21.15.110:0/2568174832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7400074e60 0x7f740019bc30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 >> 172.21.15.110:0/2568174832 conn(0x7f74000fd550 msgr2=0x7f740010c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 shutdown_connections 2024-09-21T04:15:18.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:18.873+0000 7f7404f80640 1 -- 172.21.15.110:0/2568174832 wait complete. 2024-09-21T04:15:18.879 INFO:tasks.workunit.client.1.smithi148.stdout:2/610: link d0/d9/d16/d55/c77 d0/d2f/db1/cb9 0 2024-09-21T04:15:18.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/533: truncate d5/d60/d28/d2f/f62 213616 0 2024-09-21T04:15:18.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/534: write d5/d1f/d81/f8c [4240086,45815] 0 2024-09-21T04:15:18.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/562: getdents d4/d12 0 2024-09-21T04:15:18.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/563: chown d4/d66/d7f/c80 378000 1 2024-09-21T04:15:18.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/564: fdatasync d4/d66/d7f/f2e 0 2024-09-21T04:15:18.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/478: mkdir db/dd/d21/d9b 0 2024-09-21T04:15:18.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/479: dread - db/d24/d2e/d71/f8e zero size 2024-09-21T04:15:18.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/488: mknod d3/d34/caf 0 2024-09-21T04:15:18.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/489: dread - d3/d34/f47 zero size 2024-09-21T04:15:18.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/490: creat d3/d24/fb0 x:0 0 0 2024-09-21T04:15:18.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/540: rename d5/d6/d7/d11/c78 to d5/d6/d2a/d3f/d8b/ca6 0 2024-09-21T04:15:18.895 INFO:tasks.workunit.client.1.smithi148.stdout:9/535: dread f1 [4194304,4194304] 0 2024-09-21T04:15:18.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/491: dread d3/d34/d3c/d5d/f95 [0,4194304] 0 2024-09-21T04:15:18.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/492: truncate d3/d14/d1f/f7e 4446205 0 2024-09-21T04:15:18.896 INFO:tasks.workunit.client.1.smithi148.stdout:6/524: mkdir d0/d1a/d23/d32/d93/d96 0 2024-09-21T04:15:18.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/611: dwrite d0/d2b/d48/f5c [0,4194304] 0 2024-09-21T04:15:18.900 INFO:tasks.workunit.client.1.smithi148.stdout:8/541: dwrite dc/d43/d4c/d4f/f8a [0,4194304] 0 2024-09-21T04:15:18.904 INFO:tasks.workunit.client.1.smithi148.stdout:7/526: mknod d5/d8/d6f/d7d/d8a/cba 0 2024-09-21T04:15:18.906 INFO:tasks.workunit.client.1.smithi148.stdout:4/565: dwrite d4/d12/f41 [4194304,4194304] 0 2024-09-21T04:15:18.908 INFO:tasks.workunit.client.1.smithi148.stdout:1/480: readlink db/dd/l3f 0 2024-09-21T04:15:18.908 INFO:tasks.workunit.client.1.smithi148.stdout:1/481: write db/d24/f2b [2773007,52041] 0 2024-09-21T04:15:18.911 INFO:tasks.workunit.client.1.smithi148.stdout:9/536: mknod d5/d60/d28/d2f/dbf/dc2/cc8 0 2024-09-21T04:15:18.912 INFO:tasks.workunit.client.1.smithi148.stdout:5/493: mkdir d3/d14/d1f/db1 0 2024-09-21T04:15:18.912 INFO:tasks.workunit.client.1.smithi148.stdout:5/494: write d3/d14/d1f/d26/f77 [592704,112307] 0 2024-09-21T04:15:18.918 INFO:tasks.workunit.client.1.smithi148.stdout:7/527: dwrite d5/d14/d25/fa2 [0,4194304] 0 2024-09-21T04:15:18.918 INFO:tasks.workunit.client.1.smithi148.stdout:8/542: stat dc/d13/d39/d6e/c8d 0 2024-09-21T04:15:18.920 INFO:tasks.workunit.client.1.smithi148.stdout:4/566: symlink d4/d87/d2d/la3 0 2024-09-21T04:15:18.921 INFO:tasks.workunit.client.1.smithi148.stdout:1/482: creat db/d24/d2e/f9c x:0 0 0 2024-09-21T04:15:18.922 INFO:tasks.workunit.client.1.smithi148.stdout:5/495: getdents d3/d14/d1f/d72 0 2024-09-21T04:15:18.922 INFO:tasks.workunit.client.1.smithi148.stdout:5/496: readlink d3/d14/d1a/l35 0 2024-09-21T04:15:18.922 INFO:tasks.workunit.client.1.smithi148.stdout:5/497: getdents d3/d14/d1f/d26/d2e/d3d 0 2024-09-21T04:15:18.930 INFO:tasks.workunit.client.1.smithi148.stdout:7/528: link d5/d8/d6f/d7d/d90/d28/c46 d5/d14/d25/d40/db9/cbb 0 2024-09-21T04:15:18.931 INFO:tasks.workunit.client.1.smithi148.stdout:4/567: rename d4/d12/c54 to d4/d12/ca4 0 2024-09-21T04:15:18.932 INFO:tasks.workunit.client.1.smithi148.stdout:1/483: mknod db/d46/c9d 0 2024-09-21T04:15:18.932 INFO:tasks.workunit.client.1.smithi148.stdout:1/484: truncate db/dd/f23 5081620 0 2024-09-21T04:15:18.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/498: creat d3/d14/d1f/d26/d59/fb2 x:0 0 0 2024-09-21T04:15:18.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/499: fdatasync d3/d14/d1a/f9a 0 2024-09-21T04:15:18.934 INFO:tasks.workunit.client.1.smithi148.stdout:5/500: stat d3/d14/d1f/d26/d2e/d3d/l9e 0 2024-09-21T04:15:18.934 INFO:tasks.workunit.client.1.smithi148.stdout:5/501: creat d3/d14/d1f/d26/d2e/d56/da7/fb3 x:0 0 0 2024-09-21T04:15:18.934 INFO:tasks.workunit.client.1.smithi148.stdout:5/502: write d3/d14/d1f/d26/d2e/d3d/fa2 [400685,85243] 0 2024-09-21T04:15:18.934 INFO:tasks.workunit.client.1.smithi148.stdout:6/525: rename d0/d1a/d23/d16 to d0/d34/d97 0 2024-09-21T04:15:18.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/543: dwrite dc/d13/d39/d6e/f6f [0,4194304] 0 2024-09-21T04:15:18.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/529: link d5/d8/d64/f96 d5/d14/d25/d40/dab/dad/db8/fbc 0 2024-09-21T04:15:18.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/530: fdatasync d5/d14/d25/d40/d7b/f9a 0 2024-09-21T04:15:18.937 INFO:tasks.workunit.client.1.smithi148.stdout:2/612: truncate d0/d2b/d48/f5c 3639153 0 2024-09-21T04:15:18.938 INFO:tasks.workunit.client.1.smithi148.stdout:4/568: dread d4/d66/d7f/f97 [0,4194304] 0 2024-09-21T04:15:18.938 INFO:tasks.workunit.client.1.smithi148.stdout:4/569: chown d4/d66/d94/f96 986 1 2024-09-21T04:15:18.940 INFO:tasks.workunit.client.1.smithi148.stdout:5/503: mknod d3/d14/d1f/d72/cb4 0 2024-09-21T04:15:18.940 INFO:tasks.workunit.client.1.smithi148.stdout:5/504: chown d3/d14/d1f/d26/d2e/d56 599164 1 2024-09-21T04:15:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:18 smithi148 ceph-mon[34957]: pgmap v6: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-09-21T04:15:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:18 smithi148 ceph-mon[34957]: mgrmap e25: smithi148.zjrahi(active, since 6s) 2024-09-21T04:15:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:18 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/163999997' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:15:18.942 INFO:tasks.workunit.client.1.smithi148.stdout:6/526: rename d0/d1a/d23/f3d to d0/d34/d38/d43/f98 0 2024-09-21T04:15:18.945 INFO:tasks.workunit.client.1.smithi148.stdout:1/485: rmdir db/d24/d2e/d59 39 2024-09-21T04:15:18.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/613: symlink d0/d9/d16/d55/d71/lba 0 2024-09-21T04:15:18.948 INFO:tasks.workunit.client.1.smithi148.stdout:8/544: rename dc/d13/d76/d42/f53 to dc/d13/d6a/fa5 0 2024-09-21T04:15:18.949 INFO:tasks.workunit.client.1.smithi148.stdout:6/527: mkdir d0/d1a/d23/dc/d81/d99 0 2024-09-21T04:15:18.953 INFO:tasks.workunit.client.1.smithi148.stdout:5/505: rename d3/d34/c81 to d3/d14/d1a/cb5 0 2024-09-21T04:15:18.954 INFO:tasks.workunit.client.1.smithi148.stdout:8/545: creat dc/d13/d76/d5d/fa6 x:0 0 0 2024-09-21T04:15:18.954 INFO:tasks.workunit.client.1.smithi148.stdout:8/546: creat dc/d13/d76/d5d/fa7 x:0 0 0 2024-09-21T04:15:18.954 INFO:tasks.workunit.client.1.smithi148.stdout:8/547: fdatasync dc/d13/d28/f48 0 2024-09-21T04:15:18.954 INFO:tasks.workunit.client.1.smithi148.stdout:8/548: chown f9 470 1 2024-09-21T04:15:18.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/486: rmdir db/d24 39 2024-09-21T04:15:18.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/487: readlink db/l18 0 2024-09-21T04:15:18.957 INFO:tasks.workunit.client.1.smithi148.stdout:6/528: rename d0/d34/d97/d36 to d0/d1a/d23/d27/d62/d9a 0 2024-09-21T04:15:18.957 INFO:tasks.workunit.client.1.smithi148.stdout:6/529: truncate d0/d1a/d23/dc/d68/f50 985602 0 2024-09-21T04:15:18.957 INFO:tasks.workunit.client.1.smithi148.stdout:5/506: symlink d3/d34/lb6 0 2024-09-21T04:15:18.960 INFO:tasks.workunit.client.1.smithi148.stdout:8/549: rmdir dc/d13/d76/d7c 0 2024-09-21T04:15:18.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/550: dread dc/d43/d4c/d4f/f8e [0,4194304] 0 2024-09-21T04:15:18.970 INFO:tasks.workunit.client.1.smithi148.stdout:1/488: dread db/f15 [0,4194304] 0 2024-09-21T04:15:18.970 INFO:tasks.workunit.client.1.smithi148.stdout:1/489: creat db/dd/d21/d9b/f9e x:0 0 0 2024-09-21T04:15:18.971 INFO:tasks.workunit.client.1.smithi148.stdout:1/490: getdents db/d24/d2e/d71/d94/d9a 0 2024-09-21T04:15:18.972 INFO:tasks.workunit.client.1.smithi148.stdout:1/491: truncate db/dd/f12 1490578 0 2024-09-21T04:15:18.973 INFO:tasks.workunit.client.1.smithi148.stdout:6/530: dread d0/d1a/d23/dc/d68/f86 [0,4194304] 0 2024-09-21T04:15:18.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/507: dread d3/f8 [0,4194304] 0 2024-09-21T04:15:18.974 INFO:tasks.workunit.client.1.smithi148.stdout:6/531: symlink d0/d1a/d23/l9b 0 2024-09-21T04:15:18.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/508: mknod d3/d24/cb7 0 2024-09-21T04:15:18.978 INFO:tasks.workunit.client.1.smithi148.stdout:6/532: creat d0/d1a/d23/d32/d93/f9c x:0 0 0 2024-09-21T04:15:18.978 INFO:tasks.workunit.client.1.smithi148.stdout:8/551: dread dc/d13/d76/d42/f4a [0,4194304] 0 2024-09-21T04:15:18.978 INFO:tasks.workunit.client.1.smithi148.stdout:7/531: dwrite d5/d14/f78 [0,4194304] 0 2024-09-21T04:15:18.978 INFO:tasks.workunit.client.1.smithi148.stdout:2/614: dwrite d0/d9/dc/d13/d50/f70 [0,4194304] 0 2024-09-21T04:15:18.978 INFO:tasks.workunit.client.1.smithi148.stdout:7/532: fsync d5/d8/d6f/d7d/d90/f50 0 2024-09-21T04:15:18.979 INFO:tasks.workunit.client.1.smithi148.stdout:4/570: dwrite d4/d12/d16/d29/d61/d6c/f88 [0,4194304] 0 2024-09-21T04:15:18.979 INFO:tasks.workunit.client.1.smithi148.stdout:5/509: mkdir d3/d14/d1f/db1/db8 0 2024-09-21T04:15:18.980 INFO:tasks.workunit.client.1.smithi148.stdout:6/533: write d0/d8/d18/f60 [293948,60536] 0 2024-09-21T04:15:18.980 INFO:tasks.workunit.client.1.smithi148.stdout:6/534: chown d0/d1a/d23/dc/l56 927241739 1 2024-09-21T04:15:18.985 INFO:tasks.workunit.client.0.smithi110.stdout:5/579: sync 2024-09-21T04:15:18.985 INFO:tasks.workunit.client.0.smithi110.stdout:8/557: sync 2024-09-21T04:15:18.985 INFO:tasks.workunit.client.0.smithi110.stdout:8/558: chown d0/d30/l4d 7160791 1 2024-09-21T04:15:18.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/533: getdents d5/d14/d25/d40/db9 0 2024-09-21T04:15:18.989 INFO:tasks.workunit.client.0.smithi110.stdout:5/580: mknod d1/d49/d4d/d1e/cc1 0 2024-09-21T04:15:18.991 INFO:tasks.workunit.client.1.smithi148.stdout:4/571: unlink d4/d12/d16/d26/l6e 0 2024-09-21T04:15:18.992 INFO:tasks.workunit.client.1.smithi148.stdout:4/572: write d4/d87/d2d/f6b [394633,54130] 0 2024-09-21T04:15:18.994 INFO:tasks.workunit.client.0.smithi110.stdout:8/559: mkdir d0/d7/d13/dc5 0 2024-09-21T04:15:18.994 INFO:tasks.workunit.client.0.smithi110.stdout:8/560: chown d0/d30/d54/l63 1 1 2024-09-21T04:15:18.995 INFO:tasks.workunit.client.0.smithi110.stdout:5/581: mkdir d1/d7/d18/d8f/dc2 0 2024-09-21T04:15:18.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/492: dwrite db/d24/f70 [4194304,4194304] 0 2024-09-21T04:15:18.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/493: getdents db/d24/d2e/d71/d94/d9a 0 2024-09-21T04:15:18.996 INFO:tasks.workunit.client.1.smithi148.stdout:1/494: chown db/d24/d2e/d71/f8e 0 1 2024-09-21T04:15:18.998 INFO:tasks.workunit.client.0.smithi110.stdout:8/561: unlink d0/d30/d34/d3e/c7f 0 2024-09-21T04:15:18.998 INFO:tasks.workunit.client.1.smithi148.stdout:2/615: dwrite d0/d9/dc/f46 [0,4194304] 0 2024-09-21T04:15:19.003 INFO:tasks.workunit.client.0.smithi110.stdout:5/582: link d1/f8a d1/d7/d18/d8f/dc2/fc3 0 2024-09-21T04:15:19.003 INFO:tasks.workunit.client.0.smithi110.stdout:5/583: creat d1/d49/d6f/fc4 x:0 0 0 2024-09-21T04:15:19.005 INFO:tasks.workunit.client.1.smithi148.stdout:8/552: dwrite dc/d13/d76/d5d/f83 [0,4194304] 0 2024-09-21T04:15:19.010 INFO:tasks.workunit.client.0.smithi110.stdout:5/584: dread d1/d22/d51/f70 [0,4194304] 0 2024-09-21T04:15:19.012 INFO:tasks.workunit.client.0.smithi110.stdout:8/562: dwrite d0/d7/d23/f33 [0,4194304] 0 2024-09-21T04:15:19.013 INFO:tasks.workunit.client.1.smithi148.stdout:7/534: rename d5/d8/d6f/d7d/d90/d28/f49 to d5/d14/d25/d40/db9/fbd 0 2024-09-21T04:15:19.014 INFO:tasks.workunit.client.1.smithi148.stdout:4/573: link d4/d12/d16/f19 d4/d12/d16/d29/d61/d7e/fa5 0 2024-09-21T04:15:19.015 INFO:tasks.workunit.client.1.smithi148.stdout:5/510: truncate d3/fa 228098 0 2024-09-21T04:15:19.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/563: mknod d0/d7/d6f/d86/cc6 0 2024-09-21T04:15:19.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/564: dread - d0/d30/d34/d3e/d5d/f9f zero size 2024-09-21T04:15:19.021 INFO:tasks.workunit.client.1.smithi148.stdout:1/495: symlink db/d24/d2a/l9f 0 2024-09-21T04:15:19.022 INFO:tasks.workunit.client.1.smithi148.stdout:6/535: rmdir d0/d59 39 2024-09-21T04:15:19.024 INFO:tasks.workunit.client.1.smithi148.stdout:2/616: creat d0/d2b/fbb x:0 0 0 2024-09-21T04:15:19.024 INFO:tasks.workunit.client.1.smithi148.stdout:2/617: chown d0/f23 9 1 2024-09-21T04:15:19.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/553: creat dc/fa8 x:0 0 0 2024-09-21T04:15:19.025 INFO:tasks.workunit.client.1.smithi148.stdout:7/535: mknod d5/d14/d25/d40/d8b/cbe 0 2024-09-21T04:15:19.026 INFO:tasks.workunit.client.1.smithi148.stdout:4/574: creat d4/d12/d83/fa6 x:0 0 0 2024-09-21T04:15:19.027 INFO:tasks.workunit.client.1.smithi148.stdout:5/511: mknod d3/d24/cb9 0 2024-09-21T04:15:19.027 INFO:tasks.workunit.client.0.smithi110.stdout:5/585: dwrite d1/d39/f79 [4194304,4194304] 0 2024-09-21T04:15:19.028 INFO:tasks.workunit.client.1.smithi148.stdout:1/496: mkdir db/d46/da0 0 2024-09-21T04:15:19.028 INFO:tasks.workunit.client.1.smithi148.stdout:1/497: chown db/d24/d2e/d59/l5d 511942 1 2024-09-21T04:15:19.028 INFO:tasks.workunit.client.1.smithi148.stdout:1/498: readlink db/d24/d2a/l9f 0 2024-09-21T04:15:19.029 INFO:tasks.workunit.client.1.smithi148.stdout:1/499: write db/dd/f13 [8793734,108076] 0 2024-09-21T04:15:19.029 INFO:tasks.workunit.client.1.smithi148.stdout:6/536: rmdir d0/d1a/d23 39 2024-09-21T04:15:19.031 INFO:tasks.workunit.client.1.smithi148.stdout:8/554: mknod dc/d13/ca9 0 2024-09-21T04:15:19.032 INFO:tasks.workunit.client.1.smithi148.stdout:7/536: rename d5/d8/d6f/d7d/d8a/cba to d5/d8/d6f/d7d/d90/d9b/cbf 0 2024-09-21T04:15:19.033 INFO:tasks.workunit.client.1.smithi148.stdout:4/575: mkdir d4/d12/da7 0 2024-09-21T04:15:19.035 INFO:tasks.workunit.client.1.smithi148.stdout:6/537: unlink d0/d1a/d23/d53/f5f 0 2024-09-21T04:15:19.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/555: rmdir dc/d13/d3c/d81 39 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/618: dread d0/d9/d16/f37 [0,4194304] 0 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:2/619: chown d0/d9/d16/f60 1625744 1 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:1/500: read db/d46/f53 [930594,88981] 0 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:1/501: write db/ff [4952884,33498] 0 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:6/538: unlink d0/d34/d38/d43/d63/f7c 0 2024-09-21T04:15:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:6/539: write d0/d1a/d23/d27/d80/f94 [882221,128506] 0 2024-09-21T04:15:19.039 INFO:tasks.workunit.client.1.smithi148.stdout:4/576: rmdir d4/d12/d16/d26 39 2024-09-21T04:15:19.042 INFO:tasks.workunit.client.1.smithi148.stdout:5/512: dread d3/d14/d1a/f4d [0,4194304] 0 2024-09-21T04:15:19.042 INFO:tasks.workunit.client.1.smithi148.stdout:4/577: rename d4/l25 to d4/d12/da7/la8 0 2024-09-21T04:15:19.042 INFO:tasks.workunit.client.1.smithi148.stdout:4/578: truncate d4/d12/d16/f1f 4803170 0 2024-09-21T04:15:19.043 INFO:tasks.workunit.client.1.smithi148.stdout:4/579: dread - d4/d66/f8f zero size 2024-09-21T04:15:19.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/580: mkdir d4/d66/d7f/d2a/da9 0 2024-09-21T04:15:19.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/581: chown d4/lf 7 1 2024-09-21T04:15:19.047 INFO:tasks.workunit.client.1.smithi148.stdout:1/502: fdatasync db/dd/d21/f5c 0 2024-09-21T04:15:19.048 INFO:tasks.workunit.client.0.smithi110.stdout:8/565: dwrite d0/d7/d89/f99 [0,4194304] 0 2024-09-21T04:15:19.049 INFO:tasks.workunit.client.1.smithi148.stdout:5/513: getdents d3/d34/d3c/d5d/d75 0 2024-09-21T04:15:19.052 INFO:tasks.workunit.client.0.smithi110.stdout:5/586: dwrite d1/d49/fab [0,4194304] 0 2024-09-21T04:15:19.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/582: unlink d4/d87/d2d/f6b 0 2024-09-21T04:15:19.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/583: truncate d4/d12/d16/d29/d61/d7e/fa0 608996 0 2024-09-21T04:15:19.058 INFO:tasks.workunit.client.0.smithi110.stdout:8/566: symlink d0/d7/d23/d82/d92/lc7 0 2024-09-21T04:15:19.058 INFO:tasks.workunit.client.1.smithi148.stdout:1/503: dread db/d24/f2b [0,4194304] 0 2024-09-21T04:15:19.058 INFO:tasks.workunit.client.1.smithi148.stdout:1/504: read db/d24/d2e/d71/d94/d9a/f7d [385505,113156] 0 2024-09-21T04:15:19.061 INFO:tasks.workunit.client.1.smithi148.stdout:6/540: getdents d0/d34/d38/d43/d63 0 2024-09-21T04:15:19.064 INFO:tasks.workunit.client.1.smithi148.stdout:2/620: dwrite d0/d2b/f76 [0,4194304] 0 2024-09-21T04:15:19.072 INFO:tasks.workunit.client.1.smithi148.stdout:4/584: dwrite d4/d12/d16/d29/d61/d6c/d9d/fa2 [4194304,4194304] 0 2024-09-21T04:15:19.072 INFO:tasks.workunit.client.1.smithi148.stdout:5/514: getdents d3/d14/d1f/d26/d59 0 2024-09-21T04:15:19.072 INFO:tasks.workunit.client.1.smithi148.stdout:7/537: dwrite d5/d8/d6f/d7d/d90/f10 [0,4194304] 0 2024-09-21T04:15:19.072 INFO:tasks.workunit.client.0.smithi110.stdout:5/587: symlink d1/d22/db6/lc5 0 2024-09-21T04:15:19.073 INFO:tasks.workunit.client.1.smithi148.stdout:2/621: read d0/d9/dc/d13/f21 [341061,101317] 0 2024-09-21T04:15:19.074 INFO:tasks.workunit.client.0.smithi110.stdout:8/567: mknod d0/d30/d34/d3e/cc8 0 2024-09-21T04:15:19.076 INFO:tasks.workunit.client.1.smithi148.stdout:4/585: mkdir d4/d12/d78/daa 0 2024-09-21T04:15:19.079 INFO:tasks.workunit.client.0.smithi110.stdout:5/588: mkdir d1/d22/db6/dc6 0 2024-09-21T04:15:19.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/515: mkdir d3/d14/d1f/db1/db8/dba 0 2024-09-21T04:15:19.081 INFO:tasks.workunit.client.1.smithi148.stdout:1/505: dwrite db/d24/f8b [0,4194304] 0 2024-09-21T04:15:19.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/589: rename d1/d49/lac to d1/d7/d58/d9e/lc7 0 2024-09-21T04:15:19.085 INFO:tasks.workunit.client.0.smithi110.stdout:5/590: creat d1/d49/d4d/fc8 x:0 0 0 2024-09-21T04:15:19.085 INFO:tasks.workunit.client.0.smithi110.stdout:5/591: write d1/d49/d4d/f24 [809821,77478] 0 2024-09-21T04:15:19.088 INFO:tasks.workunit.client.0.smithi110.stdout:5/592: truncate d1/d49/d4d/f1b 1211059 0 2024-09-21T04:15:19.088 INFO:tasks.workunit.client.0.smithi110.stdout:5/593: read d1/dc/f33 [450797,63287] 0 2024-09-21T04:15:19.092 INFO:tasks.workunit.client.0.smithi110.stdout:5/594: rename d1/d49/d6f/fb8 to d1/d7/d58/d72/fc9 0 2024-09-21T04:15:19.092 INFO:tasks.workunit.client.0.smithi110.stdout:5/595: write d1/d22/d51/fa5 [276473,88843] 0 2024-09-21T04:15:19.093 INFO:tasks.workunit.client.1.smithi148.stdout:2/622: symlink d0/d9/d59/lbc 0 2024-09-21T04:15:19.094 INFO:tasks.workunit.client.1.smithi148.stdout:2/623: creat d0/d9/d16/d55/d71/fbd x:0 0 0 2024-09-21T04:15:19.094 INFO:tasks.workunit.client.1.smithi148.stdout:2/624: fsync d0/d2b/f5e 0 2024-09-21T04:15:19.094 INFO:tasks.workunit.client.1.smithi148.stdout:2/625: chown d0/d9/c1c 93027317 1 2024-09-21T04:15:19.100 INFO:tasks.workunit.client.1.smithi148.stdout:0/541: sync 2024-09-21T04:15:19.100 INFO:tasks.workunit.client.1.smithi148.stdout:9/537: sync 2024-09-21T04:15:19.100 INFO:tasks.workunit.client.1.smithi148.stdout:3/481: sync 2024-09-21T04:15:19.100 INFO:tasks.workunit.client.1.smithi148.stdout:9/538: stat d5/ff 0 2024-09-21T04:15:19.101 INFO:tasks.workunit.client.1.smithi148.stdout:0/542: fdatasync d5/d6/d7/d11/f71 0 2024-09-21T04:15:19.101 INFO:tasks.workunit.client.1.smithi148.stdout:0/543: stat d5/d6/d7/d22/l69 0 2024-09-21T04:15:19.101 INFO:tasks.workunit.client.1.smithi148.stdout:2/626: rename d0/d9/d16/l81 to d0/d2b/da3/d82/dae/lbe 0 2024-09-21T04:15:19.101 INFO:tasks.workunit.client.1.smithi148.stdout:5/516: creat d3/d88/fbb x:0 0 0 2024-09-21T04:15:19.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/482: mkdir d3/d5/d6/d2b/d96 0 2024-09-21T04:15:19.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/483: truncate d3/d5/d6/d12/d15/f6c 702583 0 2024-09-21T04:15:19.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/484: chown d3/d84/f95 7240191 1 2024-09-21T04:15:19.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/485: fsync d3/d5/d5c/f58 0 2024-09-21T04:15:19.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/539: rmdir d5/d1e/d37/d46 39 2024-09-21T04:15:19.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/540: fsync d5/fa5 0 2024-09-21T04:15:19.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/541: readlink d5/d2b/d4e/la9 0 2024-09-21T04:15:19.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/542: write d5/d2b/dbe/f8f [926722,6463] 0 2024-09-21T04:15:19.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/543: chown d5/d1f/f24 1882 1 2024-09-21T04:15:19.105 INFO:tasks.workunit.client.1.smithi148.stdout:3/486: mknod d3/d1f/d4d/c97 0 2024-09-21T04:15:19.106 INFO:tasks.workunit.client.1.smithi148.stdout:0/544: rename d5/d6/f3c to d5/d6/d2a/d45/fa7 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.1.smithi148.stdout:0/545: readlink d5/d6/l14 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.1.smithi148.stdout:0/546: write d5/d6/f1f [4806751,129094] 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.1.smithi148.stdout:0/547: chown d5/d6/d2a/d2b/f2c 0 1 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.1.smithi148.stdout:0/548: fsync d5/d6/d7/d22/f8a 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/544: rmdir d5/d1f/d81/d9e/da8 39 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/596: dwrite d1/f3e [0,4194304] 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/597: stat d1/d49/d4d/fc8 0 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/598: chown d1/f35 898 1 2024-09-21T04:15:19.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/599: chown d1/d22/d6e 1 1 2024-09-21T04:15:19.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/487: mknod d3/d1f/d4d/d82/d52/c98 0 2024-09-21T04:15:19.109 INFO:tasks.workunit.client.1.smithi148.stdout:0/549: unlink d5/d6/d7/d18/c77 0 2024-09-21T04:15:19.109 INFO:tasks.workunit.client.1.smithi148.stdout:0/550: chown d5/d6/l14 267824 1 2024-09-21T04:15:19.111 INFO:tasks.workunit.client.1.smithi148.stdout:9/545: unlink d5/d60/d28/c25 0 2024-09-21T04:15:19.112 INFO:tasks.workunit.client.1.smithi148.stdout:3/488: unlink d3/d5/c18 0 2024-09-21T04:15:19.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/627: getdents d0/d9/d16/d55/d71 0 2024-09-21T04:15:19.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/628: chown d0/d9/dc/d13/d25/l75 2084 1 2024-09-21T04:15:19.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/629: chown d0/d9/d59/f65 0 1 2024-09-21T04:15:19.113 INFO:tasks.workunit.client.1.smithi148.stdout:2/630: readlink d0/d9/d16/d55/d71/l9a 0 2024-09-21T04:15:19.114 INFO:tasks.workunit.client.1.smithi148.stdout:0/551: rename d5/d6/d2a/d2b/d84/da5/f26 to d5/d6/d7/d11/d60/fa8 0 2024-09-21T04:15:19.115 INFO:tasks.workunit.client.1.smithi148.stdout:5/517: read d3/d14/d41/f4b [1230613,94875] 0 2024-09-21T04:15:19.115 INFO:tasks.workunit.client.1.smithi148.stdout:9/546: mknod d5/d2b/d32/d67/d80/cc9 0 2024-09-21T04:15:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/547: read d5/d2b/d4e/f97 [24584,42958] 0 2024-09-21T04:15:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/548: chown d5/d1e/d37/l7c 28 1 2024-09-21T04:15:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/549: truncate d5/f42 850885 0 2024-09-21T04:15:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/550: chown d5/d60/d4c/d5e/c9c 0 1 2024-09-21T04:15:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/551: creat d5/d1e/d37/fca x:0 0 0 2024-09-21T04:15:19.117 INFO:tasks.workunit.client.1.smithi148.stdout:3/489: mkdir d3/d5/d6/d2b/d96/d99 0 2024-09-21T04:15:19.117 INFO:tasks.workunit.client.1.smithi148.stdout:3/490: fdatasync d3/f55 0 2024-09-21T04:15:19.117 INFO:tasks.workunit.client.1.smithi148.stdout:2/631: creat d0/d9/dc/fbf x:0 0 0 2024-09-21T04:15:19.118 INFO:tasks.workunit.client.1.smithi148.stdout:6/541: dwrite d0/d34/d97/f39 [0,4194304] 0 2024-09-21T04:15:19.118 INFO:tasks.workunit.client.1.smithi148.stdout:4/586: dwrite d4/fb [0,4194304] 0 2024-09-21T04:15:19.118 INFO:tasks.workunit.client.1.smithi148.stdout:1/506: dwrite db/d46/f5f [4194304,4194304] 0 2024-09-21T04:15:19.118 INFO:tasks.workunit.client.1.smithi148.stdout:1/507: chown db/d24/d2e/f7b 313259781 1 2024-09-21T04:15:19.119 INFO:tasks.workunit.client.0.smithi110.stdout:8/568: dwrite d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:19.119 INFO:tasks.workunit.client.0.smithi110.stdout:8/569: write d0/d7/f78 [440699,101460] 0 2024-09-21T04:15:19.119 INFO:tasks.workunit.client.1.smithi148.stdout:4/587: write d4/d12/f63 [44144,123617] 0 2024-09-21T04:15:19.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/552: link d5/d82/l94 d5/d6/d2a/d2b/la9 0 2024-09-21T04:15:19.122 INFO:tasks.workunit.client.1.smithi148.stdout:3/491: getdents d3/d5/d6 0 2024-09-21T04:15:19.126 INFO:tasks.workunit.client.1.smithi148.stdout:1/508: rename db/d46/d4c/f6b to db/d24/d2e/fa1 0 2024-09-21T04:15:19.126 INFO:tasks.workunit.client.1.smithi148.stdout:1/509: write db/d46/f66 [1595771,88694] 0 2024-09-21T04:15:19.126 INFO:tasks.workunit.client.1.smithi148.stdout:1/510: dread - db/d24/d2e/f9c zero size 2024-09-21T04:15:19.126 INFO:tasks.workunit.client.1.smithi148.stdout:9/552: write d5/d39/f6f [8254271,123734] 0 2024-09-21T04:15:19.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/588: mknod d4/d12/d75/d85/cab 0 2024-09-21T04:15:19.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/492: truncate d3/d1f/d4d/d82/d23/f26 7200895 0 2024-09-21T04:15:19.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/493: chown d3/c8d 9 1 2024-09-21T04:15:19.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/494: creat d3/d5/d6/d12/d15/d73/f9a x:0 0 0 2024-09-21T04:15:19.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/511: rename db/d24/d2e/d71/d94/d9a/f55 to db/d24/d2e/d59/fa2 0 2024-09-21T04:15:19.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/512: truncate db/d46/f6a 476255 0 2024-09-21T04:15:19.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/513: chown db/d46/c9d 73 1 2024-09-21T04:15:19.134 INFO:tasks.workunit.client.1.smithi148.stdout:2/632: write d0/d2b/da3/f6b [4172926,112529] 0 2024-09-21T04:15:19.134 INFO:tasks.workunit.client.1.smithi148.stdout:2/633: write d0/d9/d16/d55/d71/fbd [692822,127525] 0 2024-09-21T04:15:19.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/495: mkdir d3/d5/d6/d12/d43/d54/d9b 0 2024-09-21T04:15:19.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/553: rename d5/d1f/f7d to d5/d2b/d4e/db3/fcb 0 2024-09-21T04:15:19.140 INFO:tasks.workunit.client.1.smithi148.stdout:1/514: getdents db/d24 0 2024-09-21T04:15:19.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/553: write d5/d6/d2a/d45/f50 [1132108,46202] 0 2024-09-21T04:15:19.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/554: truncate d5/d6/d7/d11/d60/f9f 1034708 0 2024-09-21T04:15:19.141 INFO:tasks.workunit.client.1.smithi148.stdout:2/634: truncate d0/d9/dc/d13/f80 2232397 0 2024-09-21T04:15:19.141 INFO:tasks.workunit.client.1.smithi148.stdout:2/635: readlink d0/d9/d16/d55/d71/l7a 0 2024-09-21T04:15:19.142 INFO:tasks.workunit.client.1.smithi148.stdout:1/515: mkdir db/d46/d4c/da3 0 2024-09-21T04:15:19.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/516: truncate db/d24/d2a/f45 718692 0 2024-09-21T04:15:19.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/517: readlink db/d24/d2a/l9f 0 2024-09-21T04:15:19.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/555: creat d5/d6/d2a/d3f/d6d/faa x:0 0 0 2024-09-21T04:15:19.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/636: mkdir d0/d9/dc/d13/d50/dc0 0 2024-09-21T04:15:19.146 INFO:tasks.workunit.client.1.smithi148.stdout:1/518: dread f6 [0,4194304] 0 2024-09-21T04:15:19.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/556: mkdir d5/d6/d2a/d3f/d8b/dab 0 2024-09-21T04:15:19.147 INFO:tasks.workunit.client.0.smithi110.stdout:5/600: dwrite d1/d22/d51/fa5 [0,4194304] 0 2024-09-21T04:15:19.147 INFO:tasks.workunit.client.0.smithi110.stdout:5/601: chown d1/d39/f2d 54587578 1 2024-09-21T04:15:19.147 INFO:tasks.workunit.client.1.smithi148.stdout:2/637: truncate d0/d9/dc/f24 67344 0 2024-09-21T04:15:19.148 INFO:tasks.workunit.client.1.smithi148.stdout:3/496: read d3/d84/f7e [1654042,65411] 0 2024-09-21T04:15:19.148 INFO:tasks.workunit.client.1.smithi148.stdout:3/497: chown d3/d5/d6/d12/d15 76585 1 2024-09-21T04:15:19.149 INFO:tasks.workunit.client.1.smithi148.stdout:0/557: rename d5/d6/d2a/d3f/f4c to d5/d6/d2a/d2b/fac 0 2024-09-21T04:15:19.152 INFO:tasks.workunit.client.1.smithi148.stdout:9/554: dread d5/d2b/f4f [0,4194304] 0 2024-09-21T04:15:19.152 INFO:tasks.workunit.client.1.smithi148.stdout:9/555: fdatasync d5/d2b/dbe/f9a 0 2024-09-21T04:15:19.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/558: getdents d5/d6/d7/d11 0 2024-09-21T04:15:19.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/589: dwrite d4/d87/f57 [0,4194304] 0 2024-09-21T04:15:19.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/590: creat d4/d12/d75/d85/fac x:0 0 0 2024-09-21T04:15:19.155 INFO:tasks.workunit.client.1.smithi148.stdout:9/556: getdents d5/d2b/d32/d67 0 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.1.smithi148.stdout:0/559: symlink d5/d6/d2a/d3f/d6d/da3/lad 0 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/449: sync 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:7/506: sync 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:4/702: sync 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:1/829: sync 2024-09-21T04:15:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:2/357: sync 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:3/436: sync 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:3/437: write d0/d12/d16/d1a/d5c/f79 [724591,59234] 0 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:0/813: sync 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:6/585: sync 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:6/586: readlink d4/d12/d17/d33/d76/l91 0 2024-09-21T04:15:19.158 INFO:tasks.workunit.client.0.smithi110.stdout:6/587: chown d4/d12/c92 1 1 2024-09-21T04:15:19.161 INFO:tasks.workunit.client.0.smithi110.stdout:5/602: dread d1/d39/f50 [0,4194304] 0 2024-09-21T04:15:19.161 INFO:tasks.workunit.client.0.smithi110.stdout:8/570: dwrite d0/d7/f78 [0,4194304] 0 2024-09-21T04:15:19.161 INFO:tasks.workunit.client.0.smithi110.stdout:8/571: write d0/d7/d13/d46/f51 [4296466,42432] 0 2024-09-21T04:15:19.161 INFO:tasks.workunit.client.0.smithi110.stdout:8/572: truncate d0/d7/d13/d4b/d91/f95 641783 0 2024-09-21T04:15:19.166 INFO:tasks.workunit.client.0.smithi110.stdout:9/450: rename d7/d96/d99/d3c/d14/d5c to d7/d96/d99/d3c/d4f/d9f 0 2024-09-21T04:15:19.166 INFO:tasks.workunit.client.0.smithi110.stdout:9/451: fsync d7/d96/d99/d3c/d11/f4e 0 2024-09-21T04:15:19.170 INFO:tasks.workunit.client.1.smithi148.stdout:1/519: dwrite db/d24/d2a/f42 [4194304,4194304] 0 2024-09-21T04:15:19.171 INFO:tasks.workunit.client.0.smithi110.stdout:7/507: creat dd/d1c/d39/d4a/d52/f9c x:0 0 0 2024-09-21T04:15:19.172 INFO:tasks.workunit.client.0.smithi110.stdout:7/508: dread - dd/d1c/d39/d4a/f77 zero size 2024-09-21T04:15:19.176 INFO:tasks.workunit.client.0.smithi110.stdout:0/814: dread d4/fb [0,4194304] 0 2024-09-21T04:15:19.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/452: write d7/d96/d99/d3c/d11/f2e [2946918,33003] 0 2024-09-21T04:15:19.179 INFO:tasks.workunit.client.0.smithi110.stdout:3/438: symlink d0/d12/d16/d1a/d6f/la0 0 2024-09-21T04:15:19.183 INFO:tasks.workunit.client.1.smithi148.stdout:1/520: mkdir db/d24/d2a/da4 0 2024-09-21T04:15:19.186 INFO:tasks.workunit.client.0.smithi110.stdout:2/358: rename d2/d1a/d1f/d5f/f4a to d2/d1a/f6c 0 2024-09-21T04:15:19.190 INFO:tasks.workunit.client.1.smithi148.stdout:1/521: unlink db/d46/f47 0 2024-09-21T04:15:19.191 INFO:tasks.workunit.client.0.smithi110.stdout:2/359: dread d2/f54 [0,4194304] 0 2024-09-21T04:15:19.191 INFO:tasks.workunit.client.0.smithi110.stdout:2/360: write d2/d50/f52 [833695,61913] 0 2024-09-21T04:15:19.191 INFO:tasks.workunit.client.0.smithi110.stdout:2/361: write d2/d1a/d1f/d65/f68 [425852,113914] 0 2024-09-21T04:15:19.192 INFO:tasks.workunit.client.0.smithi110.stdout:7/509: mkdir dd/d48/d55/d9d 0 2024-09-21T04:15:19.193 INFO:tasks.workunit.client.0.smithi110.stdout:0/815: symlink d4/dd/d53/df4/lfc 0 2024-09-21T04:15:19.195 INFO:tasks.workunit.client.0.smithi110.stdout:3/439: mkdir d0/d12/d2a/d80/da1 0 2024-09-21T04:15:19.196 INFO:tasks.workunit.client.1.smithi148.stdout:1/522: rename db/d46/l98 to db/dd/la5 0 2024-09-21T04:15:19.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/453: rename d7/d96/d99/d3c/d4f/d9f/d6f/d78/d8e to d7/d10/d88/d8f/da0 0 2024-09-21T04:15:19.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/362: stat d2/d1a/d1f/d29/c14 0 2024-09-21T04:15:19.198 INFO:tasks.workunit.client.1.smithi148.stdout:1/523: symlink db/d24/d2a/la6 0 2024-09-21T04:15:19.198 INFO:tasks.workunit.client.0.smithi110.stdout:8/573: dwrite d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:19.199 INFO:tasks.workunit.client.1.smithi148.stdout:4/591: dwrite d4/d3a/f53 [0,4194304] 0 2024-09-21T04:15:19.199 INFO:tasks.workunit.client.1.smithi148.stdout:3/498: dwrite d3/d1f/d4d/d82/d23/f26 [0,4194304] 0 2024-09-21T04:15:19.199 INFO:tasks.workunit.client.0.smithi110.stdout:7/510: mknod dd/d1c/d39/d6d/d80/c9e 0 2024-09-21T04:15:19.199 INFO:tasks.workunit.client.0.smithi110.stdout:1/830: dwrite d2/d9/d26/d3d/d8f/f90 [0,4194304] 0 2024-09-21T04:15:19.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/816: rmdir d4/dd/d60/d72/df8 0 2024-09-21T04:15:19.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/817: creat d4/d1c/d62/d69/d9b/ffd x:0 0 0 2024-09-21T04:15:19.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/818: chown d4/dd/l54 1379 1 2024-09-21T04:15:19.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/819: readlink d4/d1c/d62/l3f 0 2024-09-21T04:15:19.200 INFO:tasks.workunit.client.1.smithi148.stdout:2/638: dwrite d0/d9/d16/f56 [0,4194304] 0 2024-09-21T04:15:19.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/524: mkdir db/d24/d2e/d59/da7 0 2024-09-21T04:15:19.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/525: read - db/d24/d2e/fa1 zero size 2024-09-21T04:15:19.202 INFO:tasks.workunit.client.0.smithi110.stdout:3/440: mkdir d0/d12/d2a/d80/da2 0 2024-09-21T04:15:19.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/557: write d5/d60/d28/d2f/f62 [81845,50410] 0 2024-09-21T04:15:19.207 INFO:tasks.workunit.client.1.smithi148.stdout:4/592: creat d4/d12/d16/d26/fad x:0 0 0 2024-09-21T04:15:19.207 INFO:tasks.workunit.client.0.smithi110.stdout:9/454: creat d7/d96/d99/d3c/d4f/fa1 x:0 0 0 2024-09-21T04:15:19.209 INFO:tasks.workunit.client.0.smithi110.stdout:2/363: symlink d2/d1a/d1f/d29/l6d 0 2024-09-21T04:15:19.213 INFO:tasks.workunit.client.0.smithi110.stdout:2/364: dread d2/f3b [0,4194304] 0 2024-09-21T04:15:19.214 INFO:tasks.workunit.client.0.smithi110.stdout:8/574: link d0/d7/cf d0/d7/d23/d52/cc9 0 2024-09-21T04:15:19.218 INFO:tasks.workunit.client.0.smithi110.stdout:7/511: rmdir dd/d48/d55/d9d 0 2024-09-21T04:15:19.222 INFO:tasks.workunit.client.0.smithi110.stdout:1/831: truncate d2/d9/d23/fa2 1078134 0 2024-09-21T04:15:19.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/455: mknod d7/d96/d99/d3c/d1f/d25/ca2 0 2024-09-21T04:15:19.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/575: rmdir d0/d30/d54/d6b/dbf 0 2024-09-21T04:15:19.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/576: fdatasync d0/d30/d34/d3e/f88 0 2024-09-21T04:15:19.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/577: write d0/d30/d34/d3e/f88 [4828550,42473] 0 2024-09-21T04:15:19.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/578: truncate d0/d7/d23/d82/f83 1439834 0 2024-09-21T04:15:19.231 INFO:tasks.workunit.client.0.smithi110.stdout:7/512: unlink dd/d1c/d39/f5a 0 2024-09-21T04:15:19.231 INFO:tasks.workunit.client.0.smithi110.stdout:7/513: getdents dd/d48/d55/d7a/d9b 0 2024-09-21T04:15:19.231 INFO:tasks.workunit.client.0.smithi110.stdout:7/514: getdents dd/d48/d55/d7a/d9b 0 2024-09-21T04:15:19.232 INFO:tasks.workunit.client.1.smithi148.stdout:2/639: truncate d0/d2f/f4a 2680869 0 2024-09-21T04:15:19.232 INFO:tasks.workunit.client.1.smithi148.stdout:2/640: chown d0/d2b/da3/da1/l83 56623 1 2024-09-21T04:15:19.234 INFO:tasks.workunit.client.1.smithi148.stdout:9/558: creat d5/fcc x:0 0 0 2024-09-21T04:15:19.234 INFO:tasks.workunit.client.1.smithi148.stdout:9/559: write d5/d1e/f65 [548409,6412] 0 2024-09-21T04:15:19.235 INFO:tasks.workunit.client.1.smithi148.stdout:9/560: write d5/d1e/d37/f4d [407266,97655] 0 2024-09-21T04:15:19.235 INFO:tasks.workunit.client.1.smithi148.stdout:9/561: creat d5/d60/d28/d73/fcd x:0 0 0 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/456: rename d7/d96/d99/d3c/d11/l5d to d7/d96/d99/d3c/d14/d31/la3 0 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/457: chown d7/d96/d99/d3c/f94 30557 1 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/458: chown d7/d96/d99/d3c/d1f/d72/d59 13 1 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/459: read - d7/d96/f9a zero size 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/460: truncate d7/d26/f90 866812 0 2024-09-21T04:15:19.236 INFO:tasks.workunit.client.0.smithi110.stdout:9/461: fdatasync d7/d96/d99/d3c/d4f/d9f/d6f/f77 0 2024-09-21T04:15:19.237 INFO:tasks.workunit.client.1.smithi148.stdout:4/593: symlink d4/d12/d16/lae 0 2024-09-21T04:15:19.239 INFO:tasks.workunit.client.1.smithi148.stdout:1/526: dwrite db/f15 [0,4194304] 0 2024-09-21T04:15:19.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/832: write d2/db7/fcd [3256672,109528] 0 2024-09-21T04:15:19.240 INFO:tasks.workunit.client.0.smithi110.stdout:8/579: symlink d0/d7/d13/d1c/d41/lca 0 2024-09-21T04:15:19.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/833: chown d2/d9/d23/d4e/f97 19605 1 2024-09-21T04:15:19.240 INFO:tasks.workunit.client.0.smithi110.stdout:8/580: fdatasync d0/d30/d34/d3e/d32/da3/fa9 0 2024-09-21T04:15:19.241 INFO:tasks.workunit.client.0.smithi110.stdout:7/515: truncate dd/f1a 2348195 0 2024-09-21T04:15:19.242 INFO:tasks.workunit.client.0.smithi110.stdout:9/462: creat d7/fa4 x:0 0 0 2024-09-21T04:15:19.242 INFO:tasks.workunit.client.1.smithi148.stdout:2/641: dread d0/d9/dc/d13/f8b [0,4194304] 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:0/820: dwrite d4/dd/d10/d29/d45/fca [0,4194304] 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:3/441: dwrite d0/d12/f6c [0,4194304] 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:3/442: fsync d0/d12/d16/d1a/f4e 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:3/443: write d0/d12/f9d [386742,38073] 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:1/834: creat d2/d9/d49/d4f/d53/f104 x:0 0 0 2024-09-21T04:15:19.244 INFO:tasks.workunit.client.0.smithi110.stdout:2/365: dwrite d2/d50/d56/f51 [0,4194304] 0 2024-09-21T04:15:19.247 INFO:tasks.workunit.client.0.smithi110.stdout:9/463: symlink d7/d96/d99/d3c/d14/la5 0 2024-09-21T04:15:19.247 INFO:tasks.workunit.client.0.smithi110.stdout:9/464: fsync d7/d26/f7e 0 2024-09-21T04:15:19.248 INFO:tasks.workunit.client.0.smithi110.stdout:1/835: dread d2/f2e [0,4194304] 0 2024-09-21T04:15:19.252 INFO:tasks.workunit.client.0.smithi110.stdout:3/444: dread d0/d12/d2a/f4d [0,4194304] 0 2024-09-21T04:15:19.254 INFO:tasks.workunit.client.0.smithi110.stdout:0/821: creat d4/dd/d53/d7b/ffe x:0 0 0 2024-09-21T04:15:19.258 INFO:tasks.workunit.client.0.smithi110.stdout:2/366: getdents d2/d1a/d1f/d29/d19 0 2024-09-21T04:15:19.258 INFO:tasks.workunit.client.0.smithi110.stdout:2/367: fsync d2/d1a/d1f/d5f/f24 0 2024-09-21T04:15:19.259 INFO:tasks.workunit.client.0.smithi110.stdout:2/368: stat d2/f16 0 2024-09-21T04:15:19.259 INFO:tasks.workunit.client.0.smithi110.stdout:9/465: symlink d7/d10/d88/d8f/la6 0 2024-09-21T04:15:19.259 INFO:tasks.workunit.client.0.smithi110.stdout:9/466: chown d7/d96/d99/d3c/d1f/d43/f6c 356 1 2024-09-21T04:15:19.259 INFO:tasks.workunit.client.1.smithi148.stdout:4/594: dread d4/d66/d7f/f2b [0,4194304] 0 2024-09-21T04:15:19.260 INFO:tasks.workunit.client.0.smithi110.stdout:1/836: truncate d2/d9/d49/d4f/d87/ff4 3118467 0 2024-09-21T04:15:19.260 INFO:tasks.workunit.client.0.smithi110.stdout:1/837: write d2/db7/d86/faf [404724,57026] 0 2024-09-21T04:15:19.260 INFO:tasks.workunit.client.0.smithi110.stdout:1/838: readlink d2/d9/d26/d3d/l9a 0 2024-09-21T04:15:19.261 INFO:tasks.workunit.client.1.smithi148.stdout:9/562: dwrite d5/fb8 [0,4194304] 0 2024-09-21T04:15:19.262 INFO:tasks.workunit.client.0.smithi110.stdout:3/445: rmdir d0/d12/d2a/d80/da1 0 2024-09-21T04:15:19.262 INFO:tasks.workunit.client.1.smithi148.stdout:2/642: dwrite d0/d9/dc/fbf [0,4194304] 0 2024-09-21T04:15:19.262 INFO:tasks.workunit.client.0.smithi110.stdout:0/822: symlink d4/dd/d10/d29/d75/lff 0 2024-09-21T04:15:19.266 INFO:tasks.workunit.client.0.smithi110.stdout:2/369: rmdir d2/d1a/d1f/d29/d19/d62 39 2024-09-21T04:15:19.266 INFO:tasks.workunit.client.1.smithi148.stdout:1/527: dwrite db/d46/f97 [0,4194304] 0 2024-09-21T04:15:19.268 INFO:tasks.workunit.client.0.smithi110.stdout:9/467: unlink d7/d10/f37 0 2024-09-21T04:15:19.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/563: getdents d5/d2b/d4e/db3 0 2024-09-21T04:15:19.271 INFO:tasks.workunit.client.0.smithi110.stdout:1/839: creat d2/db7/f105 x:0 0 0 2024-09-21T04:15:19.271 INFO:tasks.workunit.client.0.smithi110.stdout:1/840: read d2/d9/d23/d3b/d9c/fa4 [67670,86187] 0 2024-09-21T04:15:19.274 INFO:tasks.workunit.client.1.smithi148.stdout:2/643: dread d0/f4 [0,4194304] 0 2024-09-21T04:15:19.274 INFO:tasks.workunit.client.1.smithi148.stdout:2/644: creat d0/d9/d59/fc1 x:0 0 0 2024-09-21T04:15:19.278 INFO:tasks.workunit.client.0.smithi110.stdout:3/446: truncate d0/f1f 6686530 0 2024-09-21T04:15:19.278 INFO:tasks.workunit.client.0.smithi110.stdout:3/447: chown d0/d12/d16/d1a/c69 44 1 2024-09-21T04:15:19.278 INFO:tasks.workunit.client.0.smithi110.stdout:9/468: write d7/d96/d99/d3c/d1f/d43/f6c [2895203,45655] 0 2024-09-21T04:15:19.278 INFO:tasks.workunit.client.0.smithi110.stdout:9/469: dread - d7/d96/d99/d3c/d11/f8c zero size 2024-09-21T04:15:19.279 INFO:tasks.workunit.client.0.smithi110.stdout:9/470: chown d7/d96 7632009 1 2024-09-21T04:15:19.279 INFO:tasks.workunit.client.0.smithi110.stdout:9/471: chown d7/d96/d99/d3c/d1f/d25/l74 6683475 1 2024-09-21T04:15:19.282 INFO:tasks.workunit.client.1.smithi148.stdout:4/595: dwrite d4/d12/d16/d29/f46 [0,4194304] 0 2024-09-21T04:15:19.282 INFO:tasks.workunit.client.0.smithi110.stdout:3/448: dread d0/d12/d16/f25 [0,4194304] 0 2024-09-21T04:15:19.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/841: rename d2/d9/d23/d3b/d9c/de6/ff2 to d2/d9/d49/d4f/f106 0 2024-09-21T04:15:19.286 INFO:tasks.workunit.client.0.smithi110.stdout:9/472: truncate d7/d96/d99/d3c/d1f/d72/f63 329143 0 2024-09-21T04:15:19.286 INFO:tasks.workunit.client.0.smithi110.stdout:9/473: read - d7/d96/d99/d3c/d1f/d43/f7b zero size 2024-09-21T04:15:19.286 INFO:tasks.workunit.client.0.smithi110.stdout:9/474: readlink d7/d96/d99/d3c/d1f/d72/l87 0 2024-09-21T04:15:19.288 INFO:tasks.workunit.client.0.smithi110.stdout:1/842: mkdir d2/d9/d23/d3b/dca/d107 0 2024-09-21T04:15:19.290 INFO:tasks.workunit.client.0.smithi110.stdout:0/823: dwrite d4/dd/d10/d29/f6b [0,4194304] 0 2024-09-21T04:15:19.290 INFO:tasks.workunit.client.0.smithi110.stdout:9/475: rename d7/d96/d99/f9b to d7/d96/d99/d3c/d35/fa7 0 2024-09-21T04:15:19.293 INFO:tasks.workunit.client.0.smithi110.stdout:1/843: creat d2/d9/d49/d4f/df0/f108 x:0 0 0 2024-09-21T04:15:19.295 INFO:tasks.workunit.client.0.smithi110.stdout:8/581: write d0/d7/f20 [201454,51041] 0 2024-09-21T04:15:19.297 INFO:tasks.workunit.client.0.smithi110.stdout:2/370: dwrite d2/f44 [0,4194304] 0 2024-09-21T04:15:19.297 INFO:tasks.workunit.client.0.smithi110.stdout:2/371: chown d2/d1a/d1f/d29/l30 9 1 2024-09-21T04:15:19.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/645: dwrite d0/d9/d16/d55/d6f/f3a [0,4194304] 0 2024-09-21T04:15:19.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/646: readlink d0/d2b/lb3 0 2024-09-21T04:15:19.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/647: stat d0/d9/d16/d55/d6f/f67 0 2024-09-21T04:15:19.298 INFO:tasks.workunit.client.1.smithi148.stdout:2/648: fdatasync d0/d2b/da3/d95/fa9 0 2024-09-21T04:15:19.298 INFO:tasks.workunit.client.0.smithi110.stdout:9/476: rename d7/d96/d99/d3c/d1f/d25/ca2 to d7/ca8 0 2024-09-21T04:15:19.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/449: write d0/d12/d16/d1a/f3b [7059023,53026] 0 2024-09-21T04:15:19.300 INFO:tasks.workunit.client.0.smithi110.stdout:3/450: write d0/d12/d3c/d5a/d65/f75 [360033,97355] 0 2024-09-21T04:15:19.301 INFO:tasks.workunit.client.0.smithi110.stdout:2/372: symlink d2/l6e 0 2024-09-21T04:15:19.304 INFO:tasks.workunit.client.0.smithi110.stdout:1/844: rename d2/d9/d49/d4f/ff9 to d2/d9/d49/d4f/d53/f109 0 2024-09-21T04:15:19.306 INFO:tasks.workunit.client.1.smithi148.stdout:1/528: link db/d24/d2e/d59/c5a db/dd/d21/d9b/ca8 0 2024-09-21T04:15:19.306 INFO:tasks.workunit.client.1.smithi148.stdout:1/529: chown db/d46/d4c/c54 6448578 1 2024-09-21T04:15:19.307 INFO:tasks.workunit.client.0.smithi110.stdout:9/477: mknod d7/d96/d99/d3c/d1f/ca9 0 2024-09-21T04:15:19.307 INFO:tasks.workunit.client.0.smithi110.stdout:0/824: dread d4/dd/d53/d66/f74 [0,4194304] 0 2024-09-21T04:15:19.308 INFO:tasks.workunit.client.0.smithi110.stdout:3/451: write d0/d12/d3c/d5a/d70/f97 [125804,25338] 0 2024-09-21T04:15:19.309 INFO:tasks.workunit.client.1.smithi148.stdout:9/564: mkdir d5/d2b/dce 0 2024-09-21T04:15:19.309 INFO:tasks.workunit.client.1.smithi148.stdout:9/565: readlink d5/d2b/d4e/l92 0 2024-09-21T04:15:19.312 INFO:tasks.workunit.client.0.smithi110.stdout:2/373: rename d2/d1a/d1f/f4d to d2/d1a/d1f/d5f/f6f 0 2024-09-21T04:15:19.314 INFO:tasks.workunit.client.0.smithi110.stdout:9/478: mkdir d7/d10/d88/daa 0 2024-09-21T04:15:19.318 INFO:tasks.workunit.client.0.smithi110.stdout:0/825: truncate d4/dd/d60/fa4 3534521 0 2024-09-21T04:15:19.320 INFO:tasks.workunit.client.1.smithi148.stdout:4/596: mkdir d4/d12/d16/d29/d61/d6c/d9d/daf 0 2024-09-21T04:15:19.320 INFO:tasks.workunit.client.1.smithi148.stdout:4/597: stat d4/d66/d7f/f2e 0 2024-09-21T04:15:19.320 INFO:tasks.workunit.client.1.smithi148.stdout:4/598: write d4/f10 [2830831,81025] 0 2024-09-21T04:15:19.320 INFO:tasks.workunit.client.1.smithi148.stdout:1/530: dread db/d24/d2e/d59/fa2 [0,4194304] 0 2024-09-21T04:15:19.321 INFO:tasks.workunit.client.0.smithi110.stdout:3/452: rename d0/d12/d16/d1a/c2c to d0/d51/d7b/ca3 0 2024-09-21T04:15:19.324 INFO:tasks.workunit.client.1.smithi148.stdout:2/649: mkdir d0/d9/d16/d55/d71/dc2 0 2024-09-21T04:15:19.324 INFO:tasks.workunit.client.1.smithi148.stdout:2/650: chown d0/d9/da2 0 1 2024-09-21T04:15:19.325 INFO:tasks.workunit.client.0.smithi110.stdout:2/374: stat d2/d1a/d1f/d2a/l11 0 2024-09-21T04:15:19.326 INFO:tasks.workunit.client.1.smithi148.stdout:4/599: creat d4/d12/d16/d29/d61/d6c/d9d/fb0 x:0 0 0 2024-09-21T04:15:19.328 INFO:tasks.workunit.client.0.smithi110.stdout:9/479: mknod d7/d96/d99/d3c/d1f/d25/cab 0 2024-09-21T04:15:19.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/531: mkdir db/d46/d4c/da3/da9 0 2024-09-21T04:15:19.331 INFO:tasks.workunit.client.0.smithi110.stdout:0/826: symlink d4/d1c/d62/l100 0 2024-09-21T04:15:19.331 INFO:tasks.workunit.client.0.smithi110.stdout:0/827: write d4/d1c/d62/d69/d9b/ffd [341577,47411] 0 2024-09-21T04:15:19.331 INFO:tasks.workunit.client.0.smithi110.stdout:7/516: dwrite dd/d1c/d39/f61 [0,4194304] 0 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:6/542: sync 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/560: sync 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/499: sync 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:8/556: sync 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:7/538: sync 2024-09-21T04:15:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/500: chown d3/d5/f41 18 1 2024-09-21T04:15:19.337 INFO:tasks.workunit.client.0.smithi110.stdout:2/375: rename d2/d1a/d1f/d29/l64 to d2/l70 0 2024-09-21T04:15:19.340 INFO:tasks.workunit.client.1.smithi148.stdout:4/600: truncate d4/d12/d16/f19 2363119 0 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/828: mkdir d4/d1c/d101 0 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/829: chown d4/d1c/d62/d69/ccf 12268 1 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/830: read d4/d47/fde [1324467,87312] 0 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/831: fdatasync d4/dd/d10/f22 0 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/832: truncate d4/dd/d10/d29/fdc 264654 0 2024-09-21T04:15:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:0/833: chown d4/d47/d2c 30782350 1 2024-09-21T04:15:19.342 INFO:tasks.workunit.client.0.smithi110.stdout:1/845: dwrite d2/d9/d49/d4f/f67 [0,4194304] 0 2024-09-21T04:15:19.342 INFO:tasks.workunit.client.1.smithi148.stdout:9/566: dwrite d5/d1e/d37/d46/f6d [0,4194304] 0 2024-09-21T04:15:19.344 INFO:tasks.workunit.client.1.smithi148.stdout:1/532: unlink db/dd/d21/l31 0 2024-09-21T04:15:19.345 INFO:tasks.workunit.client.0.smithi110.stdout:7/517: mkdir dd/d48/d9f 0 2024-09-21T04:15:19.346 INFO:tasks.workunit.client.0.smithi110.stdout:3/453: dread d0/d12/d2a/f31 [0,4194304] 0 2024-09-21T04:15:19.347 INFO:tasks.workunit.client.0.smithi110.stdout:8/582: dwrite d0/d7/d13/d4b/f7d [0,4194304] 0 2024-09-21T04:15:19.351 INFO:tasks.workunit.client.0.smithi110.stdout:1/846: mknod d2/d9/d23/d3b/dca/c10a 0 2024-09-21T04:15:19.351 INFO:tasks.workunit.client.0.smithi110.stdout:7/518: mknod dd/d1c/ca0 0 2024-09-21T04:15:19.351 INFO:tasks.workunit.client.0.smithi110.stdout:1/847: getdents d2/df3/dfb 0 2024-09-21T04:15:19.352 INFO:tasks.workunit.client.1.smithi148.stdout:5/518: sync 2024-09-21T04:15:19.352 INFO:tasks.workunit.client.1.smithi148.stdout:0/561: creat d5/d6/d2a/d3f/d6d/da3/fae x:0 0 0 2024-09-21T04:15:19.352 INFO:tasks.workunit.client.1.smithi148.stdout:0/562: chown d5/d6/d2a/d3f/f8d 0 1 2024-09-21T04:15:19.353 INFO:tasks.workunit.client.1.smithi148.stdout:3/501: rmdir d3/d1f/d4d/d82/d21 39 2024-09-21T04:15:19.355 INFO:tasks.workunit.client.0.smithi110.stdout:9/480: dwrite d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a [0,4194304] 0 2024-09-21T04:15:19.356 INFO:tasks.workunit.client.1.smithi148.stdout:2/651: getdents d0/d2b/d48 0 2024-09-21T04:15:19.357 INFO:tasks.workunit.client.0.smithi110.stdout:9/481: dread d7/d96/d99/f38 [0,4194304] 0 2024-09-21T04:15:19.360 INFO:tasks.workunit.client.1.smithi148.stdout:4/601: mknod d4/d87/d2d/cb1 0 2024-09-21T04:15:19.360 INFO:tasks.workunit.client.1.smithi148.stdout:4/602: write d4/d12/d16/d26/f8a [92513,112426] 0 2024-09-21T04:15:19.362 INFO:tasks.workunit.client.0.smithi110.stdout:1/848: dread d2/d9/d26/d3d/d8f/f90 [0,4194304] 0 2024-09-21T04:15:19.362 INFO:tasks.workunit.client.1.smithi148.stdout:8/557: dwrite dc/d3e/f8b [0,4194304] 0 2024-09-21T04:15:19.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/454: symlink d0/d12/d3c/d5a/d70/la4 0 2024-09-21T04:15:19.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/455: chown d0/d12/d3c/d5a/l83 1024157 1 2024-09-21T04:15:19.367 INFO:tasks.workunit.client.0.smithi110.stdout:7/519: rename dd/d1c/d39/d4a/f77 to dd/d4d/fa1 0 2024-09-21T04:15:19.369 INFO:tasks.workunit.client.1.smithi148.stdout:9/567: mkdir d5/d1f/d81/dcf 0 2024-09-21T04:15:19.370 INFO:tasks.workunit.client.0.smithi110.stdout:0/834: dwrite d4/d47/fcc [0,4194304] 0 2024-09-21T04:15:19.370 INFO:tasks.workunit.client.0.smithi110.stdout:2/376: dwrite d2/d1a/d1f/d5f/f58 [0,4194304] 0 2024-09-21T04:15:19.370 INFO:tasks.workunit.client.1.smithi148.stdout:4/603: write d4/d12/d16/d29/f59 [5682209,119434] 0 2024-09-21T04:15:19.372 INFO:tasks.workunit.client.1.smithi148.stdout:1/533: symlink db/d24/d2e/laa 0 2024-09-21T04:15:19.373 INFO:tasks.workunit.client.1.smithi148.stdout:6/543: getdents d0 0 2024-09-21T04:15:19.373 INFO:tasks.workunit.client.1.smithi148.stdout:6/544: chown d0/d1a/d23/dc/l56 6988183 1 2024-09-21T04:15:19.374 INFO:tasks.workunit.client.0.smithi110.stdout:3/456: write d0/d12/d3c/d5a/f64 [919139,53722] 0 2024-09-21T04:15:19.374 INFO:tasks.workunit.client.0.smithi110.stdout:3/457: stat d0/d12/l44 0 2024-09-21T04:15:19.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/482: truncate d7/d96/d99/d3c/d14/f54 3243634 0 2024-09-21T04:15:19.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/483: dread - d7/fa4 zero size 2024-09-21T04:15:19.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/484: chown d7/d96/d99/d3c/d1f/l48 5300563 1 2024-09-21T04:15:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/583: link d0/d7/d13/d4b/f5b d0/fcb 0 2024-09-21T04:15:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/584: creat d0/d30/d54/d6b/fcc x:0 0 0 2024-09-21T04:15:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/585: truncate d0/d7/d13/d1c/d41/fba 933754 0 2024-09-21T04:15:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/586: dread - d0/d7/d6f/f93 zero size 2024-09-21T04:15:19.378 INFO:tasks.workunit.client.1.smithi148.stdout:7/539: mknod d5/d8/cc0 0 2024-09-21T04:15:19.380 INFO:tasks.workunit.client.0.smithi110.stdout:1/849: mknod d2/d9/d26/d3d/d8f/c10b 0 2024-09-21T04:15:19.380 INFO:tasks.workunit.client.0.smithi110.stdout:1/850: dread - d2/fd0 zero size 2024-09-21T04:15:19.388 INFO:tasks.workunit.client.0.smithi110.stdout:7/520: mknod dd/d1c/d39/d56/ca2 0 2024-09-21T04:15:19.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/502: mknod d3/d5/d6/c9c 0 2024-09-21T04:15:19.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/652: symlink d0/d9/dc/lc3 0 2024-09-21T04:15:19.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/568: mknod d5/d2b/d4e/cd0 0 2024-09-21T04:15:19.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/521: dread dd/d1c/f2c [0,4194304] 0 2024-09-21T04:15:19.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/522: fdatasync dd/d1c/d39/f3b 0 2024-09-21T04:15:19.407 INFO:tasks.workunit.client.0.smithi110.stdout:3/458: mknod d0/d12/d16/d1a/d95/ca5 0 2024-09-21T04:15:19.407 INFO:tasks.workunit.client.0.smithi110.stdout:3/459: truncate d0/d12/d16/f96 746304 0 2024-09-21T04:15:19.409 INFO:tasks.workunit.client.1.smithi148.stdout:6/545: creat d0/d8/d85/f9d x:0 0 0 2024-09-21T04:15:19.411 INFO:tasks.workunit.client.1.smithi148.stdout:7/540: mknod d5/d8/d6f/d7d/d90/cc1 0 2024-09-21T04:15:19.412 INFO:tasks.workunit.client.1.smithi148.stdout:7/541: creat d5/d14/d25/fc2 x:0 0 0 2024-09-21T04:15:19.412 INFO:tasks.workunit.client.0.smithi110.stdout:0/835: dwrite d4/d1c/d62/d69/f77 [4194304,4194304] 0 2024-09-21T04:15:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/653: mkdir d0/d9/d16/d55/d71/dc4 0 2024-09-21T04:15:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/654: write d0/f23 [5074832,74097] 0 2024-09-21T04:15:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:9/569: symlink d5/ld1 0 2024-09-21T04:15:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:9/570: stat d5/d39 0 2024-09-21T04:15:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:1/534: dwrite db/f58 [4194304,4194304] 0 2024-09-21T04:15:19.419 INFO:tasks.workunit.client.1.smithi148.stdout:4/604: dwrite d4/d66/d94/f96 [0,4194304] 0 2024-09-21T04:15:19.419 INFO:tasks.workunit.client.1.smithi148.stdout:6/546: symlink d0/d1a/d23/dc/d68/l9e 0 2024-09-21T04:15:19.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/587: symlink d0/d30/d54/lcd 0 2024-09-21T04:15:19.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/588: write d0/d30/d34/d3e/d32/fb9 [465398,9646] 0 2024-09-21T04:15:19.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/589: chown d0/d7/d13/d4b/f4e 0 1 2024-09-21T04:15:19.420 INFO:tasks.workunit.client.0.smithi110.stdout:1/851: dwrite d2/db7/f105 [0,4194304] 0 2024-09-21T04:15:19.425 INFO:tasks.workunit.client.1.smithi148.stdout:7/542: mknod d5/d8/d64/cc3 0 2024-09-21T04:15:19.426 INFO:tasks.workunit.client.1.smithi148.stdout:3/503: dwrite d3/d1f/d4d/d82/d74/f91 [0,4194304] 0 2024-09-21T04:15:19.427 INFO:tasks.workunit.client.1.smithi148.stdout:5/519: dwrite d3/f9 [0,4194304] 0 2024-09-21T04:15:19.428 INFO:tasks.workunit.client.1.smithi148.stdout:0/563: rename d5/d6/d7/d11/d60 to d5/d6/d2a/d2b/d84/daf 0 2024-09-21T04:15:19.428 INFO:tasks.workunit.client.1.smithi148.stdout:0/564: chown d5/d6/d2a/d3f/d6d/da3/fae 40 1 2024-09-21T04:15:19.429 INFO:tasks.workunit.client.0.smithi110.stdout:3/460: dwrite d0/d12/d2a/f4d [0,4194304] 0 2024-09-21T04:15:19.429 INFO:tasks.workunit.client.0.smithi110.stdout:2/377: rmdir d2/d1a/d1f 39 2024-09-21T04:15:19.433 INFO:tasks.workunit.client.0.smithi110.stdout:7/523: mknod dd/d48/d55/d7a/d9b/ca3 0 2024-09-21T04:15:19.433 INFO:tasks.workunit.client.0.smithi110.stdout:7/524: creat dd/d48/d55/d7a/fa4 x:0 0 0 2024-09-21T04:15:19.435 INFO:tasks.workunit.client.1.smithi148.stdout:2/655: symlink d0/d9/d16/d55/d71/lc5 0 2024-09-21T04:15:19.435 INFO:tasks.workunit.client.1.smithi148.stdout:2/656: write d0/d9/d16/d55/d6f/f67 [219377,112954] 0 2024-09-21T04:15:19.436 INFO:tasks.workunit.client.0.smithi110.stdout:9/485: getdents d7/d26 0 2024-09-21T04:15:19.437 INFO:tasks.workunit.client.1.smithi148.stdout:9/571: symlink d5/d2b/d32/d67/ld2 0 2024-09-21T04:15:19.440 INFO:tasks.workunit.client.0.smithi110.stdout:0/836: rename d4/d1c/c64 to d4/d1c/d101/c102 0 2024-09-21T04:15:19.446 INFO:tasks.workunit.client.1.smithi148.stdout:4/605: creat d4/d87/d91/fb2 x:0 0 0 2024-09-21T04:15:19.446 INFO:tasks.workunit.client.1.smithi148.stdout:4/606: write d4/d12/d83/fa6 [897315,91821] 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/547: getdents d0/d34/d97 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/548: creat d0/d1a/d23/d53/f9f x:0 0 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/549: fdatasync d0/d34/d97/d4f/f8c 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/550: fdatasync d0/d34/d38/d43/f7b 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.0.smithi110.stdout:1/852: truncate d2/d9/d23/d4e/f97 619075 0 2024-09-21T04:15:19.447 INFO:tasks.workunit.client.0.smithi110.stdout:1/853: truncate d2/f45 4748716 0 2024-09-21T04:15:19.449 INFO:tasks.workunit.client.1.smithi148.stdout:7/543: mknod d5/d14/d25/d40/dab/dad/cc4 0 2024-09-21T04:15:19.449 INFO:tasks.workunit.client.1.smithi148.stdout:7/544: chown d5/d14/d25/d79/d86/fa3 11757491 1 2024-09-21T04:15:19.449 INFO:tasks.workunit.client.0.smithi110.stdout:1/854: dread d2/d9/d26/d3d/fd9 [0,4194304] 0 2024-09-21T04:15:19.451 INFO:tasks.workunit.client.0.smithi110.stdout:7/525: dread dd/d1c/f75 [0,4194304] 0 2024-09-21T04:15:19.451 INFO:tasks.workunit.client.0.smithi110.stdout:2/378: creat d2/d1a/d1f/d29/f71 x:0 0 0 2024-09-21T04:15:19.452 INFO:tasks.workunit.client.1.smithi148.stdout:3/504: truncate d3/d1f/d4d/d82/d23/f26 7030512 0 2024-09-21T04:15:19.457 INFO:tasks.workunit.client.1.smithi148.stdout:2/657: link d0/d9/dc/d13/d25/c91 d0/d9/d16/d55/d6f/da7/cc6 0 2024-09-21T04:15:19.460 INFO:tasks.workunit.client.1.smithi148.stdout:1/535: dwrite db/d24/f33 [0,4194304] 0 2024-09-21T04:15:19.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/379: write d2/d1a/d1f/d29/f45 [1712560,13964] 0 2024-09-21T04:15:19.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/380: chown d2/d50/d56/c57 5 1 2024-09-21T04:15:19.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/551: mkdir d0/d34/da0 0 2024-09-21T04:15:19.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/505: getdents d3/d5/d5c 0 2024-09-21T04:15:19.464 INFO:tasks.workunit.client.1.smithi148.stdout:5/520: mknod d3/cbc 0 2024-09-21T04:15:19.464 INFO:tasks.workunit.client.1.smithi148.stdout:5/521: creat d3/d14/d41/fbd x:0 0 0 2024-09-21T04:15:19.465 INFO:tasks.workunit.client.1.smithi148.stdout:5/522: chown d3/d14/d1f/f7e 1821710 1 2024-09-21T04:15:19.466 INFO:tasks.workunit.client.1.smithi148.stdout:2/658: mknod d0/d9/d38/cc7 0 2024-09-21T04:15:19.468 INFO:tasks.workunit.client.1.smithi148.stdout:5/523: dread d3/d14/d1f/f23 [0,4194304] 0 2024-09-21T04:15:19.468 INFO:tasks.workunit.client.1.smithi148.stdout:5/524: truncate d3/d34/d3c/d5d/f9c 882109 0 2024-09-21T04:15:19.468 INFO:tasks.workunit.client.1.smithi148.stdout:5/525: truncate d3/d14/d1f/d26/d2e/d56/f73 857115 0 2024-09-21T04:15:19.468 INFO:tasks.workunit.client.1.smithi148.stdout:9/572: dwrite d5/d60/d28/d2f/f62 [0,4194304] 0 2024-09-21T04:15:19.469 INFO:tasks.workunit.client.0.smithi110.stdout:3/461: dwrite d0/d12/d3c/d5a/d70/f97 [0,4194304] 0 2024-09-21T04:15:19.469 INFO:tasks.workunit.client.1.smithi148.stdout:0/565: dwrite d5/d6/d7/d22/fa2 [0,4194304] 0 2024-09-21T04:15:19.476 INFO:tasks.workunit.client.1.smithi148.stdout:1/536: chown db/dd/l2c 13590 1 2024-09-21T04:15:19.476 INFO:tasks.workunit.client.1.smithi148.stdout:1/537: chown db/d24/d2e/d71/d94/d9a 79504544 1 2024-09-21T04:15:19.477 INFO:tasks.workunit.client.1.smithi148.stdout:6/552: unlink d0/d34/d38/d57/d58/f69 0 2024-09-21T04:15:19.477 INFO:tasks.workunit.client.1.smithi148.stdout:6/553: creat d0/d34/d38/d43/fa1 x:0 0 0 2024-09-21T04:15:19.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/659: unlink d0/d9/f1b 0 2024-09-21T04:15:19.481 INFO:tasks.workunit.client.1.smithi148.stdout:4/607: dwrite d4/d12/d16/d29/f3d [0,4194304] 0 2024-09-21T04:15:19.481 INFO:tasks.workunit.client.1.smithi148.stdout:5/526: readlink d3/l6b 0 2024-09-21T04:15:19.481 INFO:tasks.workunit.client.1.smithi148.stdout:5/527: write d3/d14/d41/f96 [446348,13867] 0 2024-09-21T04:15:19.482 INFO:tasks.workunit.client.1.smithi148.stdout:7/545: dwrite d5/d8/f16 [0,4194304] 0 2024-09-21T04:15:19.489 INFO:tasks.workunit.client.0.smithi110.stdout:9/486: creat d7/d10/d88/fac x:0 0 0 2024-09-21T04:15:19.490 INFO:tasks.workunit.client.1.smithi148.stdout:0/566: mknod d5/d6/d7/cb0 0 2024-09-21T04:15:19.491 INFO:tasks.workunit.client.1.smithi148.stdout:2/660: dread d0/d2b/da3/d8f/fb7 [0,4194304] 0 2024-09-21T04:15:19.491 INFO:tasks.workunit.client.1.smithi148.stdout:1/538: link db/f15 db/dd/d21/d89/fab 0 2024-09-21T04:15:19.491 INFO:tasks.workunit.client.1.smithi148.stdout:1/539: truncate db/d24/d2e/fa1 177293 0 2024-09-21T04:15:19.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/837: link d4/dd/d10/d29/l2b d4/d1c/d62/d91/dd6/l103 0 2024-09-21T04:15:19.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/838: chown d4/dd/d10/d29/d75 123832 1 2024-09-21T04:15:19.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/839: write d4/d47/fc9 [817228,91140] 0 2024-09-21T04:15:19.493 INFO:tasks.workunit.client.0.smithi110.stdout:8/590: getdents d0/d30/d54/d6b 0 2024-09-21T04:15:19.495 INFO:tasks.workunit.client.1.smithi148.stdout:3/506: rmdir d3/d5/d6/d12/d43/d54 39 2024-09-21T04:15:19.496 INFO:tasks.workunit.client.0.smithi110.stdout:1/855: truncate d2/d9/d26/d3d/f58 2903833 0 2024-09-21T04:15:19.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/526: mkdir dd/d1c/d39/d4a/da5 0 2024-09-21T04:15:19.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/527: stat dd/c3d 0 2024-09-21T04:15:19.498 INFO:tasks.workunit.client.1.smithi148.stdout:4/608: mkdir d4/d12/d16/db3 0 2024-09-21T04:15:19.500 INFO:tasks.workunit.client.1.smithi148.stdout:5/528: creat d3/d34/d78/fbe x:0 0 0 2024-09-21T04:15:19.501 INFO:tasks.workunit.client.1.smithi148.stdout:7/546: creat d5/daa/fc5 x:0 0 0 2024-09-21T04:15:19.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/462: creat d0/d22/d29/d76/fa6 x:0 0 0 2024-09-21T04:15:19.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/463: stat d0/d12/c38 0 2024-09-21T04:15:19.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/464: write d0/d22/d29/d76/fa6 [632364,80367] 0 2024-09-21T04:15:19.507 INFO:tasks.workunit.client.1.smithi148.stdout:0/567: mknod d5/d6/d7/d22/cb1 0 2024-09-21T04:15:19.508 INFO:tasks.workunit.client.0.smithi110.stdout:9/487: creat d7/d96/d99/d3c/d1f/d43/fad x:0 0 0 2024-09-21T04:15:19.509 INFO:tasks.workunit.client.1.smithi148.stdout:2/661: symlink d0/d2f/lc8 0 2024-09-21T04:15:19.510 INFO:tasks.workunit.client.1.smithi148.stdout:1/540: truncate db/dd/d21/f7a 4125376 0 2024-09-21T04:15:19.511 INFO:tasks.workunit.client.0.smithi110.stdout:0/840: rmdir d4/dd/d10/d8a/d92 39 2024-09-21T04:15:19.511 INFO:tasks.workunit.client.0.smithi110.stdout:0/841: chown d4/d8b/fb4 47116 1 2024-09-21T04:15:19.511 INFO:tasks.workunit.client.0.smithi110.stdout:0/842: write d4/dd/d53/feb [338947,19106] 0 2024-09-21T04:15:19.512 INFO:tasks.workunit.client.0.smithi110.stdout:0/843: dread d4/d8b/fda [0,4194304] 0 2024-09-21T04:15:19.514 INFO:tasks.workunit.client.1.smithi148.stdout:8/558: sync 2024-09-21T04:15:19.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/554: unlink d0/d34/d38/d43/fa1 0 2024-09-21T04:15:19.515 INFO:tasks.workunit.client.1.smithi148.stdout:1/541: dread db/d24/d2e/d71/d94/d9a/f7d [0,4194304] 0 2024-09-21T04:15:19.515 INFO:tasks.workunit.client.1.smithi148.stdout:1/542: write db/d24/d2e/fa1 [716866,107547] 0 2024-09-21T04:15:19.515 INFO:tasks.workunit.client.0.smithi110.stdout:8/591: rename d0/d7/d13/d4b/c29 to d0/d30/d34/d3e/d32/cce 0 2024-09-21T04:15:19.515 INFO:tasks.workunit.client.0.smithi110.stdout:8/592: write d0/d7/d13/d46/f51 [4344110,5596] 0 2024-09-21T04:15:19.517 INFO:tasks.workunit.client.1.smithi148.stdout:4/609: dwrite d4/d12/d16/d26/f9e [0,4194304] 0 2024-09-21T04:15:19.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/507: creat d3/d5/d6/d2b/d96/f9d x:0 0 0 2024-09-21T04:15:19.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/856: mknod d2/d9/c10c 0 2024-09-21T04:15:19.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/857: readlink d2/l3a 0 2024-09-21T04:15:19.519 INFO:tasks.workunit.client.0.smithi110.stdout:1/858: truncate d2/fd0 308450 0 2024-09-21T04:15:19.521 INFO:tasks.workunit.client.0.smithi110.stdout:0/844: dread d4/dd/d10/d29/d75/f9d [0,4194304] 0 2024-09-21T04:15:19.521 INFO:tasks.workunit.client.0.smithi110.stdout:0/845: stat d4/dd/d10/cb3 0 2024-09-21T04:15:19.521 INFO:tasks.workunit.client.0.smithi110.stdout:0/846: readlink d4/dd/d10/d8a/db5/ld3 0 2024-09-21T04:15:19.523 INFO:tasks.workunit.client.0.smithi110.stdout:3/465: mkdir d0/d12/d2a/d80/da2/da7 0 2024-09-21T04:15:19.525 INFO:tasks.workunit.client.1.smithi148.stdout:7/547: mkdir d5/daa/dc6 0 2024-09-21T04:15:19.525 INFO:tasks.workunit.client.0.smithi110.stdout:9/488: write d7/d96/d99/d3c/f5a [3114004,72523] 0 2024-09-21T04:15:19.525 INFO:tasks.workunit.client.0.smithi110.stdout:9/489: chown d7/d96/d99/d3c/f20 0 1 2024-09-21T04:15:19.525 INFO:tasks.workunit.client.0.smithi110.stdout:9/490: getdents d7/d26 0 2024-09-21T04:15:19.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/491: readlink d7/d10/l34 0 2024-09-21T04:15:19.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/492: fdatasync d7/d10/d88/fac 0 2024-09-21T04:15:19.527 INFO:tasks.workunit.client.1.smithi148.stdout:0/568: creat d5/d6/d7/d18/fb2 x:0 0 0 2024-09-21T04:15:19.527 INFO:tasks.workunit.client.1.smithi148.stdout:0/569: fdatasync d5/f85 0 2024-09-21T04:15:19.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/662: mkdir d0/d2b/da3/d8f/d9f/dc9 0 2024-09-21T04:15:19.530 INFO:tasks.workunit.client.0.smithi110.stdout:0/847: write d4/d47/d2c/fce [1922190,16934] 0 2024-09-21T04:15:19.531 INFO:tasks.workunit.client.1.smithi148.stdout:9/573: write d5/f42 [343125,10368] 0 2024-09-21T04:15:19.531 INFO:tasks.workunit.client.1.smithi148.stdout:9/574: write f0 [3714331,109346] 0 2024-09-21T04:15:19.532 INFO:tasks.workunit.client.1.smithi148.stdout:8/559: dwrite dc/d43/f44 [0,4194304] 0 2024-09-21T04:15:19.532 INFO:tasks.workunit.client.1.smithi148.stdout:8/560: stat dc/f31 0 2024-09-21T04:15:19.533 INFO:tasks.workunit.client.1.smithi148.stdout:4/610: mkdir d4/d12/d75/db4 0 2024-09-21T04:15:19.534 INFO:tasks.workunit.client.1.smithi148.stdout:3/508: symlink d3/d5/d6/d2b/d96/l9e 0 2024-09-21T04:15:19.544 INFO:tasks.workunit.client.0.smithi110.stdout:3/466: dread d0/f56 [0,4194304] 0 2024-09-21T04:15:19.546 INFO:tasks.workunit.client.1.smithi148.stdout:0/570: mknod d5/d6/cb3 0 2024-09-21T04:15:19.549 INFO:tasks.workunit.client.0.smithi110.stdout:9/493: write d7/d96/d99/d3c/d35/f5e [2703132,9605] 0 2024-09-21T04:15:19.551 INFO:tasks.workunit.client.0.smithi110.stdout:3/467: creat d0/d12/d3c/d5a/d70/d81/fa8 x:0 0 0 2024-09-21T04:15:19.551 INFO:tasks.workunit.client.0.smithi110.stdout:3/468: write d0/f7 [61949,120786] 0 2024-09-21T04:15:19.552 INFO:tasks.workunit.client.1.smithi148.stdout:9/575: mknod d5/d39/dba/cd3 0 2024-09-21T04:15:19.552 INFO:tasks.workunit.client.1.smithi148.stdout:0/571: dread d5/d6/d2a/d2b/f39 [0,4194304] 0 2024-09-21T04:15:19.553 INFO:tasks.workunit.client.0.smithi110.stdout:7/528: rename dd/d48/d55 to dd/d1c/d39/d56/d66/da6 0 2024-09-21T04:15:19.555 INFO:tasks.workunit.client.1.smithi148.stdout:4/611: dwrite d4/d12/d16/d29/f3d [4194304,4194304] 0 2024-09-21T04:15:19.560 INFO:tasks.workunit.client.0.smithi110.stdout:9/494: mkdir d7/d96/d99/d3c/dae 0 2024-09-21T04:15:19.561 INFO:tasks.workunit.client.1.smithi148.stdout:0/572: dread d5/d6/d2a/d45/d70/f7f [0,4194304] 0 2024-09-21T04:15:19.564 INFO:tasks.workunit.client.0.smithi110.stdout:1/859: rename d2/d9/d26/f31 to d2/d9/d23/d4e/d57/f10d 0 2024-09-21T04:15:19.566 INFO:tasks.workunit.client.1.smithi148.stdout:8/561: creat dc/d13/d76/d50/faa x:0 0 0 2024-09-21T04:15:19.566 INFO:tasks.workunit.client.1.smithi148.stdout:5/529: rename d3/d14/d41 to d3/d34/d78/dbf 0 2024-09-21T04:15:19.567 INFO:tasks.workunit.client.1.smithi148.stdout:1/543: write db/dd/d21/f7a [24508,6935] 0 2024-09-21T04:15:19.568 INFO:tasks.workunit.client.1.smithi148.stdout:1/544: chown db/d24/d2e/l44 27 1 2024-09-21T04:15:19.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/529: mkdir dd/d1c/d39/d56/d66/da6/da7 0 2024-09-21T04:15:19.570 INFO:tasks.workunit.client.1.smithi148.stdout:7/548: truncate d5/d8/f1b 3654077 0 2024-09-21T04:15:19.570 INFO:tasks.workunit.client.0.smithi110.stdout:3/469: dwrite d0/d12/d16/d1a/f86 [0,4194304] 0 2024-09-21T04:15:19.571 INFO:tasks.workunit.client.0.smithi110.stdout:3/470: write d0/d12/d3c/f48 [5081497,126517] 0 2024-09-21T04:15:19.572 INFO:tasks.workunit.client.0.smithi110.stdout:1/860: rename d2/d9/d49/d4f/d53/f85 to d2/d9/d49/d7a/dee/f10e 0 2024-09-21T04:15:19.573 INFO:tasks.workunit.client.1.smithi148.stdout:2/663: truncate d0/d9/dc/f12 1512621 0 2024-09-21T04:15:19.575 INFO:tasks.workunit.client.1.smithi148.stdout:9/576: mknod d5/d2b/dbe/cd4 0 2024-09-21T04:15:19.575 INFO:tasks.workunit.client.1.smithi148.stdout:9/577: readlink d5/l13 0 2024-09-21T04:15:19.575 INFO:tasks.workunit.client.0.smithi110.stdout:1/861: dread d2/d9/d23/fa2 [0,4194304] 0 2024-09-21T04:15:19.575 INFO:tasks.workunit.client.0.smithi110.stdout:1/862: readlink d2/d9/d49/lfd 0 2024-09-21T04:15:19.579 INFO:tasks.workunit.client.1.smithi148.stdout:4/612: unlink d4/d87/l9b 0 2024-09-21T04:15:19.580 INFO:tasks.workunit.client.0.smithi110.stdout:0/848: rename d4/d47/l63 to d4/l104 0 2024-09-21T04:15:19.580 INFO:tasks.workunit.client.0.smithi110.stdout:0/849: stat d4/c7 0 2024-09-21T04:15:19.581 INFO:tasks.workunit.client.1.smithi148.stdout:0/573: mkdir d5/d6/d2a/d2b/d84/db4 0 2024-09-21T04:15:19.581 INFO:tasks.workunit.client.0.smithi110.stdout:7/530: getdents dd/d1c/d39/d4a/d52 0 2024-09-21T04:15:19.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/530: truncate d3/d37/f5f 1951222 0 2024-09-21T04:15:19.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/531: creat d3/d14/d1a/fc0 x:0 0 0 2024-09-21T04:15:19.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/863: mknod d2/df3/dfb/c10f 0 2024-09-21T04:15:19.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/864: truncate d2/d9/d49/d5c/d91/fde 31048 0 2024-09-21T04:15:19.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/495: dwrite d7/d26/f90 [0,4194304] 0 2024-09-21T04:15:19.586 INFO:tasks.workunit.client.1.smithi148.stdout:6/555: rename d0/d1a/d23/d53/f90 to d0/d1a/d23/d27/d62/d9a/d92/fa2 0 2024-09-21T04:15:19.588 INFO:tasks.workunit.client.0.smithi110.stdout:3/471: dread d0/d12/d16/d1a/d5c/f66 [0,4194304] 0 2024-09-21T04:15:19.593 INFO:tasks.workunit.client.1.smithi148.stdout:2/664: unlink d0/d9/d59/f5a 0 2024-09-21T04:15:19.593 INFO:tasks.workunit.client.1.smithi148.stdout:2/665: chown d0/d2f/lc8 5938 1 2024-09-21T04:15:19.593 INFO:tasks.workunit.client.1.smithi148.stdout:2/666: dread - d0/d9/d16/d55/d6f/f7e zero size 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/496: unlink d7/d96/d99/d3c/d1f/d25/f9e 0 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/497: chown d7/d96/d99/d3c/d1f/d43/f92 436402 1 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/498: write d7/d96/d99/d3c/d35/f76 [373128,44842] 0 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/499: chown d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/c93 3206444 1 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/500: truncate d7/fa4 804133 0 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/501: dread - d7/d96/d99/d3c/d1f/d43/fad zero size 2024-09-21T04:15:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/502: read d7/d96/d99/d3c/d11/f2e [447569,120760] 0 2024-09-21T04:15:19.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/613: symlink d4/d87/d91/lb5 0 2024-09-21T04:15:19.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/574: link d5/d6/d7/d22/f6c d5/d6/d2a/d2b/d84/db4/fb5 0 2024-09-21T04:15:19.602 INFO:tasks.workunit.client.0.smithi110.stdout:9/503: write d7/d96/d99/d3c/d35/f76 [282558,1365] 0 2024-09-21T04:15:19.602 INFO:tasks.workunit.client.1.smithi148.stdout:5/532: fdatasync d3/f12 0 2024-09-21T04:15:19.603 INFO:tasks.workunit.client.1.smithi148.stdout:8/562: getdents dc/d13/d76/d50 0 2024-09-21T04:15:19.603 INFO:tasks.workunit.client.1.smithi148.stdout:8/563: stat dc/d13/d28/f2f 0 2024-09-21T04:15:19.610 INFO:tasks.workunit.client.0.smithi110.stdout:7/531: dwrite dd/d1c/d39/d56/f94 [0,4194304] 0 2024-09-21T04:15:19.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/509: rename d3/d5/d5c/f58 to d3/d1f/d4d/d82/d23/d28/d46/f9f 0 2024-09-21T04:15:19.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/556: creat d0/d1a/d23/d27/d62/d9a/fa3 x:0 0 0 2024-09-21T04:15:19.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/557: chown d0/f6b 3 1 2024-09-21T04:15:19.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/558: write d0/d1a/d23/f19 [3119906,45396] 0 2024-09-21T04:15:19.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/559: fsync d0/d8/f49 0 2024-09-21T04:15:19.618 INFO:tasks.workunit.client.1.smithi148.stdout:5/533: dread d3/d14/d1f/f42 [0,4194304] 0 2024-09-21T04:15:19.618 INFO:tasks.workunit.client.1.smithi148.stdout:1/545: rmdir db/dd/d21 39 2024-09-21T04:15:19.620 INFO:tasks.workunit.client.1.smithi148.stdout:4/614: mknod d4/d66/d94/cb6 0 2024-09-21T04:15:19.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/575: mknod d5/d6/d2a/d2b/d84/cb6 0 2024-09-21T04:15:19.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/576: readlink d5/d6/l14 0 2024-09-21T04:15:19.622 INFO:tasks.workunit.client.0.smithi110.stdout:0/850: dwrite d4/d47/fb2 [0,4194304] 0 2024-09-21T04:15:19.623 INFO:tasks.workunit.client.1.smithi148.stdout:8/564: link dc/d13/d76/d5d/la3 dc/d13/d76/lab 0 2024-09-21T04:15:19.624 INFO:tasks.workunit.client.0.smithi110.stdout:7/532: dread dd/d14/d17/f85 [0,4194304] 0 2024-09-21T04:15:19.624 INFO:tasks.workunit.client.0.smithi110.stdout:7/533: dread - dd/d4d/fa1 zero size 2024-09-21T04:15:19.625 INFO:tasks.workunit.client.1.smithi148.stdout:3/510: creat d3/d1f/d4d/d82/d74/fa0 x:0 0 0 2024-09-21T04:15:19.626 INFO:tasks.workunit.client.1.smithi148.stdout:7/549: rename d5/d8/d6f/d7d/d8a/l92 to d5/daa/lc7 0 2024-09-21T04:15:19.629 INFO:tasks.workunit.client.1.smithi148.stdout:6/560: truncate d0/d1a/d23/dc/d68/f51 3708891 0 2024-09-21T04:15:19.630 INFO:tasks.workunit.client.1.smithi148.stdout:5/534: unlink d3/d14/d1f/d26/d2e/d56/l5c 0 2024-09-21T04:15:19.630 INFO:tasks.workunit.client.1.smithi148.stdout:5/535: getdents d3/d14/d1f/db1/db8/dba 0 2024-09-21T04:15:19.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/546: rename db/dd/d21/c91 to db/dd/d21/cac 0 2024-09-21T04:15:19.631 INFO:tasks.workunit.client.1.smithi148.stdout:4/615: symlink d4/d66/d7f/lb7 0 2024-09-21T04:15:19.631 INFO:tasks.workunit.client.0.smithi110.stdout:0/851: dread d4/dd/d10/fa1 [0,4194304] 0 2024-09-21T04:15:19.631 INFO:tasks.workunit.client.0.smithi110.stdout:0/852: write d4/dd/d10/d29/d45/ff5 [18431,43142] 0 2024-09-21T04:15:19.632 INFO:tasks.workunit.client.1.smithi148.stdout:0/577: unlink d5/d6/d2a/d2b/d84/daf/fa8 0 2024-09-21T04:15:19.633 INFO:tasks.workunit.client.1.smithi148.stdout:8/565: read dc/d13/d3c/d81/d85/f87 [307923,102212] 0 2024-09-21T04:15:19.633 INFO:tasks.workunit.client.0.smithi110.stdout:0/853: creat d4/d1c/d62/d91/f105 x:0 0 0 2024-09-21T04:15:19.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/578: rename d5/d2b/dbe/f8f to d5/d2b/d32/d67/d80/fd5 0 2024-09-21T04:15:19.635 INFO:tasks.workunit.client.1.smithi148.stdout:9/579: chown d5/d2b/d32/d67/c78 21561270 1 2024-09-21T04:15:19.636 INFO:tasks.workunit.client.1.smithi148.stdout:7/550: write d5/d8/f62 [581742,24494] 0 2024-09-21T04:15:19.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/854: creat d4/dd/d10/d29/d5f/dc3/f106 x:0 0 0 2024-09-21T04:15:19.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/561: mkdir d0/d1a/d23/d27/d80/da4 0 2024-09-21T04:15:19.640 INFO:tasks.workunit.client.1.smithi148.stdout:0/578: mkdir d5/d6/d2a/d2b/d99/db7 0 2024-09-21T04:15:19.641 INFO:tasks.workunit.client.1.smithi148.stdout:8/566: mkdir dc/d13/d39/d6e/dac 0 2024-09-21T04:15:19.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/562: link d0/d34/d97/d4f/c88 d0/d34/da0/ca5 0 2024-09-21T04:15:19.643 INFO:tasks.workunit.client.0.smithi110.stdout:1/865: dwrite d2/d9/d23/d3b/d9c/fa4 [0,4194304] 0 2024-09-21T04:15:19.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/536: link d3/f83 d3/d14/d1f/db1/db8/dba/fc1 0 2024-09-21T04:15:19.645 INFO:tasks.workunit.client.1.smithi148.stdout:0/579: dread d5/d6/d2a/d2b/f55 [0,4194304] 0 2024-09-21T04:15:19.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/537: write d3/d34/d78/dbf/fbd [175562,82941] 0 2024-09-21T04:15:19.645 INFO:tasks.workunit.client.1.smithi148.stdout:3/511: dread d3/d1f/f47 [8388608,4194304] 0 2024-09-21T04:15:19.645 INFO:tasks.workunit.client.0.smithi110.stdout:7/534: dread dd/f1a [0,4194304] 0 2024-09-21T04:15:19.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/567: mknod dc/d13/d76/d5d/d78/cad 0 2024-09-21T04:15:19.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/568: truncate dc/f7d 4288304 0 2024-09-21T04:15:19.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/569: fsync dc/d13/f24 0 2024-09-21T04:15:19.647 INFO:tasks.workunit.client.0.smithi110.stdout:1/866: getdents d2/d9/d23/de0 0 2024-09-21T04:15:19.648 INFO:tasks.workunit.client.1.smithi148.stdout:5/538: rmdir d3/d14/d1f/d26/d59 39 2024-09-21T04:15:19.650 INFO:tasks.workunit.client.1.smithi148.stdout:0/580: creat d5/d6/d7/d11/d7b/fb8 x:0 0 0 2024-09-21T04:15:19.651 INFO:tasks.workunit.client.1.smithi148.stdout:8/570: symlink dc/d13/d3c/lae 0 2024-09-21T04:15:19.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/539: rmdir d3/d34/d3c/d5d 39 2024-09-21T04:15:19.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/540: readlink d3/d14/d1f/d26/l86 0 2024-09-21T04:15:19.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/541: write d3/d34/d78/dbf/f96 [1489633,63946] 0 2024-09-21T04:15:19.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/542: chown d3/d34/d78 46866 1 2024-09-21T04:15:19.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/543: chown d3/d34/d3c/l40 1191632872 1 2024-09-21T04:15:19.654 INFO:tasks.workunit.client.0.smithi110.stdout:7/535: symlink dd/d1c/d39/d56/d66/da6/d7a/d9b/la8 0 2024-09-21T04:15:19.654 INFO:tasks.workunit.client.0.smithi110.stdout:7/536: dread - dd/d1c/d39/d56/d66/da6/f8e zero size 2024-09-21T04:15:19.654 INFO:tasks.workunit.client.0.smithi110.stdout:7/537: dread - dd/d4d/f71 zero size 2024-09-21T04:15:19.655 INFO:tasks.workunit.client.1.smithi148.stdout:5/544: truncate d3/d14/d1f/d26/d2e/f8f 2427500 0 2024-09-21T04:15:19.656 INFO:tasks.workunit.client.0.smithi110.stdout:3/472: dwrite d0/d12/f6c [0,4194304] 0 2024-09-21T04:15:19.656 INFO:tasks.workunit.client.0.smithi110.stdout:1/867: creat d2/d9/d23/d3b/dca/f110 x:0 0 0 2024-09-21T04:15:19.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/512: dread d3/d5/f1e [0,4194304] 0 2024-09-21T04:15:19.657 INFO:tasks.workunit.client.0.smithi110.stdout:1/868: symlink d2/d9/l111 0 2024-09-21T04:15:19.657 INFO:tasks.workunit.client.0.smithi110.stdout:1/869: creat d2/d9/d49/d4f/d53/f112 x:0 0 0 2024-09-21T04:15:19.661 INFO:tasks.workunit.client.1.smithi148.stdout:5/545: rmdir d3/d14/d1f/db1/db8 39 2024-09-21T04:15:19.662 INFO:tasks.workunit.client.0.smithi110.stdout:9/504: dwrite d7/d96/d99/d3c/d35/f9c [0,4194304] 0 2024-09-21T04:15:19.662 INFO:tasks.workunit.client.0.smithi110.stdout:9/505: readlink d7/d96/d99/d3c/d1f/d43/l62 0 2024-09-21T04:15:19.663 INFO:tasks.workunit.client.1.smithi148.stdout:5/546: dread d3/d14/d1f/f23 [0,4194304] 0 2024-09-21T04:15:19.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/667: dwrite d0/d9/d16/f60 [0,4194304] 0 2024-09-21T04:15:19.664 INFO:tasks.workunit.client.1.smithi148.stdout:1/547: dwrite db/dd/f8c [0,4194304] 0 2024-09-21T04:15:19.665 INFO:tasks.workunit.client.1.smithi148.stdout:4/616: fsync d4/d12/d16/d29/f59 0 2024-09-21T04:15:19.665 INFO:tasks.workunit.client.1.smithi148.stdout:4/617: readlink d4/d66/d7f/lb7 0 2024-09-21T04:15:19.665 INFO:tasks.workunit.client.1.smithi148.stdout:4/618: readlink d4/d12/da7/la8 0 2024-09-21T04:15:19.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/551: dwrite d5/d8/d6f/d7d/f7e [0,4194304] 0 2024-09-21T04:15:19.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/552: dread - d5/fa7 zero size 2024-09-21T04:15:19.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/619: write d4/f98 [149196,122127] 0 2024-09-21T04:15:19.666 INFO:tasks.workunit.client.0.smithi110.stdout:7/538: dread dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:19.668 INFO:tasks.workunit.client.0.smithi110.stdout:9/506: symlink d7/d96/d99/d3c/d11/laf 0 2024-09-21T04:15:19.668 INFO:tasks.workunit.client.1.smithi148.stdout:8/571: dread dc/d3e/f70 [4194304,4194304] 0 2024-09-21T04:15:19.670 INFO:tasks.workunit.client.1.smithi148.stdout:5/547: mkdir d3/d34/d78/dbf/dc2 0 2024-09-21T04:15:19.671 INFO:tasks.workunit.client.0.smithi110.stdout:1/870: rename d2/d9/d49/c6e to d2/d9/d49/d5c/d91/c113 0 2024-09-21T04:15:19.672 INFO:tasks.workunit.client.0.smithi110.stdout:1/871: chown d2/d9/d49/f99 2 1 2024-09-21T04:15:19.676 INFO:tasks.workunit.client.0.smithi110.stdout:7/539: symlink dd/d1c/d39/d56/d66/da6/da7/la9 0 2024-09-21T04:15:19.676 INFO:tasks.workunit.client.0.smithi110.stdout:7/540: readlink dd/d1c/l47 0 2024-09-21T04:15:19.677 INFO:tasks.workunit.client.1.smithi148.stdout:2/668: rename d0/d2f/c6c to d0/d9/d16/d9e/cca 0 2024-09-21T04:15:19.677 INFO:tasks.workunit.client.1.smithi148.stdout:2/669: write d0/d2b/f5e [4543396,92909] 0 2024-09-21T04:15:19.679 INFO:tasks.workunit.client.0.smithi110.stdout:1/872: rename d2/db7/f5f to d2/d9/d49/d5c/d91/d101/f114 0 2024-09-21T04:15:19.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/553: dread d5/d14/d25/d40/dab/dad/fb4 [0,4194304] 0 2024-09-21T04:15:19.681 INFO:tasks.workunit.client.0.smithi110.stdout:7/541: symlink dd/d1c/d39/d56/d66/da6/d7a/d82/laa 0 2024-09-21T04:15:19.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/572: write dc/f21 [3795624,108843] 0 2024-09-21T04:15:19.683 INFO:tasks.workunit.client.1.smithi148.stdout:1/548: mkdir db/d24/d2e/d71/dad 0 2024-09-21T04:15:19.684 INFO:tasks.workunit.client.0.smithi110.stdout:1/873: rename d2/d9/d49/d4f/d53/f104 to d2/d9/d49/d7a/dee/f115 0 2024-09-21T04:15:19.685 INFO:tasks.workunit.client.0.smithi110.stdout:1/874: dread d2/d9/d49/d5c/d91/f96 [0,4194304] 0 2024-09-21T04:15:19.686 INFO:tasks.workunit.client.0.smithi110.stdout:7/542: mknod dd/d48/cab 0 2024-09-21T04:15:19.688 INFO:tasks.workunit.client.1.smithi148.stdout:6/563: dwrite d0/d1a/d23/d53/f9f [0,4194304] 0 2024-09-21T04:15:19.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/875: symlink d2/d9/d49/d7a/dee/de1/l116 0 2024-09-21T04:15:19.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/876: chown d2/d9/d49/d7a/dee/de1 844 1 2024-09-21T04:15:19.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/877: readlink d2/d9/d26/d3d/d8f/l98 0 2024-09-21T04:15:19.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/878: truncate d2/d9/d23/de0/ff5 67714 0 2024-09-21T04:15:19.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/879: stat d2/d9/d49/d5c/d91 0 2024-09-21T04:15:19.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/620: link d4/d87/c31 d4/d12/d16/d29/d61/d6c/d9d/daf/cb8 0 2024-09-21T04:15:19.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/621: write d4/f10 [3372373,92079] 0 2024-09-21T04:15:19.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/622: read d4/d12/f41 [3696912,37280] 0 2024-09-21T04:15:19.694 INFO:tasks.workunit.client.1.smithi148.stdout:5/548: rename d3/d34/d3c/fa8 to d3/d14/d1f/d26/d2e/d56/fc3 0 2024-09-21T04:15:19.696 INFO:tasks.workunit.client.0.smithi110.stdout:1/880: link d2/d9/d23/d3b/f55 d2/d9/d26/f117 0 2024-09-21T04:15:19.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/670: mkdir d0/d9/d16/d55/d71/dc2/dcb 0 2024-09-21T04:15:19.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/671: write d0/d9/dc/d13/f1a [887049,116266] 0 2024-09-21T04:15:19.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/672: dread - d0/d9/d16/f5b zero size 2024-09-21T04:15:19.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/673: truncate d0/d9/dc/f24 4514445 0 2024-09-21T04:15:19.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/674: readlink d0/d9/dc/l5d 0 2024-09-21T04:15:19.700 INFO:tasks.workunit.client.0.smithi110.stdout:7/543: dwrite dd/d1c/d39/f76 [0,4194304] 0 2024-09-21T04:15:19.701 INFO:tasks.workunit.client.1.smithi148.stdout:7/554: rmdir d5/d14/d25/d40 39 2024-09-21T04:15:19.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/549: dread d3/d34/d3c/d5d/f95 [0,4194304] 0 2024-09-21T04:15:19.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/550: readlink d3/d14/l5a 0 2024-09-21T04:15:19.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/551: dread - d3/d24/fb0 zero size 2024-09-21T04:15:19.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/552: creat d3/d14/d1f/d26/d2e/d56/fc4 x:0 0 0 2024-09-21T04:15:19.703 INFO:tasks.workunit.client.0.smithi110.stdout:1/881: symlink d2/d9/d26/d3d/d8f/de7/l118 0 2024-09-21T04:15:19.705 INFO:tasks.workunit.client.1.smithi148.stdout:8/573: creat dc/d13/d39/d6e/dac/faf x:0 0 0 2024-09-21T04:15:19.709 INFO:tasks.workunit.client.1.smithi148.stdout:0/581: dwrite d5/d6/d2a/d3f/d6d/faa [0,4194304] 0 2024-09-21T04:15:19.711 INFO:tasks.workunit.client.1.smithi148.stdout:9/580: dwrite d5/f42 [0,4194304] 0 2024-09-21T04:15:19.711 INFO:tasks.workunit.client.0.smithi110.stdout:3/473: dwrite d0/d22/d29/d76/fa6 [0,4194304] 0 2024-09-21T04:15:19.711 INFO:tasks.workunit.client.0.smithi110.stdout:3/474: readlink d0/d12/d3c/l9e 0 2024-09-21T04:15:19.714 INFO:tasks.workunit.client.1.smithi148.stdout:6/564: creat d0/d1a/d23/dc/d68/fa6 x:0 0 0 2024-09-21T04:15:19.715 INFO:tasks.workunit.client.0.smithi110.stdout:1/882: unlink d2/d9/d23/d4e/cf1 0 2024-09-21T04:15:19.715 INFO:tasks.workunit.client.0.smithi110.stdout:1/883: write d2/d9/d49/d4f/df0/f108 [663858,1846] 0 2024-09-21T04:15:19.719 INFO:tasks.workunit.client.1.smithi148.stdout:3/513: dwrite d3/d5/f35 [4194304,4194304] 0 2024-09-21T04:15:19.719 INFO:tasks.workunit.client.1.smithi148.stdout:2/675: unlink d0/d9/dc/d13/d50/fb5 0 2024-09-21T04:15:19.720 INFO:tasks.workunit.client.0.smithi110.stdout:3/475: write d0/d12/d16/d1a/d5c/f66 [1614493,64714] 0 2024-09-21T04:15:19.721 INFO:tasks.workunit.client.1.smithi148.stdout:3/514: dread d3/f85 [0,4194304] 0 2024-09-21T04:15:19.722 INFO:tasks.workunit.client.1.smithi148.stdout:7/555: symlink d5/d8/d6f/d7d/d90/d28/lc8 0 2024-09-21T04:15:19.727 INFO:tasks.workunit.client.1.smithi148.stdout:7/556: dread d5/d14/d25/d79/d86/fa3 [0,4194304] 0 2024-09-21T04:15:19.727 INFO:tasks.workunit.client.1.smithi148.stdout:7/557: chown d5/f5a 666 1 2024-09-21T04:15:19.730 INFO:tasks.workunit.client.1.smithi148.stdout:4/623: dwrite d4/d66/d7f/f4a [0,4194304] 0 2024-09-21T04:15:19.732 INFO:tasks.workunit.client.1.smithi148.stdout:8/574: dwrite dc/d43/d4c/d4f/f8e [0,4194304] 0 2024-09-21T04:15:19.732 INFO:tasks.workunit.client.0.smithi110.stdout:1/884: mknod d2/d9/d49/d4f/df0/c119 0 2024-09-21T04:15:19.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/544: getdents dd/d1c/d39/d56/d66/da6/da7 0 2024-09-21T04:15:19.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/545: creat dd/d1c/d39/d56/fac x:0 0 0 2024-09-21T04:15:19.737 INFO:tasks.workunit.client.1.smithi148.stdout:9/581: symlink d5/d1f/ld6 0 2024-09-21T04:15:19.739 INFO:tasks.workunit.client.0.smithi110.stdout:3/476: dwrite d0/d12/d16/d1a/d5c/f85 [0,4194304] 0 2024-09-21T04:15:19.739 INFO:tasks.workunit.client.0.smithi110.stdout:3/477: fdatasync d0/d12/d16/d1a/f86 0 2024-09-21T04:15:19.739 INFO:tasks.workunit.client.0.smithi110.stdout:3/478: truncate d0/f7 4446821 0 2024-09-21T04:15:19.739 INFO:tasks.workunit.client.1.smithi148.stdout:8/575: dread dc/d13/d28/f48 [0,4194304] 0 2024-09-21T04:15:19.742 INFO:tasks.workunit.client.0.smithi110.stdout:1/885: getdents d2/d9/d23/d4e/d57/dc9 0 2024-09-21T04:15:19.748 INFO:tasks.workunit.client.0.smithi110.stdout:1/886: rename d2/d9/d49/d5c/f84 to d2/d9/d26/d3d/f11a 0 2024-09-21T04:15:19.752 INFO:tasks.workunit.client.1.smithi148.stdout:1/549: rename db/d46/f5f to db/dd/fae 0 2024-09-21T04:15:19.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/676: unlink d0/d9/d16/l34 0 2024-09-21T04:15:19.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/677: write d0/d9/dc/f99 [2437251,7835] 0 2024-09-21T04:15:19.756 INFO:tasks.workunit.client.1.smithi148.stdout:2/678: write d0/d2b/da3/d8f/fa4 [595908,124645] 0 2024-09-21T04:15:19.757 INFO:tasks.workunit.client.1.smithi148.stdout:2/679: readlink d0/d9/dc/lc3 0 2024-09-21T04:15:19.762 INFO:tasks.workunit.client.0.smithi110.stdout:3/479: dwrite d0/d12/d3c/f4f [0,4194304] 0 2024-09-21T04:15:19.764 INFO:tasks.workunit.client.0.smithi110.stdout:4/703: sync 2024-09-21T04:15:19.764 INFO:tasks.workunit.client.0.smithi110.stdout:5/603: sync 2024-09-21T04:15:19.764 INFO:tasks.workunit.client.0.smithi110.stdout:2/381: sync 2024-09-21T04:15:19.764 INFO:tasks.workunit.client.0.smithi110.stdout:6/588: sync 2024-09-21T04:15:19.766 INFO:tasks.workunit.client.0.smithi110.stdout:6/589: dread d4/d78/daa/f84 [0,4194304] 0 2024-09-21T04:15:19.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/382: rmdir d2/d1a/d1f/d2a 39 2024-09-21T04:15:19.770 INFO:tasks.workunit.client.0.smithi110.stdout:6/590: truncate d4/d12/d17/d33/d3e/f53 3644690 0 2024-09-21T04:15:19.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/383: mknod d2/d1a/d1f/d2a/c72 0 2024-09-21T04:15:19.775 INFO:tasks.workunit.client.1.smithi148.stdout:6/565: rename d0/f20 to d0/d34/d97/d4f/fa7 0 2024-09-21T04:15:19.775 INFO:tasks.workunit.client.1.smithi148.stdout:6/566: creat d0/d1a/d23/d32/d93/fa8 x:0 0 0 2024-09-21T04:15:19.775 INFO:tasks.workunit.client.0.smithi110.stdout:3/480: write d0/d12/d16/d1a/f86 [670015,95807] 0 2024-09-21T04:15:19.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/550: getdents db/d24/d2a 0 2024-09-21T04:15:19.778 INFO:tasks.workunit.client.0.smithi110.stdout:3/481: write d0/d12/d16/f25 [31786,26192] 0 2024-09-21T04:15:19.779 INFO:tasks.workunit.client.0.smithi110.stdout:7/546: dwrite dd/d1c/d39/f4f [0,4194304] 0 2024-09-21T04:15:19.781 INFO:tasks.workunit.client.1.smithi148.stdout:2/680: creat d0/d9/d38/fcc x:0 0 0 2024-09-21T04:15:19.782 INFO:tasks.workunit.client.1.smithi148.stdout:3/515: rmdir d3/d84 39 2024-09-21T04:15:19.782 INFO:tasks.workunit.client.0.smithi110.stdout:3/482: read d0/d12/f82 [168113,60653] 0 2024-09-21T04:15:19.782 INFO:tasks.workunit.client.0.smithi110.stdout:3/483: chown d0/f56 122096685 1 2024-09-21T04:15:19.782 INFO:tasks.workunit.client.0.smithi110.stdout:3/484: fdatasync d0/f56 0 2024-09-21T04:15:19.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/485: readlink d0/l18 0 2024-09-21T04:15:19.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/486: chown d0/d22/d29/d76/fa6 3054053 1 2024-09-21T04:15:19.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/487: dread - d0/d51/d7b/f8d zero size 2024-09-21T04:15:19.784 INFO:tasks.workunit.client.0.smithi110.stdout:1/887: dwrite d2/d9/d49/d7a/da8/fd3 [0,4194304] 0 2024-09-21T04:15:19.784 INFO:tasks.workunit.client.0.smithi110.stdout:4/704: dread d17/d34/d3d/d20/d43/d51/d7a/fa7 [4194304,4194304] 0 2024-09-21T04:15:19.784 INFO:tasks.workunit.client.1.smithi148.stdout:4/624: getdents d4 0 2024-09-21T04:15:19.785 INFO:tasks.workunit.client.0.smithi110.stdout:5/604: dwrite d1/d39/f79 [4194304,4194304] 0 2024-09-21T04:15:19.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/488: rename d0/l18 to d0/d12/d16/d1a/la9 0 2024-09-21T04:15:19.787 INFO:tasks.workunit.client.0.smithi110.stdout:6/591: dread d4/f79 [0,4194304] 0 2024-09-21T04:15:19.788 INFO:tasks.workunit.client.1.smithi148.stdout:7/558: rename d5/d8/d6f/d7d/f5c to d5/d8/d6f/fc9 0 2024-09-21T04:15:19.788 INFO:tasks.workunit.client.0.smithi110.stdout:7/547: rmdir dd/d1c/d39/d56 39 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/567: rmdir d0/d1a/d23/dc/d81/d99 0 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/568: chown d0/d1a/d23/ca 2 1 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/569: fdatasync d0/d1a/d23/f82 0 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/551: mkdir db/d24/d2e/daf 0 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/552: write db/dd/d21/f93 [74989,73379] 0 2024-09-21T04:15:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:2/681: getdents d0/d9/dc/d13/d25 0 2024-09-21T04:15:19.795 INFO:tasks.workunit.client.0.smithi110.stdout:1/888: mkdir d2/d9/d26/d3d/d8f/d11b 0 2024-09-21T04:15:19.795 INFO:tasks.workunit.client.0.smithi110.stdout:6/592: mkdir d4/d12/d17/d2e/d40/dbc 0 2024-09-21T04:15:19.795 INFO:tasks.workunit.client.0.smithi110.stdout:6/593: write f3 [4298850,13756] 0 2024-09-21T04:15:19.795 INFO:tasks.workunit.client.1.smithi148.stdout:4/625: mknod d4/d12/cb9 0 2024-09-21T04:15:19.796 INFO:tasks.workunit.client.0.smithi110.stdout:4/705: dread d17/f1a [0,4194304] 0 2024-09-21T04:15:19.797 INFO:tasks.workunit.client.0.smithi110.stdout:7/548: rename dd/d1c/d39/d6d/d80 to dd/d1c/dad 0 2024-09-21T04:15:19.797 INFO:tasks.workunit.client.0.smithi110.stdout:7/549: chown dd/d48/d9f 3615 1 2024-09-21T04:15:19.799 INFO:tasks.workunit.client.0.smithi110.stdout:1/889: mkdir d2/df3/dfb/d11c 0 2024-09-21T04:15:19.804 INFO:tasks.workunit.client.1.smithi148.stdout:9/582: rename d5/d1e/d37/d46/f7e to d5/d2b/d4e/db3/fd7 0 2024-09-21T04:15:19.805 INFO:tasks.workunit.client.0.smithi110.stdout:6/594: unlink d4/l1a 0 2024-09-21T04:15:19.805 INFO:tasks.workunit.client.0.smithi110.stdout:6/595: creat d4/d12/d16/d66/d7a/d63/fbd x:0 0 0 2024-09-21T04:15:19.808 INFO:tasks.workunit.client.1.smithi148.stdout:7/559: creat d5/d14/d25/d79/d86/fca x:0 0 0 2024-09-21T04:15:19.808 INFO:tasks.workunit.client.0.smithi110.stdout:1/890: creat d2/d9/d23/de0/f11d x:0 0 0 2024-09-21T04:15:19.809 INFO:tasks.workunit.client.0.smithi110.stdout:1/891: read d2/fd0 [232489,115470] 0 2024-09-21T04:15:19.809 INFO:tasks.workunit.client.1.smithi148.stdout:1/553: mkdir db/dd/d21/db0 0 2024-09-21T04:15:19.811 INFO:tasks.workunit.client.1.smithi148.stdout:2/682: mkdir d0/d2b/da3/da1/dcd 0 2024-09-21T04:15:19.811 INFO:tasks.workunit.client.1.smithi148.stdout:9/583: symlink d5/d1f/d81/dcf/ld8 0 2024-09-21T04:15:19.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/892: symlink d2/d9/d49/d4f/d53/l11e 0 2024-09-21T04:15:19.812 INFO:tasks.workunit.client.0.smithi110.stdout:1/893: fsync d2/fd0 0 2024-09-21T04:15:19.813 INFO:tasks.workunit.client.1.smithi148.stdout:1/554: unlink db/d24/d2a/f42 0 2024-09-21T04:15:19.814 INFO:tasks.workunit.client.1.smithi148.stdout:2/683: rename d0/d2b/da3/d82/dae/lbe to d0/d9/d16/lce 0 2024-09-21T04:15:19.814 INFO:tasks.workunit.client.1.smithi148.stdout:6/570: dwrite d0/d34/d97/f1f [0,4194304] 0 2024-09-21T04:15:19.815 INFO:tasks.workunit.client.1.smithi148.stdout:9/584: rmdir d5/d60/d28/d2f/dbf 39 2024-09-21T04:15:19.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/555: dread db/dd/f12 [0,4194304] 0 2024-09-21T04:15:19.816 INFO:tasks.workunit.client.0.smithi110.stdout:1/894: link d2/d9/d49/fbe d2/d9/d49/d5c/d91/d101/d103/f11f 0 2024-09-21T04:15:19.816 INFO:tasks.workunit.client.0.smithi110.stdout:1/895: chown d2/d9/d26/d3d/d8f/fa3 1599048 1 2024-09-21T04:15:19.817 INFO:tasks.workunit.client.1.smithi148.stdout:6/571: rename d0/d1a/d23/f82 to d0/d34/da0/fa9 0 2024-09-21T04:15:19.818 INFO:tasks.workunit.client.1.smithi148.stdout:9/585: mknod d5/d60/d28/d2f/dbf/dc2/cd9 0 2024-09-21T04:15:19.818 INFO:tasks.workunit.client.1.smithi148.stdout:9/586: fsync d5/d2b/fab 0 2024-09-21T04:15:19.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/556: creat db/d24/d2e/daf/fb1 x:0 0 0 2024-09-21T04:15:19.819 INFO:tasks.workunit.client.1.smithi148.stdout:6/572: truncate d0/d1a/d23/f14 250281 0 2024-09-21T04:15:19.819 INFO:tasks.workunit.client.1.smithi148.stdout:9/587: link d5/d1e/d37/l7c d5/d60/d28/d2f/dbf/lda 0 2024-09-21T04:15:19.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/573: rename d0/f21 to d0/d1a/d23/d32/faa 0 2024-09-21T04:15:19.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/574: write d0/f6b [257173,33570] 0 2024-09-21T04:15:19.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/575: chown d0/d1a/d23/l31 3676 1 2024-09-21T04:15:19.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/576: write d0/d1a/d23/d32/d93/fa8 [581601,27708] 0 2024-09-21T04:15:19.821 INFO:tasks.workunit.client.1.smithi148.stdout:9/588: mknod d5/d1e/d37/cdb 0 2024-09-21T04:15:19.822 INFO:tasks.workunit.client.1.smithi148.stdout:6/577: unlink d0/l47 0 2024-09-21T04:15:19.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/589: truncate d5/d39/f6f 6250821 0 2024-09-21T04:15:19.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/590: getdents d5/d1f/d81/d9e/da2 0 2024-09-21T04:15:19.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/591: fsync d5/d2b/fab 0 2024-09-21T04:15:19.824 INFO:tasks.workunit.client.0.smithi110.stdout:7/550: dwrite dd/d4d/f59 [0,4194304] 0 2024-09-21T04:15:19.824 INFO:tasks.workunit.client.1.smithi148.stdout:9/592: mknod d5/d1f/d81/d9e/da8/cdc 0 2024-09-21T04:15:19.824 INFO:tasks.workunit.client.1.smithi148.stdout:9/593: fdatasync d5/d60/d4c/dbd/f99 0 2024-09-21T04:15:19.827 INFO:tasks.workunit.client.1.smithi148.stdout:9/594: creat d5/d2b/dce/fdd x:0 0 0 2024-09-21T04:15:19.830 INFO:tasks.workunit.client.1.smithi148.stdout:1/557: dread db/dd/d21/d89/fab [0,4194304] 0 2024-09-21T04:15:19.832 INFO:tasks.workunit.client.1.smithi148.stdout:9/595: rmdir d5/d2b/d4e/d95 39 2024-09-21T04:15:19.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/558: link db/d24/d2e/d71/d94/d9a/c69 db/d24/d2e/d71/dad/cb2 0 2024-09-21T04:15:19.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/559: truncate db/d46/f6a 676069 0 2024-09-21T04:15:19.834 INFO:tasks.workunit.client.0.smithi110.stdout:7/551: write dd/d14/d17/f3a [7783637,10451] 0 2024-09-21T04:15:19.834 INFO:tasks.workunit.client.0.smithi110.stdout:1/896: dread d2/d9/d26/f4d [0,4194304] 0 2024-09-21T04:15:19.834 INFO:tasks.workunit.client.0.smithi110.stdout:1/897: dread - d2/db7/d86/f95 zero size 2024-09-21T04:15:19.835 INFO:tasks.workunit.client.1.smithi148.stdout:9/596: link d5/d60/cb1 d5/d39/dba/cde 0 2024-09-21T04:15:19.838 INFO:tasks.workunit.client.0.smithi110.stdout:4/706: dread d17/d1e/d7b/f5e [0,4194304] 0 2024-09-21T04:15:19.839 INFO:tasks.workunit.client.0.smithi110.stdout:4/707: readlink d17/d1e/d21/l4f 0 2024-09-21T04:15:19.839 INFO:tasks.workunit.client.1.smithi148.stdout:2/684: dread d0/d9/d16/d55/d6f/f3a [0,4194304] 0 2024-09-21T04:15:19.840 INFO:tasks.workunit.client.1.smithi148.stdout:1/560: creat db/d24/d2a/da4/fb3 x:0 0 0 2024-09-21T04:15:19.841 INFO:tasks.workunit.client.1.smithi148.stdout:9/597: link d5/d60/d28/f30 d5/d60/d28/d73/fdf 0 2024-09-21T04:15:19.842 INFO:tasks.workunit.client.0.smithi110.stdout:2/384: read d2/d1a/d1f/d29/f4f [2682036,128453] 0 2024-09-21T04:15:19.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/605: read d1/d7/d58/f60 [416706,89442] 0 2024-09-21T04:15:19.843 INFO:tasks.workunit.client.1.smithi148.stdout:4/626: dwrite d4/f98 [0,4194304] 0 2024-09-21T04:15:19.844 INFO:tasks.workunit.client.0.smithi110.stdout:4/708: symlink d17/led 0 2024-09-21T04:15:19.844 INFO:tasks.workunit.client.0.smithi110.stdout:4/709: write d17/d34/d3d/d20/d43/f8c [3791590,28487] 0 2024-09-21T04:15:19.844 INFO:tasks.workunit.client.0.smithi110.stdout:2/385: read d2/f3b [767682,56824] 0 2024-09-21T04:15:19.845 INFO:tasks.workunit.client.0.smithi110.stdout:7/552: dread dd/d1c/d39/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:19.845 INFO:tasks.workunit.client.0.smithi110.stdout:7/553: creat dd/fae x:0 0 0 2024-09-21T04:15:19.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/516: dwrite d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:19.847 INFO:tasks.workunit.client.1.smithi148.stdout:2/685: rename d0/l8 to d0/d9/d38/lcf 0 2024-09-21T04:15:19.849 INFO:tasks.workunit.client.1.smithi148.stdout:1/561: creat db/d24/d2e/d59/da7/fb4 x:0 0 0 2024-09-21T04:15:19.849 INFO:tasks.workunit.client.1.smithi148.stdout:1/562: chown db/d24/d2e/d59/l5d 1 1 2024-09-21T04:15:19.850 INFO:tasks.workunit.client.0.smithi110.stdout:5/606: creat d1/d49/d69/fca x:0 0 0 2024-09-21T04:15:19.851 INFO:tasks.workunit.client.1.smithi148.stdout:9/598: symlink d5/d2b/d4e/le0 0 2024-09-21T04:15:19.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/578: dwrite d0/d8/f49 [0,4194304] 0 2024-09-21T04:15:19.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/579: chown d0/d1a/d23/dc/d68/fa6 98510 1 2024-09-21T04:15:19.853 INFO:tasks.workunit.client.1.smithi148.stdout:6/580: creat d0/d1a/fab x:0 0 0 2024-09-21T04:15:19.853 INFO:tasks.workunit.client.0.smithi110.stdout:4/710: rename d17/d1e/d7b/dc5/fdd to d17/d34/d3d/d57/fee 0 2024-09-21T04:15:19.854 INFO:tasks.workunit.client.1.smithi148.stdout:2/686: dread d0/d9/f17 [0,4194304] 0 2024-09-21T04:15:19.854 INFO:tasks.workunit.client.0.smithi110.stdout:4/711: read d17/d34/d3d/d20/d43/d51/d7a/d86/fae [47313,66314] 0 2024-09-21T04:15:19.854 INFO:tasks.workunit.client.0.smithi110.stdout:2/386: truncate d2/f44 4142178 0 2024-09-21T04:15:19.854 INFO:tasks.workunit.client.0.smithi110.stdout:2/387: readlink d2/d1a/d1f/d5f/l55 0 2024-09-21T04:15:19.856 INFO:tasks.workunit.client.0.smithi110.stdout:6/596: dwrite d4/d12/d16/d66/d7a/d83/f9f [0,4194304] 0 2024-09-21T04:15:19.857 INFO:tasks.workunit.client.0.smithi110.stdout:6/597: stat d4/d3c/la5 0 2024-09-21T04:15:19.857 INFO:tasks.workunit.client.1.smithi148.stdout:4/627: unlink d4/d87/d2d/l69 0 2024-09-21T04:15:19.857 INFO:tasks.workunit.client.1.smithi148.stdout:4/628: creat d4/d87/fba x:0 0 0 2024-09-21T04:15:19.857 INFO:tasks.workunit.client.1.smithi148.stdout:4/629: creat d4/d12/d75/d85/fbb x:0 0 0 2024-09-21T04:15:19.857 INFO:tasks.workunit.client.1.smithi148.stdout:4/630: fdatasync d4/f6 0 2024-09-21T04:15:19.859 INFO:tasks.workunit.client.0.smithi110.stdout:5/607: mkdir d1/d7/d58/d72/dcb 0 2024-09-21T04:15:19.859 INFO:tasks.workunit.client.0.smithi110.stdout:5/608: dread - d1/d39/f63 zero size 2024-09-21T04:15:19.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/554: rename dd/d4d/f59 to dd/d1c/d39/d4a/d52/faf 0 2024-09-21T04:15:19.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/555: write dd/d1c/d39/d56/d66/da6/d7a/f97 [282654,10258] 0 2024-09-21T04:15:19.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/712: rmdir d17/d34/d3d/d20 39 2024-09-21T04:15:19.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/713: read d17/d34/d3d/d20/d43/d51/d52/fb1 [1538445,25285] 0 2024-09-21T04:15:19.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/388: symlink d2/d1a/d1f/d5f/l73 0 2024-09-21T04:15:19.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/517: unlink d3/d5/d6/d12/d15/l78 0 2024-09-21T04:15:19.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/518: chown d3/f85 389126 1 2024-09-21T04:15:19.869 INFO:tasks.workunit.client.1.smithi148.stdout:4/631: dread d4/d12/d16/d29/f46 [0,4194304] 0 2024-09-21T04:15:19.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/598: mkdir d4/d12/d17/d9a/dbe 0 2024-09-21T04:15:19.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/599: readlink d4/d12/l6f 0 2024-09-21T04:15:19.871 INFO:tasks.workunit.client.1.smithi148.stdout:1/563: mknod db/d24/d2e/d71/cb5 0 2024-09-21T04:15:19.873 INFO:tasks.workunit.client.1.smithi148.stdout:1/564: dread db/dd/d21/d89/fab [4194304,4194304] 0 2024-09-21T04:15:19.873 INFO:tasks.workunit.client.1.smithi148.stdout:1/565: creat db/d46/fb6 x:0 0 0 2024-09-21T04:15:19.873 INFO:tasks.workunit.client.0.smithi110.stdout:5/609: rename d1/d22/l56 to d1/d49/d4d/d1e/d4a/dae/lcc 0 2024-09-21T04:15:19.874 INFO:tasks.workunit.client.1.smithi148.stdout:9/599: getdents d5/d2b/dbe 0 2024-09-21T04:15:19.875 INFO:tasks.workunit.client.1.smithi148.stdout:9/600: stat d5/d1f/c20 0 2024-09-21T04:15:19.876 INFO:tasks.workunit.client.0.smithi110.stdout:7/556: write dd/d1c/d39/d56/d66/da6/f5f [134579,74540] 0 2024-09-21T04:15:19.876 INFO:tasks.workunit.client.0.smithi110.stdout:7/557: write dd/d1c/d39/d56/f94 [4882885,75558] 0 2024-09-21T04:15:19.876 INFO:tasks.workunit.client.1.smithi148.stdout:9/601: dread d5/d2b/d4e/fa3 [0,4194304] 0 2024-09-21T04:15:19.876 INFO:tasks.workunit.client.1.smithi148.stdout:9/602: truncate d5/d1e/d37/d50/f55 902653 0 2024-09-21T04:15:19.879 INFO:tasks.workunit.client.0.smithi110.stdout:7/558: dread dd/d1c/d39/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:19.879 INFO:tasks.workunit.client.0.smithi110.stdout:7/559: chown dd/d14/c42 11240285 1 2024-09-21T04:15:19.879 INFO:tasks.workunit.client.0.smithi110.stdout:7/560: fdatasync dd/d1c/f34 0 2024-09-21T04:15:19.880 INFO:tasks.workunit.client.0.smithi110.stdout:4/714: mkdir d17/d3e/d6b/def 0 2024-09-21T04:15:19.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/389: mknod d2/d1a/d1f/d5f/c74 0 2024-09-21T04:15:19.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/390: chown d2/d1a/d1f/d29 409 1 2024-09-21T04:15:19.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/391: truncate d2/d1a/d1f/d65/f68 735289 0 2024-09-21T04:15:19.886 INFO:tasks.workunit.client.1.smithi148.stdout:1/566: dread db/dd/f8c [0,4194304] 0 2024-09-21T04:15:19.886 INFO:tasks.workunit.client.0.smithi110.stdout:6/600: truncate d4/d12/d16/d66/d7a/d4b/fa1 3117156 0 2024-09-21T04:15:19.886 INFO:tasks.workunit.client.0.smithi110.stdout:6/601: truncate d4/f13 7104691 0 2024-09-21T04:15:19.886 INFO:tasks.workunit.client.0.smithi110.stdout:6/602: chown d4/d78/daa/lb5 1241 1 2024-09-21T04:15:19.888 INFO:tasks.workunit.client.1.smithi148.stdout:6/581: link d0/d8/d18/c6d d0/d1a/d23/d32/d93/d96/cac 0 2024-09-21T04:15:19.890 INFO:tasks.workunit.client.0.smithi110.stdout:5/610: symlink d1/d7/d18/d8f/lcd 0 2024-09-21T04:15:19.890 INFO:tasks.workunit.client.0.smithi110.stdout:5/611: readlink d1/d39/l2a 0 2024-09-21T04:15:19.890 INFO:tasks.workunit.client.0.smithi110.stdout:5/612: creat d1/fce x:0 0 0 2024-09-21T04:15:19.891 INFO:tasks.workunit.client.1.smithi148.stdout:9/603: dwrite d5/d1e/f29 [0,4194304] 0 2024-09-21T04:15:19.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/687: rename d0/d2b/lb3 to d0/ld0 0 2024-09-21T04:15:19.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/688: chown d0/d2b/da3/d95 324008 1 2024-09-21T04:15:19.892 INFO:tasks.workunit.client.1.smithi148.stdout:9/604: dread d5/d1f/f4b [0,4194304] 0 2024-09-21T04:15:19.892 INFO:tasks.workunit.client.1.smithi148.stdout:9/605: dread - d5/d1e/faa zero size 2024-09-21T04:15:19.894 INFO:tasks.workunit.client.0.smithi110.stdout:7/561: unlink dd/d1c/d39/f76 0 2024-09-21T04:15:19.896 INFO:tasks.workunit.client.0.smithi110.stdout:4/715: rmdir d17/d34/d3d/d20/d43/d51/d7a/d96 39 2024-09-21T04:15:19.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/392: creat d2/d1a/d1f/d65/f75 x:0 0 0 2024-09-21T04:15:19.900 INFO:tasks.workunit.client.1.smithi148.stdout:3/519: mkdir d3/d1f/d4d/d82/d23/da1 0 2024-09-21T04:15:19.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/520: write d3/d5/d6/d12/d15/d73/f7c [15665,5410] 0 2024-09-21T04:15:19.905 INFO:tasks.workunit.client.0.smithi110.stdout:5/613: creat d1/d22/d6e/fcf x:0 0 0 2024-09-21T04:15:19.906 INFO:tasks.workunit.client.0.smithi110.stdout:5/614: write d1/d22/f27 [261440,89274] 0 2024-09-21T04:15:19.906 INFO:tasks.workunit.client.0.smithi110.stdout:5/615: stat d1/d49/d4d/d1e/d5a/c99 0 2024-09-21T04:15:19.907 INFO:tasks.workunit.client.0.smithi110.stdout:7/562: symlink dd/d1c/lb0 0 2024-09-21T04:15:19.908 INFO:tasks.workunit.client.0.smithi110.stdout:6/603: dwrite d4/d86/d8f/f9b [0,4194304] 0 2024-09-21T04:15:19.908 INFO:tasks.workunit.client.0.smithi110.stdout:6/604: chown d4/d12/d17/d9a/dba 0 1 2024-09-21T04:15:19.909 INFO:tasks.workunit.client.0.smithi110.stdout:2/393: symlink d2/d50/l76 0 2024-09-21T04:15:19.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:19 smithi110 ceph-mon[27723]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:19.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:19 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2568174832' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:15:19.921 INFO:tasks.workunit.client.1.smithi148.stdout:4/632: rename d4/d12/f5b to d4/d66/d7f/d2a/fbc 0 2024-09-21T04:15:19.921 INFO:tasks.workunit.client.1.smithi148.stdout:4/633: write d4/d66/d7f/d2a/fbc [391852,97653] 0 2024-09-21T04:15:19.923 INFO:tasks.workunit.client.1.smithi148.stdout:2/689: creat d0/d9/d16/fd1 x:0 0 0 2024-09-21T04:15:19.924 INFO:tasks.workunit.client.0.smithi110.stdout:6/605: mkdir d4/d12/d17/d81/dbf 0 2024-09-21T04:15:19.925 INFO:tasks.workunit.client.1.smithi148.stdout:6/582: dwrite d0/d34/d38/d43/f98 [0,4194304] 0 2024-09-21T04:15:19.925 INFO:tasks.workunit.client.1.smithi148.stdout:6/583: write d0/d8/d18/f60 [586204,68392] 0 2024-09-21T04:15:19.925 INFO:tasks.workunit.client.1.smithi148.stdout:6/584: dread - d0/d1a/d23/d27/d62/d9a/d92/fa2 zero size 2024-09-21T04:15:19.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/606: mkdir d5/d60/de1 0 2024-09-21T04:15:19.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/607: fdatasync d5/d1f/d81/f8c 0 2024-09-21T04:15:19.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/608: read - d5/d2b/d32/d67/d80/fa1 zero size 2024-09-21T04:15:19.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/609: fsync d5/d1f/f4b 0 2024-09-21T04:15:19.927 INFO:tasks.workunit.client.0.smithi110.stdout:2/394: unlink d2/d1a/f6c 0 2024-09-21T04:15:19.929 INFO:tasks.workunit.client.1.smithi148.stdout:3/521: mkdir d3/d5/d6/d2b/da2 0 2024-09-21T04:15:19.929 INFO:tasks.workunit.client.1.smithi148.stdout:3/522: dread - d3/d1f/d4d/d82/d21/f4e zero size 2024-09-21T04:15:19.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:19 smithi148 ceph-mon[34957]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:19.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:19 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2568174832' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:15:19.943 INFO:tasks.workunit.client.1.smithi148.stdout:3/523: mkdir d3/d5/d6/d2b/da2/da3 0 2024-09-21T04:15:19.943 INFO:tasks.workunit.client.1.smithi148.stdout:3/524: readlink d3/d5/d6/d12/l1c 0 2024-09-21T04:15:19.944 INFO:tasks.workunit.client.0.smithi110.stdout:7/563: getdents dd/d14/d17/d8a 0 2024-09-21T04:15:19.944 INFO:tasks.workunit.client.0.smithi110.stdout:7/564: write dd/d1c/d39/d56/f86 [295847,103602] 0 2024-09-21T04:15:19.944 INFO:tasks.workunit.client.0.smithi110.stdout:7/565: chown dd/d14/f2a 128235484 1 2024-09-21T04:15:19.947 INFO:tasks.workunit.client.0.smithi110.stdout:6/606: rename d4/d78/daa/l19 to d4/d86/lc0 0 2024-09-21T04:15:19.949 INFO:tasks.workunit.client.0.smithi110.stdout:2/395: mknod d2/d50/c77 0 2024-09-21T04:15:19.949 INFO:tasks.workunit.client.0.smithi110.stdout:2/396: chown d2/d1a/d1f/d29/c3e 90189258 1 2024-09-21T04:15:19.950 INFO:tasks.workunit.client.1.smithi148.stdout:3/525: symlink d3/d5/d6/la4 0 2024-09-21T04:15:19.950 INFO:tasks.workunit.client.1.smithi148.stdout:4/634: rename d4/d12/d16/d26 to d4/d12/d16/d29/dbd 0 2024-09-21T04:15:19.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/526: link d3/d5/d6/c9c d3/d5/ca5 0 2024-09-21T04:15:19.954 INFO:tasks.workunit.client.0.smithi110.stdout:2/397: creat d2/d1a/d1f/d29/d19/d62/f78 x:0 0 0 2024-09-21T04:15:19.955 INFO:tasks.workunit.client.1.smithi148.stdout:3/527: rename d3/d5/d6/d12/d43/l5a to d3/d5/d6/d2b/da2/la6 0 2024-09-21T04:15:19.955 INFO:tasks.workunit.client.1.smithi148.stdout:0/582: sync 2024-09-21T04:15:19.955 INFO:tasks.workunit.client.1.smithi148.stdout:5/553: sync 2024-09-21T04:15:19.955 INFO:tasks.workunit.client.0.smithi110.stdout:2/398: link d2/d1a/d1f/d29/l6d d2/d1a/d1f/d5f/l79 0 2024-09-21T04:15:19.957 INFO:tasks.workunit.client.0.smithi110.stdout:2/399: dread d2/d50/d56/f53 [0,4194304] 0 2024-09-21T04:15:19.958 INFO:tasks.workunit.client.1.smithi148.stdout:0/583: link d5/d6/d7/d18/c30 d5/d6/d2a/d45/cb9 0 2024-09-21T04:15:19.959 INFO:tasks.workunit.client.0.smithi110.stdout:2/400: creat d2/d50/d56/f7a x:0 0 0 2024-09-21T04:15:19.961 INFO:tasks.workunit.client.1.smithi148.stdout:0/584: unlink d5/d6/f88 0 2024-09-21T04:15:19.961 INFO:tasks.workunit.client.0.smithi110.stdout:2/401: unlink d2/d1a/f20 0 2024-09-21T04:15:19.963 INFO:tasks.workunit.client.0.smithi110.stdout:6/607: dread d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:19.963 INFO:tasks.workunit.client.0.smithi110.stdout:6/608: creat d4/d12/d17/d33/d76/fc1 x:0 0 0 2024-09-21T04:15:19.963 INFO:tasks.workunit.client.0.smithi110.stdout:6/609: truncate d4/d12/d16/d66/fac 106616 0 2024-09-21T04:15:19.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/528: dread d3/d1f/d4d/d82/d23/f20 [0,4194304] 0 2024-09-21T04:15:19.967 INFO:tasks.workunit.client.1.smithi148.stdout:3/529: unlink d3/d5/d6/d12/d15/c3c 0 2024-09-21T04:15:19.969 INFO:tasks.workunit.client.1.smithi148.stdout:3/530: rmdir d3/d1f/d4d/d8b 0 2024-09-21T04:15:19.969 INFO:tasks.workunit.client.1.smithi148.stdout:3/531: fdatasync d3/d1f/d4d/d82/d21/f4e 0 2024-09-21T04:15:19.969 INFO:tasks.workunit.client.1.smithi148.stdout:4/635: write d4/d12/d16/d29/dbd/f70 [948952,130396] 0 2024-09-21T04:15:19.969 INFO:tasks.workunit.client.1.smithi148.stdout:4/636: creat d4/d66/fbe x:0 0 0 2024-09-21T04:15:19.972 INFO:tasks.workunit.client.1.smithi148.stdout:8/576: sync 2024-09-21T04:15:19.973 INFO:tasks.workunit.client.1.smithi148.stdout:8/577: mkdir dc/d13/d3c/d81/db0 0 2024-09-21T04:15:19.974 INFO:tasks.workunit.client.1.smithi148.stdout:8/578: mkdir dc/d13/d76/d50/db1 0 2024-09-21T04:15:19.974 INFO:tasks.workunit.client.1.smithi148.stdout:8/579: stat dc/d13/d76/d5d 0 2024-09-21T04:15:19.975 INFO:tasks.workunit.client.1.smithi148.stdout:8/580: chown dc/d13/d76/f57 327443435 1 2024-09-21T04:15:19.976 INFO:tasks.workunit.client.1.smithi148.stdout:8/581: unlink dc/d13/ca9 0 2024-09-21T04:15:19.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/554: dread d3/d14/d1f/d26/d2e/d3d/f89 [0,4194304] 0 2024-09-21T04:15:19.977 INFO:tasks.workunit.client.1.smithi148.stdout:7/560: sync 2024-09-21T04:15:19.982 INFO:tasks.workunit.client.0.smithi110.stdout:6/610: write d4/d12/d16/f7e [5358945,125186] 0 2024-09-21T04:15:19.982 INFO:tasks.workunit.client.0.smithi110.stdout:6/611: write d4/d12/f2b [1363800,3862] 0 2024-09-21T04:15:19.982 INFO:tasks.workunit.client.0.smithi110.stdout:6/612: stat d4/d78/daa/lb5 0 2024-09-21T04:15:19.984 INFO:tasks.workunit.client.0.smithi110.stdout:6/613: creat d4/d12/d17/d2e/d40/fc2 x:0 0 0 2024-09-21T04:15:19.984 INFO:tasks.workunit.client.0.smithi110.stdout:6/614: write d4/d12/d16/f59 [1829653,98092] 0 2024-09-21T04:15:19.984 INFO:tasks.workunit.client.1.smithi148.stdout:5/555: dread d3/f1b [0,4194304] 0 2024-09-21T04:15:19.984 INFO:tasks.workunit.client.1.smithi148.stdout:3/532: dread d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:19.985 INFO:tasks.workunit.client.1.smithi148.stdout:3/533: truncate d3/d1f/d4d/d82/d74/f91 785039 0 2024-09-21T04:15:19.988 INFO:tasks.workunit.client.1.smithi148.stdout:3/534: mkdir d3/d5/d6/d12/d15/da7 0 2024-09-21T04:15:19.988 INFO:tasks.workunit.client.1.smithi148.stdout:3/535: read d3/f49 [219138,13089] 0 2024-09-21T04:15:19.989 INFO:tasks.workunit.client.0.smithi110.stdout:6/615: rename d4/d12/d16/d66/d7a/d63 to d4/d12/d17/d81/d90/dc3 0 2024-09-21T04:15:19.989 INFO:tasks.workunit.client.0.smithi110.stdout:6/616: dread - d4/d12/d17/d33/d76/fc1 zero size 2024-09-21T04:15:19.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/561: write d5/d8/d6f/d7d/d90/d28/f29 [1097932,55142] 0 2024-09-21T04:15:20.009 INFO:tasks.workunit.client.1.smithi148.stdout:7/562: dread d5/d8/d6f/d7d/d90/f19 [0,4194304] 0 2024-09-21T04:15:20.018 INFO:tasks.workunit.client.1.smithi148.stdout:6/585: dwrite d0/d34/da0/fa9 [0,4194304] 0 2024-09-21T04:15:20.021 INFO:tasks.workunit.client.0.smithi110.stdout:6/617: link d4/d12/d17/d33/l43 d4/d12/d16/d66/d7a/d54/lc4 0 2024-09-21T04:15:20.022 INFO:tasks.workunit.client.1.smithi148.stdout:3/536: truncate d3/d1f/d4d/d82/f11 43061 0 2024-09-21T04:15:20.037 INFO:tasks.workunit.client.1.smithi148.stdout:9/610: dwrite d5/d1f/d81/f8c [0,4194304] 0 2024-09-21T04:15:20.047 INFO:tasks.workunit.client.0.smithi110.stdout:8/593: sync 2024-09-21T04:15:20.048 INFO:tasks.workunit.client.0.smithi110.stdout:8/594: write d0/d7/f20 [1272960,95091] 0 2024-09-21T04:15:20.056 INFO:tasks.workunit.client.1.smithi148.stdout:5/556: dwrite d3/d14/d1f/f23 [0,4194304] 0 2024-09-21T04:15:20.056 INFO:tasks.workunit.client.1.smithi148.stdout:5/557: truncate d3/d14/d1f/d26/d59/fb2 891298 0 2024-09-21T04:15:20.057 INFO:tasks.workunit.client.0.smithi110.stdout:2/402: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:20.057 INFO:tasks.workunit.client.1.smithi148.stdout:0/585: dwrite d5/d6/d7/d11/d48/f80 [0,4194304] 0 2024-09-21T04:15:20.057 INFO:tasks.workunit.client.1.smithi148.stdout:0/586: chown d5/d6/c76 46685 1 2024-09-21T04:15:20.057 INFO:tasks.workunit.client.1.smithi148.stdout:4/637: dwrite d4/d12/d75/f7a [0,4194304] 0 2024-09-21T04:15:20.061 INFO:tasks.workunit.client.1.smithi148.stdout:7/563: rmdir d5/d8/d6f/d7d/d90 39 2024-09-21T04:15:20.063 INFO:tasks.workunit.client.0.smithi110.stdout:6/618: dwrite d4/d12/d16/d66/d7a/d4b/f5e [0,4194304] 0 2024-09-21T04:15:20.063 INFO:tasks.workunit.client.0.smithi110.stdout:6/619: write d4/d12/d17/f8e [1592349,104483] 0 2024-09-21T04:15:20.063 INFO:tasks.workunit.client.0.smithi110.stdout:6/620: write d4/d12/d17/d2e/d40/fc2 [701741,114219] 0 2024-09-21T04:15:20.063 INFO:tasks.workunit.client.0.smithi110.stdout:6/621: truncate d4/d12/d16/f50 2046804 0 2024-09-21T04:15:20.073 INFO:tasks.workunit.client.0.smithi110.stdout:2/403: mkdir d2/d50/d7b 0 2024-09-21T04:15:20.074 INFO:tasks.workunit.client.1.smithi148.stdout:0/587: dwrite d5/d6/d7/f83 [0,4194304] 0 2024-09-21T04:15:20.074 INFO:tasks.workunit.client.1.smithi148.stdout:0/588: creat d5/d6/d7/d11/d7b/fba x:0 0 0 2024-09-21T04:15:20.081 INFO:tasks.workunit.client.0.smithi110.stdout:2/404: write d2/f44 [2149778,95972] 0 2024-09-21T04:15:20.084 INFO:tasks.workunit.client.0.smithi110.stdout:6/622: mkdir d4/d12/d17/d81/dc5 0 2024-09-21T04:15:20.084 INFO:tasks.workunit.client.0.smithi110.stdout:6/623: truncate d4/d12/d16/d66/fac 323189 0 2024-09-21T04:15:20.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/586: creat d0/d1a/d23/d32/fad x:0 0 0 2024-09-21T04:15:20.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/587: stat d0/d34/d38/d43 0 2024-09-21T04:15:20.093 INFO:tasks.workunit.client.0.smithi110.stdout:6/624: read d4/d12/d16/f7e [3635636,96072] 0 2024-09-21T04:15:20.095 INFO:tasks.workunit.client.1.smithi148.stdout:9/611: rename d5/l13 to d5/le2 0 2024-09-21T04:15:20.095 INFO:tasks.workunit.client.1.smithi148.stdout:9/612: dread - d5/d1e/d37/fca zero size 2024-09-21T04:15:20.099 INFO:tasks.workunit.client.1.smithi148.stdout:5/558: readlink d3/l30 0 2024-09-21T04:15:20.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/613: dread d5/d1e/d37/f64 [0,4194304] 0 2024-09-21T04:15:20.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/614: write d5/d1e/d37/fca [91369,46983] 0 2024-09-21T04:15:20.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/615: write d5/f1b [1946193,102809] 0 2024-09-21T04:15:20.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/616: dread - d5/d1e/faa zero size 2024-09-21T04:15:20.110 INFO:tasks.workunit.client.1.smithi148.stdout:4/638: rmdir d4/d12/d78/daa 0 2024-09-21T04:15:20.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/559: dwrite d3/d14/d1f/d26/d2e/d56/f8a [0,4194304] 0 2024-09-21T04:15:20.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/560: chown d3/d14/d1f/d26/d2e/d3d/f89 25007 1 2024-09-21T04:15:20.118 INFO:tasks.workunit.client.1.smithi148.stdout:5/561: readlink d3/l6b 0 2024-09-21T04:15:20.119 INFO:tasks.workunit.client.1.smithi148.stdout:6/588: truncate d0/d34/d38/d43/f98 3465832 0 2024-09-21T04:15:20.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/537: getdents d3/d1f/d4d/d82/d23/d28/d65 0 2024-09-21T04:15:20.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/538: fdatasync d3/d1f/d4d/d82/d52/f59 0 2024-09-21T04:15:20.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/539: fdatasync d3/d5/d6/d12/f72 0 2024-09-21T04:15:20.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/540: read d3/d1f/d4d/d82/f1d [1750677,77220] 0 2024-09-21T04:15:20.124 INFO:tasks.workunit.client.1.smithi148.stdout:0/589: creat d5/d6/d2a/d2b/d99/db7/fbb x:0 0 0 2024-09-21T04:15:20.124 INFO:tasks.workunit.client.1.smithi148.stdout:0/590: dread - d5/d6/d7/d22/f8a zero size 2024-09-21T04:15:20.127 INFO:tasks.workunit.client.1.smithi148.stdout:5/562: fsync d3/f36 0 2024-09-21T04:15:20.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/591: dread d5/d82/f87 [0,4194304] 0 2024-09-21T04:15:20.130 INFO:tasks.workunit.client.1.smithi148.stdout:0/592: link d5/d6/d7/d22/f46 d5/d6/d7/d11/fbc 0 2024-09-21T04:15:20.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/563: creat d3/d14/d1f/d72/fc5 x:0 0 0 2024-09-21T04:15:20.133 INFO:tasks.workunit.client.1.smithi148.stdout:0/593: link d5/d6/d7/l33 d5/d6/d2a/d2b/d99/lbd 0 2024-09-21T04:15:20.134 INFO:tasks.workunit.client.1.smithi148.stdout:0/594: mknod d5/d6/d2a/d45/cbe 0 2024-09-21T04:15:20.136 INFO:tasks.workunit.client.1.smithi148.stdout:0/595: creat d5/d6/d2a/d2b/d84/da5/fbf x:0 0 0 2024-09-21T04:15:20.137 INFO:tasks.workunit.client.1.smithi148.stdout:0/596: creat d5/d6/d7/d11/d48/fc0 x:0 0 0 2024-09-21T04:15:20.139 INFO:tasks.workunit.client.1.smithi148.stdout:0/597: rename d5/d6/d2a/d45/f98 to d5/d6/d2a/d2b/d84/daf/fc1 0 2024-09-21T04:15:20.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/598: rename d5/d6/d2a/d3f/f41 to d5/d6/d2a/d3f/d8b/fc2 0 2024-09-21T04:15:20.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/599: rename d5/f52 to d5/d6/d7/d18/fc3 0 2024-09-21T04:15:20.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/600: read d5/d6/f15 [1752444,85535] 0 2024-09-21T04:15:20.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/601: read d5/d6/d2a/d45/f6e [97843,56704] 0 2024-09-21T04:15:20.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/602: readlink d5/d6/d7/d18/l6a 0 2024-09-21T04:15:20.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/603: chown d5/d6/d2a/c7e 110 1 2024-09-21T04:15:20.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/564: dread d3/d37/f7f [0,4194304] 0 2024-09-21T04:15:20.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/565: chown d3/d14/d1f/db1/db8/dba 11311 1 2024-09-21T04:15:20.148 INFO:tasks.workunit.client.1.smithi148.stdout:0/604: creat d5/d82/fc4 x:0 0 0 2024-09-21T04:15:20.149 INFO:tasks.workunit.client.1.smithi148.stdout:5/566: truncate d3/d37/f5f 229954 0 2024-09-21T04:15:20.150 INFO:tasks.workunit.client.1.smithi148.stdout:0/605: creat d5/d6/d7/d11/fc5 x:0 0 0 2024-09-21T04:15:20.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/606: unlink d5/d6/d7/cb0 0 2024-09-21T04:15:20.154 INFO:tasks.workunit.client.1.smithi148.stdout:0/607: link d5/d6/d2a/d2b/d84/da5/f27 d5/d6/d2a/d45/d70/fc6 0 2024-09-21T04:15:20.154 INFO:tasks.workunit.client.1.smithi148.stdout:0/608: creat d5/d6/d2a/d2b/d99/db7/fc7 x:0 0 0 2024-09-21T04:15:20.155 INFO:tasks.workunit.client.1.smithi148.stdout:6/589: dwrite d0/d34/d97/f30 [0,4194304] 0 2024-09-21T04:15:20.155 INFO:tasks.workunit.client.1.smithi148.stdout:3/541: dwrite d3/d5/d6/d12/d15/d73/f7c [0,4194304] 0 2024-09-21T04:15:20.156 INFO:tasks.workunit.client.1.smithi148.stdout:0/609: mkdir d5/d6/d2a/d2b/d84/daf/dc8 0 2024-09-21T04:15:20.161 INFO:tasks.workunit.client.1.smithi148.stdout:6/590: creat d0/d1a/d23/d32/d93/d96/fae x:0 0 0 2024-09-21T04:15:20.161 INFO:tasks.workunit.client.1.smithi148.stdout:0/610: dread - d5/d6/d2a/d45/f9b zero size 2024-09-21T04:15:20.164 INFO:tasks.workunit.client.1.smithi148.stdout:0/611: creat d5/d6/d2a/d3f/d8b/dab/fc9 x:0 0 0 2024-09-21T04:15:20.165 INFO:tasks.workunit.client.1.smithi148.stdout:6/591: mknod d0/d34/d38/d43/caf 0 2024-09-21T04:15:20.167 INFO:tasks.workunit.client.1.smithi148.stdout:6/592: mkdir d0/d1a/d23/d27/d80/db0 0 2024-09-21T04:15:20.172 INFO:tasks.workunit.client.1.smithi148.stdout:6/593: dread d0/d34/d38/d43/f78 [0,4194304] 0 2024-09-21T04:15:20.174 INFO:tasks.workunit.client.1.smithi148.stdout:0/612: dread d5/d6/d2a/d2b/d84/da5/f64 [0,4194304] 0 2024-09-21T04:15:20.174 INFO:tasks.workunit.client.1.smithi148.stdout:0/613: dread - d5/d6/d2a/d3f/d6d/da3/fae zero size 2024-09-21T04:15:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:0/614: chown d5/d6/d2a/d45/cb9 182363566 1 2024-09-21T04:15:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:0/615: chown d5/d6/d2a/d3f/d6d/l72 2567140 1 2024-09-21T04:15:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:6/594: mknod d0/d1a/d23/cb1 0 2024-09-21T04:15:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:6/595: fsync d0/d8/f24 0 2024-09-21T04:15:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:6/596: readlink d0/d1a/d23/l31 0 2024-09-21T04:15:20.176 INFO:tasks.workunit.client.1.smithi148.stdout:0/616: getdents d5/d6/d2a/d2b/d84 0 2024-09-21T04:15:20.178 INFO:tasks.workunit.client.0.smithi110.stdout:2/405: symlink d2/d1a/d1f/d29/d19/l7c 0 2024-09-21T04:15:20.185 INFO:tasks.workunit.client.1.smithi148.stdout:0/617: dread d5/d6/d2a/d45/f50 [0,4194304] 0 2024-09-21T04:15:20.185 INFO:tasks.workunit.client.1.smithi148.stdout:0/618: fdatasync f4 0 2024-09-21T04:15:20.186 INFO:tasks.workunit.client.1.smithi148.stdout:6/597: read d0/d34/d97/f1f [2728964,698] 0 2024-09-21T04:15:20.186 INFO:tasks.workunit.client.1.smithi148.stdout:6/598: chown d0/d1a/d23/d32/d93/fa8 4323 1 2024-09-21T04:15:20.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/406: read d2/d1a/d1f/d5f/f1d [3804725,62086] 0 2024-09-21T04:15:20.198 INFO:tasks.workunit.client.1.smithi148.stdout:6/599: dread d0/d34/d38/d43/f5e [0,4194304] 0 2024-09-21T04:15:20.199 INFO:tasks.workunit.client.1.smithi148.stdout:6/600: link d0/d34/d97/f39 d0/d1a/d23/d27/d80/fb2 0 2024-09-21T04:15:20.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/601: stat d0/d34/d38/c6a 0 2024-09-21T04:15:20.201 INFO:tasks.workunit.client.1.smithi148.stdout:6/602: chown d0/d8/c9 7002764 1 2024-09-21T04:15:20.201 INFO:tasks.workunit.client.1.smithi148.stdout:2/690: sync 2024-09-21T04:15:20.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/567: sync 2024-09-21T04:15:20.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/568: readlink db/dd/l3f 0 2024-09-21T04:15:20.202 INFO:tasks.workunit.client.1.smithi148.stdout:5/567: dwrite d3/d14/d1f/d26/d2e/d56/f54 [0,4194304] 0 2024-09-21T04:15:20.202 INFO:tasks.workunit.client.1.smithi148.stdout:6/603: mknod d0/d34/d97/cb3 0 2024-09-21T04:15:20.202 INFO:tasks.workunit.client.0.smithi110.stdout:2/407: dread d2/d1a/d1f/f38 [0,4194304] 0 2024-09-21T04:15:20.203 INFO:tasks.workunit.client.1.smithi148.stdout:2/691: mknod d0/d2b/da3/cd2 0 2024-09-21T04:15:20.204 INFO:tasks.workunit.client.0.smithi110.stdout:2/408: readlink d2/d1a/d1f/d29/d19/l40 0 2024-09-21T04:15:20.204 INFO:tasks.workunit.client.0.smithi110.stdout:2/409: chown d2/d50/d56/c33 466 1 2024-09-21T04:15:20.205 INFO:tasks.workunit.client.1.smithi148.stdout:5/568: dread d3/d14/d1a/f57 [0,4194304] 0 2024-09-21T04:15:20.205 INFO:tasks.workunit.client.1.smithi148.stdout:5/569: readlink d3/d34/d3c/l40 0 2024-09-21T04:15:20.206 INFO:tasks.workunit.client.1.smithi148.stdout:2/692: mknod d0/d9/d16/d55/d6f/cd3 0 2024-09-21T04:15:20.212 INFO:tasks.workunit.client.1.smithi148.stdout:7/564: symlink d5/d8/d6f/lcb 0 2024-09-21T04:15:20.212 INFO:tasks.workunit.client.1.smithi148.stdout:7/565: stat d5/d14/d25/d40/d8b/ca8 0 2024-09-21T04:15:20.214 INFO:tasks.workunit.client.1.smithi148.stdout:3/542: dwrite d3/d1f/d4d/d82/d23/d28/f42 [0,4194304] 0 2024-09-21T04:15:20.214 INFO:tasks.workunit.client.1.smithi148.stdout:7/566: symlink d5/d14/d25/d40/dab/lcc 0 2024-09-21T04:15:20.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/543: mknod d3/d1f/d4d/d82/d23/d28/ca8 0 2024-09-21T04:15:20.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/544: chown d3/d1f/d4d/c57 874294 1 2024-09-21T04:15:20.216 INFO:tasks.workunit.client.1.smithi148.stdout:3/545: stat d3/d5/d6/d12/f72 0 2024-09-21T04:15:20.217 INFO:tasks.workunit.client.1.smithi148.stdout:7/567: mkdir d5/d8/d6f/dcd 0 2024-09-21T04:15:20.217 INFO:tasks.workunit.client.1.smithi148.stdout:5/570: creat d3/d14/fc6 x:0 0 0 2024-09-21T04:15:20.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/546: symlink d3/d84/la9 0 2024-09-21T04:15:20.220 INFO:tasks.workunit.client.1.smithi148.stdout:5/571: rename d3/d34/caf to d3/d34/cc7 0 2024-09-21T04:15:20.222 INFO:tasks.workunit.client.1.smithi148.stdout:7/568: creat d5/d8/d6f/d7d/d90/fce x:0 0 0 2024-09-21T04:15:20.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/619: dwrite d5/d6/d2a/d3f/f8d [0,4194304] 0 2024-09-21T04:15:20.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/620: creat d5/d6/d7/d11/fca x:0 0 0 2024-09-21T04:15:20.224 INFO:tasks.workunit.client.1.smithi148.stdout:7/569: link d5/d8/d6f/d7d/l5f d5/d14/d25/d40/dab/dad/lcf 0 2024-09-21T04:15:20.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/410: dread d2/f44 [0,4194304] 0 2024-09-21T04:15:20.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/411: creat d2/d1a/d1f/d29/f7d x:0 0 0 2024-09-21T04:15:20.229 INFO:tasks.workunit.client.1.smithi148.stdout:0/621: mkdir d5/dcb 0 2024-09-21T04:15:20.231 INFO:tasks.workunit.client.1.smithi148.stdout:0/622: truncate d5/d6/f62 2205821 0 2024-09-21T04:15:20.231 INFO:tasks.workunit.client.1.smithi148.stdout:0/623: write d5/d6/d2a/d2b/d84/da5/fbf [163552,83397] 0 2024-09-21T04:15:20.235 INFO:tasks.workunit.client.1.smithi148.stdout:6/604: symlink d0/d34/d38/lb4 0 2024-09-21T04:15:20.235 INFO:tasks.workunit.client.1.smithi148.stdout:6/605: chown d0/d1a/d23/d32/d93/d96 230538436 1 2024-09-21T04:15:20.235 INFO:tasks.workunit.client.1.smithi148.stdout:6/606: write d0/d8/d18/f4d [1893246,38387] 0 2024-09-21T04:15:20.237 INFO:tasks.workunit.client.1.smithi148.stdout:6/607: symlink d0/d1a/lb5 0 2024-09-21T04:15:20.239 INFO:tasks.workunit.client.1.smithi148.stdout:4/639: unlink d4/d12/d16/d29/l77 0 2024-09-21T04:15:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:4/640: mknod d4/d66/d7f/d2a/da9/cbf 0 2024-09-21T04:15:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:4/641: fsync d4/d12/d16/d29/d61/d6c/d9d/fb0 0 2024-09-21T04:15:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:4/642: read - d4/d12/d16/d29/d61/d6c/d9d/fb0 zero size 2024-09-21T04:15:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:4/643: readlink d4/d12/l21 0 2024-09-21T04:15:20.242 INFO:tasks.workunit.client.0.smithi110.stdout:2/412: unlink d2/d1a/d1f/l10 0 2024-09-21T04:15:20.243 INFO:tasks.workunit.client.0.smithi110.stdout:2/413: readlink d2/d1a/d1f/d5f/l59 0 2024-09-21T04:15:20.243 INFO:tasks.workunit.client.1.smithi148.stdout:4/644: truncate d4/d66/d7f/f4a 3918888 0 2024-09-21T04:15:20.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/625: symlink d4/d12/d17/d33/lc6 0 2024-09-21T04:15:20.245 INFO:tasks.workunit.client.1.smithi148.stdout:4/645: symlink d4/d12/da7/lc0 0 2024-09-21T04:15:20.245 INFO:tasks.workunit.client.1.smithi148.stdout:4/646: fsync d4/d12/f63 0 2024-09-21T04:15:20.246 INFO:tasks.workunit.client.0.smithi110.stdout:6/626: truncate d4/d78/daa/f11 2885876 0 2024-09-21T04:15:20.247 INFO:tasks.workunit.client.1.smithi148.stdout:3/547: rename d3/d1f/c37 to d3/d5/d5c/caa 0 2024-09-21T04:15:20.247 INFO:tasks.workunit.client.1.smithi148.stdout:3/548: chown d3/d5/f1e 31024842 1 2024-09-21T04:15:20.248 INFO:tasks.workunit.client.1.smithi148.stdout:4/647: mkdir d4/d87/dc1 0 2024-09-21T04:15:20.249 INFO:tasks.workunit.client.0.smithi110.stdout:6/627: creat d4/d86/d8f/fc7 x:0 0 0 2024-09-21T04:15:20.250 INFO:tasks.workunit.client.1.smithi148.stdout:2/693: dwrite d0/f14 [0,4194304] 0 2024-09-21T04:15:20.251 INFO:tasks.workunit.client.1.smithi148.stdout:6/608: rename d0/d8/d18/c6d to d0/d1a/d23/dc/d68/cb6 0 2024-09-21T04:15:20.251 INFO:tasks.workunit.client.1.smithi148.stdout:6/609: creat d0/d8/d18/fb7 x:0 0 0 2024-09-21T04:15:20.252 INFO:tasks.workunit.client.1.smithi148.stdout:3/549: creat d3/d1f/d4d/d82/d52/fab x:0 0 0 2024-09-21T04:15:20.253 INFO:tasks.workunit.client.0.smithi110.stdout:6/628: mkdir d4/d12/d17/d33/d3e/dc8 0 2024-09-21T04:15:20.254 INFO:tasks.workunit.client.1.smithi148.stdout:4/648: mkdir d4/d66/d7f/d9a/dc2 0 2024-09-21T04:15:20.255 INFO:tasks.workunit.client.1.smithi148.stdout:2/694: mknod d0/d9/da2/cd4 0 2024-09-21T04:15:20.256 INFO:tasks.workunit.client.0.smithi110.stdout:6/629: link d4/d86/d8f/fc7 d4/d12/d17/d2e/d40/fc9 0 2024-09-21T04:15:20.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/630: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca 0 2024-09-21T04:15:20.260 INFO:tasks.workunit.client.1.smithi148.stdout:2/695: symlink d0/d9/d16/d55/d71/dc2/ld5 0 2024-09-21T04:15:20.263 INFO:tasks.workunit.client.1.smithi148.stdout:6/610: rmdir d0/d34/d38/d57 39 2024-09-21T04:15:20.263 INFO:tasks.workunit.client.0.smithi110.stdout:6/631: rmdir d4/d12/d17/d2e/d38 39 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/611: mkdir d0/d1a/db8 0 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/696: dread d0/d9/d59/f65 [0,4194304] 0 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/612: chown d0/d34/d97/f52 0 1 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/697: readlink d0/d9/d16/d55/d71/l7a 0 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:8/582: sync 2024-09-21T04:15:20.265 INFO:tasks.workunit.client.0.smithi110.stdout:6/632: symlink d4/d12/d17/d33/dab/lcb 0 2024-09-21T04:15:20.266 INFO:tasks.workunit.client.1.smithi148.stdout:6/613: symlink d0/d1a/d23/dc/lb9 0 2024-09-21T04:15:20.268 INFO:tasks.workunit.client.1.smithi148.stdout:2/698: creat d0/d9/d16/d55/d71/dc2/dcb/fd6 x:0 0 0 2024-09-21T04:15:20.269 INFO:tasks.workunit.client.1.smithi148.stdout:2/699: write d0/d2f/f84 [1532612,63408] 0 2024-09-21T04:15:20.269 INFO:tasks.workunit.client.1.smithi148.stdout:5/572: dwrite d3/d24/fb0 [0,4194304] 0 2024-09-21T04:15:20.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/614: mkdir d0/d8/dba 0 2024-09-21T04:15:20.271 INFO:tasks.workunit.client.1.smithi148.stdout:2/700: unlink d0/d2f/l3e 0 2024-09-21T04:15:20.275 INFO:tasks.workunit.client.1.smithi148.stdout:5/573: creat d3/d34/d78/dbf/d6f/fc8 x:0 0 0 2024-09-21T04:15:20.275 INFO:tasks.workunit.client.1.smithi148.stdout:2/701: mknod d0/d9/dc/d13/d50/cd7 0 2024-09-21T04:15:20.275 INFO:tasks.workunit.client.1.smithi148.stdout:2/702: chown d0/d9/dc/d13/f1a 57364578 1 2024-09-21T04:15:20.284 INFO:tasks.workunit.client.0.smithi110.stdout:2/414: dwrite d2/d1a/d1f/d29/f4c [0,4194304] 0 2024-09-21T04:15:20.286 INFO:tasks.workunit.client.1.smithi148.stdout:9/617: sync 2024-09-21T04:15:20.289 INFO:tasks.workunit.client.1.smithi148.stdout:9/618: mkdir d5/de3 0 2024-09-21T04:15:20.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/619: write d5/d1e/d37/fca [229569,117031] 0 2024-09-21T04:15:20.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/620: write d5/d60/d28/d73/fc3 [182899,5013] 0 2024-09-21T04:15:20.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/621: mkdir d5/d2b/d4e/de4 0 2024-09-21T04:15:20.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/622: chown d5/d1f/d81/f8c 102 1 2024-09-21T04:15:20.291 INFO:tasks.workunit.client.1.smithi148.stdout:9/623: mkdir d5/d60/d69/de5 0 2024-09-21T04:15:20.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/570: dwrite d5/d8/d6f/d7d/fac [0,4194304] 0 2024-09-21T04:15:20.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/571: truncate d5/d8/d6f/d7d/d90/d28/f9c 963849 0 2024-09-21T04:15:20.307 INFO:tasks.workunit.client.1.smithi148.stdout:0/624: dwrite d5/d6/d2a/d2b/d84/f89 [0,4194304] 0 2024-09-21T04:15:20.307 INFO:tasks.workunit.client.1.smithi148.stdout:0/625: dread - d5/d6/d2a/d2b/d84/da5/f95 zero size 2024-09-21T04:15:20.308 INFO:tasks.workunit.client.1.smithi148.stdout:9/624: dwrite d5/fd [0,4194304] 0 2024-09-21T04:15:20.308 INFO:tasks.workunit.client.1.smithi148.stdout:4/649: dwrite d4/d12/d16/d29/dbd/f3e [0,4194304] 0 2024-09-21T04:15:20.311 INFO:tasks.workunit.client.1.smithi148.stdout:9/625: creat d5/d1f/fe6 x:0 0 0 2024-09-21T04:15:20.312 INFO:tasks.workunit.client.1.smithi148.stdout:4/650: creat d4/d87/d2d/fc3 x:0 0 0 2024-09-21T04:15:20.314 INFO:tasks.workunit.client.1.smithi148.stdout:9/626: unlink d5/d39/f77 0 2024-09-21T04:15:20.315 INFO:tasks.workunit.client.1.smithi148.stdout:7/572: write d5/d14/f4b [950294,64229] 0 2024-09-21T04:15:20.315 INFO:tasks.workunit.client.1.smithi148.stdout:7/573: creat d5/daa/fd0 x:0 0 0 2024-09-21T04:15:20.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/574: truncate d5/d8/f1b 1441608 0 2024-09-21T04:15:20.318 INFO:tasks.workunit.client.1.smithi148.stdout:7/575: creat d5/d14/d25/d40/db9/fd1 x:0 0 0 2024-09-21T04:15:20.318 INFO:tasks.workunit.client.1.smithi148.stdout:7/576: chown d5/d14/d25 16290572 1 2024-09-21T04:15:20.325 INFO:tasks.workunit.client.1.smithi148.stdout:4/651: dread d4/d12/d16/d29/d61/d6c/d9d/fa2 [4194304,4194304] 0 2024-09-21T04:15:20.330 INFO:tasks.workunit.client.1.smithi148.stdout:9/627: write d5/d1f/f24 [7264057,52318] 0 2024-09-21T04:15:20.336 INFO:tasks.workunit.client.1.smithi148.stdout:7/577: write d5/d14/d25/f42 [1184130,58098] 0 2024-09-21T04:15:20.340 INFO:tasks.workunit.client.0.smithi110.stdout:6/633: dwrite d4/d12/d17/d2e/f47 [0,4194304] 0 2024-09-21T04:15:20.341 INFO:tasks.workunit.client.0.smithi110.stdout:6/634: truncate d4/d12/d17/d2e/fb9 6169 0 2024-09-21T04:15:20.342 INFO:tasks.workunit.client.1.smithi148.stdout:4/652: dread d4/d12/f41 [0,4194304] 0 2024-09-21T04:15:20.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/574: dwrite d3/d14/fc6 [0,4194304] 0 2024-09-21T04:15:20.342 INFO:tasks.workunit.client.0.smithi110.stdout:6/635: creat d4/d12/d17/d81/fcc x:0 0 0 2024-09-21T04:15:20.342 INFO:tasks.workunit.client.0.smithi110.stdout:6/636: creat d4/d78/daa/fcd x:0 0 0 2024-09-21T04:15:20.343 INFO:tasks.workunit.client.0.smithi110.stdout:6/637: fdatasync d4/d12/d17/d33/d76/fae 0 2024-09-21T04:15:20.345 INFO:tasks.workunit.client.1.smithi148.stdout:9/628: write d5/d2b/dbe/f9a [1869801,94103] 0 2024-09-21T04:15:20.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/626: truncate d5/d6/d2a/d2b/d84/f89 246498 0 2024-09-21T04:15:20.356 INFO:tasks.workunit.client.1.smithi148.stdout:9/629: write d5/d1e/f29 [2476713,82664] 0 2024-09-21T04:15:20.356 INFO:tasks.workunit.client.1.smithi148.stdout:9/630: write d5/d60/d28/f30 [4027903,5892] 0 2024-09-21T04:15:20.358 INFO:tasks.workunit.client.1.smithi148.stdout:5/575: mkdir d3/d14/d1f/d72/dc9 0 2024-09-21T04:15:20.358 INFO:tasks.workunit.client.1.smithi148.stdout:5/576: write d3/d14/d1a/f4f [4725299,34599] 0 2024-09-21T04:15:20.358 INFO:tasks.workunit.client.1.smithi148.stdout:5/577: write d3/d14/d1f/db1/db8/dba/fc1 [143348,75359] 0 2024-09-21T04:15:20.359 INFO:tasks.workunit.client.1.smithi148.stdout:0/627: chown d5/d6/d2a/d2b/la9 270391378 1 2024-09-21T04:15:20.362 INFO:tasks.workunit.client.1.smithi148.stdout:9/631: truncate d5/fb8 806833 0 2024-09-21T04:15:20.363 INFO:tasks.workunit.client.1.smithi148.stdout:9/632: readlink d5/d60/d28/l89 0 2024-09-21T04:15:20.364 INFO:tasks.workunit.client.1.smithi148.stdout:5/578: creat d3/d34/d3c/d5d/fca x:0 0 0 2024-09-21T04:15:20.365 INFO:tasks.workunit.client.1.smithi148.stdout:5/579: chown d3/d34/d78/c87 1 1 2024-09-21T04:15:20.367 INFO:tasks.workunit.client.1.smithi148.stdout:5/580: rename d3/d14/d1a/cb5 to d3/d14/d1f/d26/d2e/ccb 0 2024-09-21T04:15:20.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/581: creat d3/d14/d1f/d26/d2e/d3d/da0/fcc x:0 0 0 2024-09-21T04:15:20.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/582: truncate d3/d34/d78/dbf/f96 1824719 0 2024-09-21T04:15:20.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/583: rename d3/d14/d1a/f57 to d3/d34/d3c/d5d/fcd 0 2024-09-21T04:15:20.370 INFO:tasks.workunit.client.1.smithi148.stdout:1/569: sync 2024-09-21T04:15:20.370 INFO:tasks.workunit.client.0.smithi110.stdout:6/638: dwrite d4/d12/f1d [4194304,4194304] 0 2024-09-21T04:15:20.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/578: dwrite d5/d14/d25/f42 [0,4194304] 0 2024-09-21T04:15:20.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/579: chown d5/d14/d25/d40/dab/dad/db8 88105 1 2024-09-21T04:15:20.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/580: dread - d5/d8/d6f/d7d/d90/d28/f70 zero size 2024-09-21T04:15:20.372 INFO:tasks.workunit.client.0.smithi110.stdout:6/639: unlink d4/d12/d16/f5b 0 2024-09-21T04:15:20.374 INFO:tasks.workunit.client.1.smithi148.stdout:7/581: mknod d5/d14/cd2 0 2024-09-21T04:15:20.376 INFO:tasks.workunit.client.0.smithi110.stdout:6/640: symlink d4/d12/d17/d2e/d38/lce 0 2024-09-21T04:15:20.377 INFO:tasks.workunit.client.1.smithi148.stdout:7/582: rmdir d5/d8/d6f/d7d/d8a 39 2024-09-21T04:15:20.377 INFO:tasks.workunit.client.1.smithi148.stdout:7/583: chown d5/f3d 58041 1 2024-09-21T04:15:20.378 INFO:tasks.workunit.client.1.smithi148.stdout:7/584: truncate d5/d14/d25/f42 5064762 0 2024-09-21T04:15:20.378 INFO:tasks.workunit.client.0.smithi110.stdout:6/641: truncate d4/d12/d17/d2e/d38/f4f 515605 0 2024-09-21T04:15:20.380 INFO:tasks.workunit.client.1.smithi148.stdout:7/585: link d5/d8/d6f/d7d/d90/l59 d5/d8/d6f/d7d/d8a/ld3 0 2024-09-21T04:15:20.382 INFO:tasks.workunit.client.1.smithi148.stdout:7/586: creat d5/d14/d25/d40/dab/fd4 x:0 0 0 2024-09-21T04:15:20.383 INFO:tasks.workunit.client.1.smithi148.stdout:7/587: creat d5/d14/d25/d40/d7b/fd5 x:0 0 0 2024-09-21T04:15:20.385 INFO:tasks.workunit.client.1.smithi148.stdout:7/588: mknod d5/d8/d6f/cd6 0 2024-09-21T04:15:20.386 INFO:tasks.workunit.client.1.smithi148.stdout:9/633: dread d5/d60/d28/d73/fdf [0,4194304] 0 2024-09-21T04:15:20.387 INFO:tasks.workunit.client.1.smithi148.stdout:7/589: rename d5/d8/f16 to d5/d8/d6f/dcd/fd7 0 2024-09-21T04:15:20.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/634: rename d5/d1f/d81/d9e/da8 to d5/de3/de7 0 2024-09-21T04:15:20.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/635: fdatasync d5/f23 0 2024-09-21T04:15:20.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/636: truncate d5/fcc 915100 0 2024-09-21T04:15:20.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/637: write d5/d1e/d37/f64 [5064376,98796] 0 2024-09-21T04:15:20.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/638: readlink d5/d60/d4c/l57 0 2024-09-21T04:15:20.391 INFO:tasks.workunit.client.1.smithi148.stdout:7/590: mknod d5/d8/d6f/d7d/cd8 0 2024-09-21T04:15:20.394 INFO:tasks.workunit.client.1.smithi148.stdout:9/639: link d5/d2b/f5d d5/d2b/d32/d67/fe8 0 2024-09-21T04:15:20.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/591: unlink d5/d14/c72 0 2024-09-21T04:15:20.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/592: read - d5/daa/fc5 zero size 2024-09-21T04:15:20.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/593: fdatasync d5/d8/d6f/d7d/d90/d28/f87 0 2024-09-21T04:15:20.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/594: creat d5/d14/d25/d79/d86/fd9 x:0 0 0 2024-09-21T04:15:20.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/595: read - d5/d14/d25/d40/d7b/f9e zero size 2024-09-21T04:15:20.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/640: mknod d5/d2b/d4e/db3/ce9 0 2024-09-21T04:15:20.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/641: stat d5/d1e/c94 0 2024-09-21T04:15:20.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/642: getdents d5/d60/de1 0 2024-09-21T04:15:20.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/643: write d5/d1e/d37/f3b [2974919,32252] 0 2024-09-21T04:15:20.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/644: write f1 [318446,117937] 0 2024-09-21T04:15:20.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/645: creat d5/d1e/fea x:0 0 0 2024-09-21T04:15:20.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/646: chown d5/d1f/d81/dcf 1327422 1 2024-09-21T04:15:20.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/647: read - d5/d1e/faa zero size 2024-09-21T04:15:20.407 INFO:tasks.workunit.client.1.smithi148.stdout:9/648: creat d5/d2b/feb x:0 0 0 2024-09-21T04:15:20.416 INFO:tasks.workunit.client.1.smithi148.stdout:1/570: dwrite db/d24/d2e/d59/f7e [0,4194304] 0 2024-09-21T04:15:20.416 INFO:tasks.workunit.client.1.smithi148.stdout:5/584: dwrite d3/f83 [0,4194304] 0 2024-09-21T04:15:20.416 INFO:tasks.workunit.client.1.smithi148.stdout:5/585: chown d3/d14/d1f/d26 2972 1 2024-09-21T04:15:20.420 INFO:tasks.workunit.client.1.smithi148.stdout:5/586: mknod d3/d4a/cce 0 2024-09-21T04:15:20.428 INFO:tasks.workunit.client.1.smithi148.stdout:5/587: dread d3/d34/d78/dbf/f96 [0,4194304] 0 2024-09-21T04:15:20.430 INFO:tasks.workunit.client.1.smithi148.stdout:5/588: rename d3/d14/d1f/d26/d2e/l69 to d3/d14/d1f/d26/d2e/d56/lcf 0 2024-09-21T04:15:20.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/649: dwrite d5/d60/f3c [0,4194304] 0 2024-09-21T04:15:20.441 INFO:tasks.workunit.client.1.smithi148.stdout:9/650: rename d5/d60/d28/c6e to d5/d2b/dbe/cec 0 2024-09-21T04:15:20.442 INFO:tasks.workunit.client.1.smithi148.stdout:1/571: dwrite db/dd/d21/f5c [4194304,4194304] 0 2024-09-21T04:15:20.442 INFO:tasks.workunit.client.1.smithi148.stdout:7/596: dwrite d5/d14/d25/d40/dab/fd4 [0,4194304] 0 2024-09-21T04:15:20.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/572: rename db/c3e to db/d46/d4c/cb7 0 2024-09-21T04:15:20.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/573: write db/dd/d21/d9b/f9e [121700,20676] 0 2024-09-21T04:15:20.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/589: dwrite d3/d14/d1f/d26/d2e/d56/fc4 [0,4194304] 0 2024-09-21T04:15:20.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/590: stat d3/d14/d1a/f4f 0 2024-09-21T04:15:20.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/591: fsync d3/d14/f44 0 2024-09-21T04:15:20.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/597: creat d5/d8/d6f/d7d/d90/d28/fda x:0 0 0 2024-09-21T04:15:20.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/598: fdatasync d5/daa/fd0 0 2024-09-21T04:15:20.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/599: write d5/daa/fc5 [321212,117149] 0 2024-09-21T04:15:20.455 INFO:tasks.workunit.client.1.smithi148.stdout:1/574: getdents db/d46/d4c/da3 0 2024-09-21T04:15:20.457 INFO:tasks.workunit.client.1.smithi148.stdout:5/592: symlink d3/d34/d78/dbf/dc2/ld0 0 2024-09-21T04:15:20.459 INFO:tasks.workunit.client.1.smithi148.stdout:7/600: unlink d5/d14/d25/f42 0 2024-09-21T04:15:20.461 INFO:tasks.workunit.client.1.smithi148.stdout:1/575: unlink db/d24/f33 0 2024-09-21T04:15:20.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/593: creat d3/d34/d78/dbf/fd1 x:0 0 0 2024-09-21T04:15:20.463 INFO:tasks.workunit.client.1.smithi148.stdout:5/594: chown d3/d34/f47 6774261 1 2024-09-21T04:15:20.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/601: symlink d5/d14/d25/d40/ldb 0 2024-09-21T04:15:20.465 INFO:tasks.workunit.client.1.smithi148.stdout:1/576: mknod db/d24/cb8 0 2024-09-21T04:15:20.465 INFO:tasks.workunit.client.1.smithi148.stdout:1/577: write db/d24/d2e/f9c [240179,97456] 0 2024-09-21T04:15:20.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/651: truncate d5/d39/f6f 685239 0 2024-09-21T04:15:20.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/652: fdatasync d5/d2b/f5d 0 2024-09-21T04:15:20.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/653: dread - d5/d2b/dce/fdd zero size 2024-09-21T04:15:20.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/654: unlink d5/d39/l3a 0 2024-09-21T04:15:20.471 INFO:tasks.workunit.client.1.smithi148.stdout:5/595: read d3/d14/f2d [1772123,70850] 0 2024-09-21T04:15:20.471 INFO:tasks.workunit.client.1.smithi148.stdout:9/655: truncate d5/d2b/dbe/f52 3983654 0 2024-09-21T04:15:20.471 INFO:tasks.workunit.client.1.smithi148.stdout:9/656: fsync d5/d2b/f4f 0 2024-09-21T04:15:20.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/596: mknod d3/d14/d1f/db1/db8/dba/cd2 0 2024-09-21T04:15:20.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/597: write d3/d34/d3c/d5d/f93 [1504642,20736] 0 2024-09-21T04:15:20.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/598: chown d3/d14/d1a/l35 27394 1 2024-09-21T04:15:20.477 INFO:tasks.workunit.client.1.smithi148.stdout:9/657: rmdir d5/d2b/d4e/db3 39 2024-09-21T04:15:20.480 INFO:tasks.workunit.client.1.smithi148.stdout:5/599: truncate d3/d37/f7f 483971 0 2024-09-21T04:15:20.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/602: dwrite d5/d8/d6f/dcd/fd7 [0,4194304] 0 2024-09-21T04:15:20.486 INFO:tasks.workunit.client.1.smithi148.stdout:9/658: mkdir d5/d2b/d4e/db3/ded 0 2024-09-21T04:15:20.487 INFO:tasks.workunit.client.1.smithi148.stdout:6/615: sync 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/653: sync 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/616: chown d0/d34/d97/f3f 78699 1 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/617: readlink d0/l35 0 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/618: chown d0/d34/d97/c1e 201317 1 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:2/703: sync 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:8/583: sync 2024-09-21T04:15:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:8/584: write dc/f68 [2187647,58391] 0 2024-09-21T04:15:20.490 INFO:tasks.workunit.client.1.smithi148.stdout:0/628: sync 2024-09-21T04:15:20.490 INFO:tasks.workunit.client.1.smithi148.stdout:3/550: sync 2024-09-21T04:15:20.493 INFO:tasks.workunit.client.1.smithi148.stdout:0/629: write d5/d6/d7/d11/f1d [888487,97565] 0 2024-09-21T04:15:20.495 INFO:tasks.workunit.client.1.smithi148.stdout:2/704: rename d0/d2f/f4a to d0/d9/d16/d55/d71/dc2/dcb/fd8 0 2024-09-21T04:15:20.495 INFO:tasks.workunit.client.1.smithi148.stdout:2/705: chown d0/d9/d16/f5b 1785494 1 2024-09-21T04:15:20.496 INFO:tasks.workunit.client.1.smithi148.stdout:8/585: getdents dc/d13/d76/d42 0 2024-09-21T04:15:20.499 INFO:tasks.workunit.client.1.smithi148.stdout:3/551: truncate d3/d1f/f47 9768586 0 2024-09-21T04:15:20.500 INFO:tasks.workunit.client.1.smithi148.stdout:0/630: mkdir d5/d6/d2a/d3f/d6d/da3/dcc 0 2024-09-21T04:15:20.500 INFO:tasks.workunit.client.1.smithi148.stdout:4/654: dread d4/d66/d7f/f2b [0,4194304] 0 2024-09-21T04:15:20.500 INFO:tasks.workunit.client.1.smithi148.stdout:4/655: write d4/d66/d7f/f97 [946336,85227] 0 2024-09-21T04:15:20.502 INFO:tasks.workunit.client.1.smithi148.stdout:6/619: rmdir d0/d1a/d23/d32/d93 39 2024-09-21T04:15:20.502 INFO:tasks.workunit.client.1.smithi148.stdout:2/706: creat d0/d9/d16/d55/fd9 x:0 0 0 2024-09-21T04:15:20.502 INFO:tasks.workunit.client.1.smithi148.stdout:3/552: rmdir d3/d5/d5c 39 2024-09-21T04:15:20.503 INFO:tasks.workunit.client.1.smithi148.stdout:0/631: mkdir d5/d6/d2a/dcd 0 2024-09-21T04:15:20.503 INFO:tasks.workunit.client.1.smithi148.stdout:2/707: dread d0/d9/dc/d13/f21 [0,4194304] 0 2024-09-21T04:15:20.504 INFO:tasks.workunit.client.1.smithi148.stdout:4/656: creat d4/d12/d75/d9c/fc4 x:0 0 0 2024-09-21T04:15:20.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/620: link d0/d1a/d23/dc/lb9 d0/d1a/d23/d27/d62/d9a/d92/lbb 0 2024-09-21T04:15:20.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/621: chown d0/d34/d38/d57/d58 7 1 2024-09-21T04:15:20.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/553: mkdir d3/d5/dac 0 2024-09-21T04:15:20.508 INFO:tasks.workunit.client.1.smithi148.stdout:0/632: write d5/d6/d2a/d2b/f39 [759365,47665] 0 2024-09-21T04:15:20.508 INFO:tasks.workunit.client.1.smithi148.stdout:0/633: readlink d5/d6/d2a/d3f/d6d/da3/lad 0 2024-09-21T04:15:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:3/554: mknod d3/d5/d6/d12/d15/d73/cad 0 2024-09-21T04:15:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:3/555: stat d3/d1f/d4d/d82/d52/f4a 0 2024-09-21T04:15:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:3/556: write d3/d1f/d4d/d82/f11 [674204,32015] 0 2024-09-21T04:15:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/657: rmdir d4/d66/d94 39 2024-09-21T04:15:20.513 INFO:tasks.workunit.client.1.smithi148.stdout:7/603: dwrite d5/d8/d6f/d7d/d90/faf [0,4194304] 0 2024-09-21T04:15:20.514 INFO:tasks.workunit.client.1.smithi148.stdout:5/600: dwrite d3/d14/d1f/d26/d2e/d56/f17 [0,4194304] 0 2024-09-21T04:15:20.516 INFO:tasks.workunit.client.1.smithi148.stdout:3/557: mknod d3/d5/d6/d12/d43/cae 0 2024-09-21T04:15:20.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/658: symlink d4/d66/d7f/lc5 0 2024-09-21T04:15:20.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/659: readlink d4/d87/d2d/la3 0 2024-09-21T04:15:20.516 INFO:tasks.workunit.client.1.smithi148.stdout:3/558: write d3/d1f/d4d/d82/d74/fa0 [109707,21026] 0 2024-09-21T04:15:20.519 INFO:tasks.workunit.client.1.smithi148.stdout:5/601: mknod d3/d4a/cd3 0 2024-09-21T04:15:20.519 INFO:tasks.workunit.client.1.smithi148.stdout:5/602: fsync d3/d14/fc6 0 2024-09-21T04:15:20.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/559: mkdir d3/d1f/d4d/d82/d23/daf 0 2024-09-21T04:15:20.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/560: mknod d3/d1f/d4d/cb0 0 2024-09-21T04:15:20.524 INFO:tasks.workunit.client.1.smithi148.stdout:3/561: write d3/d5/d6/f17 [1950217,27906] 0 2024-09-21T04:15:20.524 INFO:tasks.workunit.client.1.smithi148.stdout:5/603: getdents d3/d14/d1f/d72 0 2024-09-21T04:15:20.525 INFO:tasks.workunit.client.1.smithi148.stdout:3/562: mknod d3/d5/dac/cb1 0 2024-09-21T04:15:20.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/660: dread d4/d66/d7f/d2a/fbc [0,4194304] 0 2024-09-21T04:15:20.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/604: mknod d3/d14/d1f/d26/d59/cd4 0 2024-09-21T04:15:20.527 INFO:tasks.workunit.client.1.smithi148.stdout:3/563: creat d3/d5/d6/d2b/da2/da3/fb2 x:0 0 0 2024-09-21T04:15:20.527 INFO:tasks.workunit.client.1.smithi148.stdout:3/564: write d3/d1f/d4d/d82/d74/fa0 [350232,122010] 0 2024-09-21T04:15:20.527 INFO:tasks.workunit.client.1.smithi148.stdout:3/565: getdents d3/d1f/d4d/d82/d23/d28/d65 0 2024-09-21T04:15:20.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/661: unlink d4/d87/d2d/l4e 0 2024-09-21T04:15:20.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/662: write d4/f11 [1455698,97200] 0 2024-09-21T04:15:20.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/663: write d4/d12/d16/d29/dbd/fad [506055,14555] 0 2024-09-21T04:15:20.530 INFO:tasks.workunit.client.1.smithi148.stdout:5/605: dread d3/d14/d1f/d26/d2e/d3d/f89 [0,4194304] 0 2024-09-21T04:15:20.530 INFO:tasks.workunit.client.1.smithi148.stdout:5/606: dread - d3/d14/d1f/d26/d2e/d56/fc3 zero size 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/607: write d3/d37/f7f [1115594,71164] 0 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/608: chown d3/d14/d1f/d26/l86 2 1 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/609: write d3/d34/d3c/d5d/d75/fa9 [68392,20416] 0 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/610: dread - d3/d14/d1f/f80 zero size 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/611: chown d3/d4a/l6c 2539957 1 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/566: dread d3/d1f/f8e [0,4194304] 0 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:4/664: mknod d4/d12/d16/d29/cc6 0 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:4/665: chown d4/d66/d7f/c80 1 1 2024-09-21T04:15:20.531 INFO:tasks.workunit.client.1.smithi148.stdout:4/666: creat d4/d66/d7f/d2a/fc7 x:0 0 0 2024-09-21T04:15:20.532 INFO:tasks.workunit.client.1.smithi148.stdout:5/612: rename d3/d14/d1f/d26/l38 to d3/d14/d1f/d26/d2e/d3d/ld5 0 2024-09-21T04:15:20.533 INFO:tasks.workunit.client.1.smithi148.stdout:4/667: dread d4/d12/d83/fa6 [0,4194304] 0 2024-09-21T04:15:20.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/567: mkdir d3/d1f/d4d/d82/d23/d28/d46/db3 0 2024-09-21T04:15:20.535 INFO:tasks.workunit.client.1.smithi148.stdout:5/613: unlink d3/d14/d1f/d26/d59/cd4 0 2024-09-21T04:15:20.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/668: truncate d4/d66/d7f/f2b 3577915 0 2024-09-21T04:15:20.538 INFO:tasks.workunit.client.1.smithi148.stdout:3/568: mkdir d3/d5/db4 0 2024-09-21T04:15:20.539 INFO:tasks.workunit.client.1.smithi148.stdout:5/614: symlink d3/d14/ld6 0 2024-09-21T04:15:20.540 INFO:tasks.workunit.client.1.smithi148.stdout:3/569: mkdir d3/d1f/d4d/d82/d23/db5 0 2024-09-21T04:15:20.541 INFO:tasks.workunit.client.1.smithi148.stdout:5/615: dread d3/d34/d78/dbf/fbd [0,4194304] 0 2024-09-21T04:15:20.541 INFO:tasks.workunit.client.1.smithi148.stdout:5/616: write d3/d14/d1f/d26/d2e/d56/da7/fab [91069,118760] 0 2024-09-21T04:15:20.541 INFO:tasks.workunit.client.1.smithi148.stdout:3/570: mkdir d3/d5/d6/d12/d15/da7/db6 0 2024-09-21T04:15:20.543 INFO:tasks.workunit.client.1.smithi148.stdout:5/617: mknod d3/d14/d1f/d26/cd7 0 2024-09-21T04:15:20.544 INFO:tasks.workunit.client.1.smithi148.stdout:5/618: mkdir d3/d14/d1f/d72/dc9/dd8 0 2024-09-21T04:15:20.546 INFO:tasks.workunit.client.1.smithi148.stdout:3/571: dread d3/f55 [0,4194304] 0 2024-09-21T04:15:20.546 INFO:tasks.workunit.client.1.smithi148.stdout:2/708: dwrite d0/d9/dc/d13/f69 [0,4194304] 0 2024-09-21T04:15:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:2/709: dread - d0/d9/d59/fc1 zero size 2024-09-21T04:15:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:2/710: chown d0/d2b/da3/d8f/laa 281374 1 2024-09-21T04:15:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:7/604: dwrite d5/d8/d6f/d7d/d90/f6e [0,4194304] 0 2024-09-21T04:15:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:8/586: dwrite dc/d13/d76/f57 [0,4194304] 0 2024-09-21T04:15:20.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/669: dread d4/d3a/f53 [0,4194304] 0 2024-09-21T04:15:20.550 INFO:tasks.workunit.client.1.smithi148.stdout:3/572: creat d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 x:0 0 0 2024-09-21T04:15:20.551 INFO:tasks.workunit.client.1.smithi148.stdout:3/573: readlink d3/d5/d6/d12/l3f 0 2024-09-21T04:15:20.551 INFO:tasks.workunit.client.1.smithi148.stdout:5/619: dread d3/d14/f44 [0,4194304] 0 2024-09-21T04:15:20.552 INFO:tasks.workunit.client.1.smithi148.stdout:5/620: chown d3/d14/d1f/f80 1 1 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/605: write d5/d14/d25/fa2 [2731840,23862] 0 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/606: creat d5/d8/d6f/d7d/fdc x:0 0 0 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/607: chown d5/d8/d6f/d7d/d90/fce 1814679 1 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/608: chown d5/d14/lae 13205 1 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/609: chown d5/l32 0 1 2024-09-21T04:15:20.560 INFO:tasks.workunit.client.1.smithi148.stdout:7/610: fsync d5/d14/d25/d40/dab/fd4 0 2024-09-21T04:15:20.579 INFO:tasks.workunit.client.0.smithi110.stdout:0/855: sync 2024-09-21T04:15:20.579 INFO:tasks.workunit.client.0.smithi110.stdout:0/856: readlink d4/d47/l6a 0 2024-09-21T04:15:20.579 INFO:tasks.workunit.client.0.smithi110.stdout:0/857: chown d4/d47/d2c/d3a/f51 44 1 2024-09-21T04:15:20.582 INFO:tasks.workunit.client.0.smithi110.stdout:0/858: truncate d4/d8b/fda 409444 0 2024-09-21T04:15:20.584 INFO:tasks.workunit.client.1.smithi148.stdout:4/670: rename d4/d66/d7f/d2a/l7c to d4/d87/d91/lc8 0 2024-09-21T04:15:20.584 INFO:tasks.workunit.client.1.smithi148.stdout:4/671: chown d4/f6 986 1 2024-09-21T04:15:20.585 INFO:tasks.workunit.client.1.smithi148.stdout:3/574: mkdir d3/d5/d6/d2b/d96/db8 0 2024-09-21T04:15:20.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/575: write d3/d5/d6/d12/d43/d54/f71 [137589,48616] 0 2024-09-21T04:15:20.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/576: write d3/d1f/d4d/d82/d23/d28/d65/f67 [10494459,24152] 0 2024-09-21T04:15:20.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/577: chown d3/d1f/d4d/d82 24509 1 2024-09-21T04:15:20.587 INFO:tasks.workunit.client.0.smithi110.stdout:0/859: rename d4/f58 to d4/dd/d60/d72/f107 0 2024-09-21T04:15:20.589 INFO:tasks.workunit.client.1.smithi148.stdout:2/711: dwrite d0/d2b/f5e [4194304,4194304] 0 2024-09-21T04:15:20.590 INFO:tasks.workunit.client.1.smithi148.stdout:5/621: dwrite d3/d14/d1a/f4f [0,4194304] 0 2024-09-21T04:15:20.590 INFO:tasks.workunit.client.1.smithi148.stdout:7/611: dwrite d5/d8/f62 [0,4194304] 0 2024-09-21T04:15:20.591 INFO:tasks.workunit.client.0.smithi110.stdout:0/860: link d4/d1c/d62/l100 d4/dd/d10/d29/l108 0 2024-09-21T04:15:20.591 INFO:tasks.workunit.client.0.smithi110.stdout:0/861: fdatasync d4/dd/f32 0 2024-09-21T04:15:20.591 INFO:tasks.workunit.client.0.smithi110.stdout:0/862: fsync d4/fbc 0 2024-09-21T04:15:20.591 INFO:tasks.workunit.client.0.smithi110.stdout:0/863: fdatasync d4/dd/d10/d29/d5f/dc3/fe9 0 2024-09-21T04:15:20.593 INFO:tasks.workunit.client.1.smithi148.stdout:8/587: read dc/d13/d39/f51 [48334,115424] 0 2024-09-21T04:15:20.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/672: link d4/d66/d7f/d2a/l64 d4/d87/d91/lc9 0 2024-09-21T04:15:20.597 INFO:tasks.workunit.client.1.smithi148.stdout:4/673: readlink d4/d12/d16/l71 0 2024-09-21T04:15:20.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/578: rename d3/d1f/d4d/d82/d23/l6f to d3/d1f/d4d/d82/d74/lb9 0 2024-09-21T04:15:20.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/579: readlink d3/d1f/d4d/d82/l1b 0 2024-09-21T04:15:20.604 INFO:tasks.workunit.client.1.smithi148.stdout:5/622: dread d3/d14/d1a/f4f [0,4194304] 0 2024-09-21T04:15:20.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/712: mkdir d0/d9/d59/dda 0 2024-09-21T04:15:20.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/713: chown d0/d9/d59/f65 18864870 1 2024-09-21T04:15:20.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/714: dread - d0/d9/d16/d55/d71/dc2/dcb/fd6 zero size 2024-09-21T04:15:20.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/715: write d0/d2b/da3/d8f/f9d [493149,112721] 0 2024-09-21T04:15:20.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/716: chown d0/f1e 4301 1 2024-09-21T04:15:20.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/717: write d0/d9/f17 [2564468,109586] 0 2024-09-21T04:15:20.610 INFO:tasks.workunit.client.1.smithi148.stdout:8/588: dread dc/d13/d76/f57 [0,4194304] 0 2024-09-21T04:15:20.610 INFO:tasks.workunit.client.1.smithi148.stdout:8/589: fdatasync dc/d3e/f70 0 2024-09-21T04:15:20.610 INFO:tasks.workunit.client.0.smithi110.stdout:0/864: dwrite d4/dd/d10/d29/d5f/dc3/f106 [0,4194304] 0 2024-09-21T04:15:20.615 INFO:tasks.workunit.client.1.smithi148.stdout:2/718: write d0/d9/dc/d13/f69 [2512442,58171] 0 2024-09-21T04:15:20.617 INFO:tasks.workunit.client.0.smithi110.stdout:0/865: dread d4/d8b/fda [0,4194304] 0 2024-09-21T04:15:20.617 INFO:tasks.workunit.client.0.smithi110.stdout:0/866: fsync d4/d47/fab 0 2024-09-21T04:15:20.622 INFO:tasks.workunit.client.0.smithi110.stdout:0/867: symlink d4/d1c/d62/d91/l109 0 2024-09-21T04:15:20.624 INFO:tasks.workunit.client.1.smithi148.stdout:4/674: dwrite d4/d87/f32 [4194304,4194304] 0 2024-09-21T04:15:20.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/623: link d3/d4a/f52 d3/d14/d1f/d26/d2e/d56/da7/fd9 0 2024-09-21T04:15:20.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/624: creat d3/d14/d1a/fda x:0 0 0 2024-09-21T04:15:20.625 INFO:tasks.workunit.client.1.smithi148.stdout:7/612: dwrite d5/d14/d25/f52 [0,4194304] 0 2024-09-21T04:15:20.628 INFO:tasks.workunit.client.1.smithi148.stdout:8/590: symlink dc/d13/d76/d5d/lb2 0 2024-09-21T04:15:20.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/719: creat d0/d2f/fdb x:0 0 0 2024-09-21T04:15:20.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/720: write d0/d9/d16/d55/d71/dc2/dcb/fd6 [382646,90861] 0 2024-09-21T04:15:20.633 INFO:tasks.workunit.client.1.smithi148.stdout:3/580: getdents d3/d5/d6/d12 0 2024-09-21T04:15:20.633 INFO:tasks.workunit.client.1.smithi148.stdout:4/675: write d4/d87/f32 [8178197,31579] 0 2024-09-21T04:15:20.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/625: mkdir d3/d14/d1f/d26/d2e/ddb 0 2024-09-21T04:15:20.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/626: dread - d3/d14/d1f/d72/fc5 zero size 2024-09-21T04:15:20.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/627: chown d3/d14/d1f/d26/d2e/d56/da7/fab 3 1 2024-09-21T04:15:20.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/613: creat d5/d14/d25/fdd x:0 0 0 2024-09-21T04:15:20.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/614: readlink d5/d14/lb0 0 2024-09-21T04:15:20.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/591: mknod dc/d13/d39/d6e/dac/cb3 0 2024-09-21T04:15:20.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/592: write dc/f14 [4502036,121283] 0 2024-09-21T04:15:20.646 INFO:tasks.workunit.client.0.smithi110.stdout:0/868: dwrite d4/dd/d10/d29/d5f/fd8 [0,4194304] 0 2024-09-21T04:15:20.647 INFO:tasks.workunit.client.1.smithi148.stdout:2/721: rename d0/d9/d38/lcf to d0/d9/d16/d55/d6f/ldc 0 2024-09-21T04:15:20.647 INFO:tasks.workunit.client.1.smithi148.stdout:2/722: write d0/d9/d38/f6d [294920,22889] 0 2024-09-21T04:15:20.648 INFO:tasks.workunit.client.1.smithi148.stdout:2/723: readlink d0/l97 0 2024-09-21T04:15:20.648 INFO:tasks.workunit.client.1.smithi148.stdout:2/724: chown d0/d9/f17 6 1 2024-09-21T04:15:20.648 INFO:tasks.workunit.client.1.smithi148.stdout:2/725: readlink d0/d9/d16/d55/l86 0 2024-09-21T04:15:20.648 INFO:tasks.workunit.client.0.smithi110.stdout:0/869: dread d4/dd/d10/d29/d45/ff5 [0,4194304] 0 2024-09-21T04:15:20.649 INFO:tasks.workunit.client.1.smithi148.stdout:3/581: creat d3/d1f/d4d/d82/d23/fba x:0 0 0 2024-09-21T04:15:20.652 INFO:tasks.workunit.client.1.smithi148.stdout:1/578: sync 2024-09-21T04:15:20.652 INFO:tasks.workunit.client.1.smithi148.stdout:1/579: stat db/d46/d4c/l85 0 2024-09-21T04:15:20.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/580: write db/f75 [361883,44272] 0 2024-09-21T04:15:20.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/581: stat c8 0 2024-09-21T04:15:20.654 INFO:tasks.workunit.client.1.smithi148.stdout:8/593: mkdir dc/d3e/db4 0 2024-09-21T04:15:20.656 INFO:tasks.workunit.client.1.smithi148.stdout:4/676: getdents d4/d66 0 2024-09-21T04:15:20.656 INFO:tasks.workunit.client.1.smithi148.stdout:4/677: chown d4/c3b 44551980 1 2024-09-21T04:15:20.659 INFO:tasks.workunit.client.1.smithi148.stdout:1/582: readlink db/dd/l34 0 2024-09-21T04:15:20.659 INFO:tasks.workunit.client.0.smithi110.stdout:0/870: read d4/dd/d60/fa4 [1364612,3322] 0 2024-09-21T04:15:20.659 INFO:tasks.workunit.client.1.smithi148.stdout:8/594: symlink dc/d13/d76/d50/d80/lb5 0 2024-09-21T04:15:20.659 INFO:tasks.workunit.client.1.smithi148.stdout:8/595: write f5 [3197573,88271] 0 2024-09-21T04:15:20.661 INFO:tasks.workunit.client.1.smithi148.stdout:5/628: dwrite d3/d14/d1f/f42 [0,4194304] 0 2024-09-21T04:15:20.662 INFO:tasks.workunit.client.0.smithi110.stdout:0/871: rename d4/d1c/d62/d91/f105 to d4/dd/d10/d29/d5f/dc3/f10a 0 2024-09-21T04:15:20.662 INFO:tasks.workunit.client.0.smithi110.stdout:0/872: chown d4/dd/d53/d66 194245381 1 2024-09-21T04:15:20.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/629: rmdir d3/d14/d1f 39 2024-09-21T04:15:20.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/630: chown d3/d4a/cd3 283842871 1 2024-09-21T04:15:20.665 INFO:tasks.workunit.client.1.smithi148.stdout:1/583: mkdir db/dd/db9 0 2024-09-21T04:15:20.665 INFO:tasks.workunit.client.1.smithi148.stdout:1/584: chown db/d24/d2e/d71/cb5 30593 1 2024-09-21T04:15:20.665 INFO:tasks.workunit.client.1.smithi148.stdout:1/585: readlink db/dd/d21/l80 0 2024-09-21T04:15:20.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/678: link d4/d12/d16/d29/d61/f8e d4/d12/d16/d29/d61/d6c/d9d/fca 0 2024-09-21T04:15:20.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/679: truncate d4/d12/d16/d29/d61/d6c/d9d/fca 874982 0 2024-09-21T04:15:20.667 INFO:tasks.workunit.client.1.smithi148.stdout:4/680: chown d4/ca 82 1 2024-09-21T04:15:20.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/681: creat d4/d12/d16/fcb x:0 0 0 2024-09-21T04:15:20.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/682: mkdir d4/d12/d83/dcc 0 2024-09-21T04:15:20.675 INFO:tasks.workunit.client.1.smithi148.stdout:6/622: sync 2024-09-21T04:15:20.675 INFO:tasks.workunit.client.1.smithi148.stdout:0/634: sync 2024-09-21T04:15:20.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/659: sync 2024-09-21T04:15:20.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/660: write d5/d60/f3f [2169446,86047] 0 2024-09-21T04:15:20.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/661: mknod d5/d60/d4c/d5e/cee 0 2024-09-21T04:15:20.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/662: getdents d5/de3/de7 0 2024-09-21T04:15:20.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/663: dread - d5/d2b/d32/d67/d80/fa1 zero size 2024-09-21T04:15:20.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/664: write d5/d1e/f29 [1618374,74879] 0 2024-09-21T04:15:20.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/665: write d5/d1e/f65 [1413690,23220] 0 2024-09-21T04:15:20.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/873: write d4/dd/d10/d29/f6b [3188573,123429] 0 2024-09-21T04:15:20.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/874: write d4/d1c/d62/d69/d9b/ffd [695572,36759] 0 2024-09-21T04:15:20.695 INFO:tasks.workunit.client.0.smithi110.stdout:0/875: dread f2 [4194304,4194304] 0 2024-09-21T04:15:20.695 INFO:tasks.workunit.client.0.smithi110.stdout:0/876: fsync d4/d47/d2c/d3a/f43 0 2024-09-21T04:15:20.696 INFO:tasks.workunit.client.0.smithi110.stdout:0/877: getdents d4/d47/d2c 0 2024-09-21T04:15:20.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/878: creat d4/dd/d60/de5/f10b x:0 0 0 2024-09-21T04:15:20.697 INFO:tasks.workunit.client.0.smithi110.stdout:0/879: write d4/f85 [2553394,120961] 0 2024-09-21T04:15:20.698 INFO:tasks.workunit.client.0.smithi110.stdout:0/880: unlink d4/d8b/cd0 0 2024-09-21T04:15:20.698 INFO:tasks.workunit.client.1.smithi148.stdout:7/615: dwrite d5/d8/d6f/fc9 [0,4194304] 0 2024-09-21T04:15:20.699 INFO:tasks.workunit.client.1.smithi148.stdout:2/726: dwrite d0/d2b/f53 [0,4194304] 0 2024-09-21T04:15:20.699 INFO:tasks.workunit.client.0.smithi110.stdout:0/881: rename d4/d47/l6a to d4/dd/d53/df4/l10c 0 2024-09-21T04:15:20.701 INFO:tasks.workunit.client.1.smithi148.stdout:7/616: unlink d5/d8/d6f/d7d/d90/d9b/ca4 0 2024-09-21T04:15:20.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/727: creat d0/d2b/fdd x:0 0 0 2024-09-21T04:15:20.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/631: dwrite d3/d34/d78/fbe [0,4194304] 0 2024-09-21T04:15:20.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/632: chown d3/d37/f5f 71 1 2024-09-21T04:15:20.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/633: write d3/d14/d1f/f80 [99608,68292] 0 2024-09-21T04:15:20.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/666: dwrite d5/f42 [0,4194304] 0 2024-09-21T04:15:20.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/586: dwrite db/dd/d21/d9b/f9e [0,4194304] 0 2024-09-21T04:15:20.706 INFO:tasks.workunit.client.1.smithi148.stdout:7/617: truncate d5/daa/fc5 46479 0 2024-09-21T04:15:20.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/683: dwrite d4/d12/d16/d29/dbd/fad [0,4194304] 0 2024-09-21T04:15:20.709 INFO:tasks.workunit.client.1.smithi148.stdout:2/728: link d0/d2b/f7d d0/d9/dc/d13/d50/dc0/fde 0 2024-09-21T04:15:20.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/667: mknod d5/d1e/d37/cef 0 2024-09-21T04:15:20.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/668: truncate d5/d2b/d4e/d95/fad 238427 0 2024-09-21T04:15:20.719 INFO:tasks.workunit.client.1.smithi148.stdout:1/587: dread db/d24/f70 [0,4194304] 0 2024-09-21T04:15:20.721 INFO:tasks.workunit.client.1.smithi148.stdout:2/729: getdents d0/d2b/da3/d8f/d9f 0 2024-09-21T04:15:20.721 INFO:tasks.workunit.client.1.smithi148.stdout:7/618: dread d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:20.722 INFO:tasks.workunit.client.1.smithi148.stdout:1/588: link db/dd/c1c db/dd/d21/d9b/cba 0 2024-09-21T04:15:20.723 INFO:tasks.workunit.client.1.smithi148.stdout:2/730: mkdir d0/d9/dc/ddf 0 2024-09-21T04:15:20.723 INFO:tasks.workunit.client.1.smithi148.stdout:2/731: stat d0/d9/d38/fcc 0 2024-09-21T04:15:20.723 INFO:tasks.workunit.client.1.smithi148.stdout:2/732: write d0/d2b/fbb [7238,114836] 0 2024-09-21T04:15:20.724 INFO:tasks.workunit.client.1.smithi148.stdout:2/733: creat d0/d2b/da3/d8f/d9f/fe0 x:0 0 0 2024-09-21T04:15:20.726 INFO:tasks.workunit.client.1.smithi148.stdout:2/734: rename d0/d9/d16/f56 to d0/d9/d16/fe1 0 2024-09-21T04:15:20.729 INFO:tasks.workunit.client.1.smithi148.stdout:9/669: write d5/d60/d4c/dbd/f99 [3832069,93068] 0 2024-09-21T04:15:20.730 INFO:tasks.workunit.client.1.smithi148.stdout:9/670: creat d5/d2b/d4e/d95/ff0 x:0 0 0 2024-09-21T04:15:20.731 INFO:tasks.workunit.client.1.smithi148.stdout:9/671: symlink d5/d2b/d4e/db3/lf1 0 2024-09-21T04:15:20.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/672: rename d5/d60/d28/d73/lae to d5/d2b/lf2 0 2024-09-21T04:15:20.733 INFO:tasks.workunit.client.1.smithi148.stdout:9/673: symlink d5/d60/d28/d2f/dbf/lf3 0 2024-09-21T04:15:20.734 INFO:tasks.workunit.client.1.smithi148.stdout:1/589: dread db/dd/d21/d89/fab [0,4194304] 0 2024-09-21T04:15:20.734 INFO:tasks.workunit.client.1.smithi148.stdout:9/674: link d5/d1e/l68 d5/d2b/d32/d67/lf4 0 2024-09-21T04:15:20.736 INFO:tasks.workunit.client.1.smithi148.stdout:1/590: creat db/d46/d8f/fbb x:0 0 0 2024-09-21T04:15:20.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/675: mknod d5/d1f/d81/d9e/da2/cf5 0 2024-09-21T04:15:20.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/676: read - d5/d60/d28/fbb zero size 2024-09-21T04:15:20.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/677: chown d5/d60/d4c 28 1 2024-09-21T04:15:20.741 INFO:tasks.workunit.client.1.smithi148.stdout:9/678: dread d5/d60/d28/d73/fdf [0,4194304] 0 2024-09-21T04:15:20.742 INFO:tasks.workunit.client.1.smithi148.stdout:9/679: mkdir d5/d2b/dce/df6 0 2024-09-21T04:15:20.746 INFO:tasks.workunit.client.1.smithi148.stdout:5/634: dwrite d3/d14/d1f/d26/d2e/d3d/da0/fcc [0,4194304] 0 2024-09-21T04:15:20.746 INFO:tasks.workunit.client.1.smithi148.stdout:5/635: stat d3/d14/d1f/d26/d2e/d3d/f60 0 2024-09-21T04:15:20.746 INFO:tasks.workunit.client.0.smithi110.stdout:0/882: dwrite d4/dd/d10/d29/d45/fca [0,4194304] 0 2024-09-21T04:15:20.746 INFO:tasks.workunit.client.0.smithi110.stdout:0/883: fsync d4/dd/d10/f2a 0 2024-09-21T04:15:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/680: dwrite d5/d60/d28/fc7 [0,4194304] 0 2024-09-21T04:15:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/681: chown d5/d2b/d4e/db3/fcb 987648 1 2024-09-21T04:15:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/682: write d5/d1e/fea [555285,93416] 0 2024-09-21T04:15:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/591: dwrite db/d46/d4c/d82/f90 [0,4194304] 0 2024-09-21T04:15:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/592: creat db/d24/d2a/da4/fbc x:0 0 0 2024-09-21T04:15:20.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/735: dwrite d0/d2b/fbb [0,4194304] 0 2024-09-21T04:15:20.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/683: creat d5/d2b/d4e/db3/ded/ff7 x:0 0 0 2024-09-21T04:15:20.778 INFO:tasks.workunit.client.1.smithi148.stdout:9/684: write d5/fcc [269699,97890] 0 2024-09-21T04:15:20.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/593: mkdir db/d24/d2a/dbd 0 2024-09-21T04:15:20.782 INFO:tasks.workunit.client.1.smithi148.stdout:2/736: creat d0/d9/d16/d55/d71/dc4/fe2 x:0 0 0 2024-09-21T04:15:20.782 INFO:tasks.workunit.client.1.smithi148.stdout:9/685: truncate d5/d2b/d32/f5b 745683 0 2024-09-21T04:15:20.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/594: creat db/d46/d4c/da3/fbe x:0 0 0 2024-09-21T04:15:20.784 INFO:tasks.workunit.client.1.smithi148.stdout:2/737: creat d0/d9/dc/d13/fe3 x:0 0 0 2024-09-21T04:15:20.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/738: fdatasync d0/d9/d38/f6d 0 2024-09-21T04:15:20.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/739: rename d0/d9 to d0/d9/d59/dda/de4 22 2024-09-21T04:15:20.785 INFO:tasks.workunit.client.1.smithi148.stdout:9/686: unlink d5/d39/dba/cd3 0 2024-09-21T04:15:20.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/595: mkdir db/dd/d21/dbf 0 2024-09-21T04:15:20.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/596: write db/d24/d2e/daf/fb1 [749775,50480] 0 2024-09-21T04:15:20.786 INFO:tasks.workunit.client.1.smithi148.stdout:2/740: symlink d0/d9/dc/ddf/le5 0 2024-09-21T04:15:20.787 INFO:tasks.workunit.client.1.smithi148.stdout:9/687: mkdir d5/d2b/d32/d67/df8 0 2024-09-21T04:15:20.790 INFO:tasks.workunit.client.1.smithi148.stdout:1/597: write db/dd/fae [651511,95924] 0 2024-09-21T04:15:20.790 INFO:tasks.workunit.client.1.smithi148.stdout:1/598: dread - db/d24/d2e/d59/da7/fb4 zero size 2024-09-21T04:15:20.791 INFO:tasks.workunit.client.1.smithi148.stdout:1/599: mknod db/d24/d2e/d71/dad/cc0 0 2024-09-21T04:15:20.792 INFO:tasks.workunit.client.1.smithi148.stdout:1/600: write db/d46/d4c/d82/f88 [689124,105239] 0 2024-09-21T04:15:20.793 INFO:tasks.workunit.client.1.smithi148.stdout:1/601: mkdir db/d24/d2e/d59/da7/dc1 0 2024-09-21T04:15:20.793 INFO:tasks.workunit.client.1.smithi148.stdout:1/602: dread - db/d24/d2a/da4/fbc zero size 2024-09-21T04:15:20.793 INFO:tasks.workunit.client.1.smithi148.stdout:1/603: chown db/d24/c4b 268 1 2024-09-21T04:15:20.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/604: mkdir db/dd/db9/dc2 0 2024-09-21T04:15:20.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/605: fdatasync db/dd/d21/d89/fab 0 2024-09-21T04:15:20.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/606: stat db/dd/f12 0 2024-09-21T04:15:20.795 INFO:tasks.workunit.client.1.smithi148.stdout:1/607: read db/d46/d4c/d82/f88 [136425,70595] 0 2024-09-21T04:15:20.795 INFO:tasks.workunit.client.1.smithi148.stdout:1/608: chown db/d24/d2e 2525813 1 2024-09-21T04:15:20.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/609: creat db/dd/fc3 x:0 0 0 2024-09-21T04:15:20.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/610: mknod db/d46/d4c/d82/cc4 0 2024-09-21T04:15:20.798 INFO:tasks.workunit.client.1.smithi148.stdout:1/611: symlink db/d24/d2a/lc5 0 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:2/741: write d0/d9/dc/d13/f8b [839213,7771] 0 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:2/742: write d0/d9/dc/d13/f74 [19686,95429] 0 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:2/743: chown d0/f14 2310253 1 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:2/744: chown d0/d2b/da3/d8f/d9f/dc9 194 1 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/612: creat db/dd/d21/fc6 x:0 0 0 2024-09-21T04:15:20.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/613: dread - db/dd/d21/fc6 zero size 2024-09-21T04:15:20.807 INFO:tasks.workunit.client.1.smithi148.stdout:2/745: read d0/d2b/da3/da1/f2e [4029991,54905] 0 2024-09-21T04:15:20.812 INFO:tasks.workunit.client.1.smithi148.stdout:9/688: dwrite d5/d60/d28/f51 [0,4194304] 0 2024-09-21T04:15:20.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/689: dread d5/d60/d28/f30 [0,4194304] 0 2024-09-21T04:15:20.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/690: chown d5/d1e/d37/d46 3816 1 2024-09-21T04:15:20.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/691: creat d5/d1e/d37/d50/ff9 x:0 0 0 2024-09-21T04:15:20.834 INFO:tasks.workunit.client.1.smithi148.stdout:9/692: dwrite d5/fd [0,4194304] 0 2024-09-21T04:15:20.868 INFO:tasks.workunit.client.1.smithi148.stdout:7/619: dwrite d5/fa7 [0,4194304] 0 2024-09-21T04:15:20.869 INFO:tasks.workunit.client.1.smithi148.stdout:5/636: dwrite d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:20.869 INFO:tasks.workunit.client.1.smithi148.stdout:1/614: getdents db/d46/d8f 0 2024-09-21T04:15:20.871 INFO:tasks.workunit.client.0.smithi110.stdout:0/884: dwrite d4/d1c/d62/fef [0,4194304] 0 2024-09-21T04:15:20.871 INFO:tasks.workunit.client.0.smithi110.stdout:0/885: creat d4/dd/d10/d29/d45/f10d x:0 0 0 2024-09-21T04:15:20.872 INFO:tasks.workunit.client.0.smithi110.stdout:0/886: write d4/dd/d53/d66/f74 [5161369,9427] 0 2024-09-21T04:15:20.872 INFO:tasks.workunit.client.1.smithi148.stdout:7/620: link d5/d8/d6f/d7d/d90/l73 d5/lde 0 2024-09-21T04:15:20.879 INFO:tasks.workunit.client.0.smithi110.stdout:0/887: truncate d4/f1f 1820113 0 2024-09-21T04:15:20.879 INFO:tasks.workunit.client.0.smithi110.stdout:0/888: fsync d4/dd/d60/fa4 0 2024-09-21T04:15:20.880 INFO:tasks.workunit.client.1.smithi148.stdout:7/621: write d5/d8/ff [1265452,56836] 0 2024-09-21T04:15:20.881 INFO:tasks.workunit.client.1.smithi148.stdout:9/693: fsync d5/d2b/d4e/db3/ded/ff7 0 2024-09-21T04:15:20.881 INFO:tasks.workunit.client.1.smithi148.stdout:9/694: write d5/d1e/d37/fb0 [609057,117022] 0 2024-09-21T04:15:20.881 INFO:tasks.workunit.client.1.smithi148.stdout:9/695: readlink d5/ld1 0 2024-09-21T04:15:20.883 INFO:tasks.workunit.client.1.smithi148.stdout:1/615: rename db/dd to db/dd/dc7 22 2024-09-21T04:15:20.885 INFO:tasks.workunit.client.1.smithi148.stdout:7/622: mknod d5/d14/d25/d40/d8b/cdf 0 2024-09-21T04:15:20.885 INFO:tasks.workunit.client.1.smithi148.stdout:7/623: read d5/d14/d25/f54 [3321954,3003] 0 2024-09-21T04:15:20.885 INFO:tasks.workunit.client.1.smithi148.stdout:7/624: write d5/d14/d25/f54 [4036978,5500] 0 2024-09-21T04:15:20.887 INFO:tasks.workunit.client.1.smithi148.stdout:9/696: rename d5/fb8 to d5/d1e/ffa 0 2024-09-21T04:15:20.888 INFO:tasks.workunit.client.0.smithi110.stdout:0/889: write d4/d47/fb2 [2580715,31579] 0 2024-09-21T04:15:20.889 INFO:tasks.workunit.client.1.smithi148.stdout:9/697: rename d5/d2b/feb to d5/d60/d69/ffb 0 2024-09-21T04:15:20.889 INFO:tasks.workunit.client.1.smithi148.stdout:9/698: write d5/d2b/d4e/db3/ded/ff7 [809887,72313] 0 2024-09-21T04:15:20.894 INFO:tasks.workunit.client.1.smithi148.stdout:2/746: truncate d0/d2b/fbb 4049787 0 2024-09-21T04:15:20.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/637: dwrite d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:20.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/638: chown d3/d37 0 1 2024-09-21T04:15:20.896 INFO:tasks.workunit.client.1.smithi148.stdout:2/747: unlink d0/d9/d16/f37 0 2024-09-21T04:15:20.897 INFO:tasks.workunit.client.1.smithi148.stdout:2/748: link d0/d9/dc/d13/f74 d0/d9/d16/d55/d71/dc2/dcb/fe6 0 2024-09-21T04:15:20.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/625: dread d5/d14/d25/d40/dab/dad/fb4 [4194304,4194304] 0 2024-09-21T04:15:20.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/626: fdatasync d5/d8/d6f/d7d/fdc 0 2024-09-21T04:15:20.899 INFO:tasks.workunit.client.1.smithi148.stdout:1/616: write db/d24/d2e/d59/f7e [206041,122894] 0 2024-09-21T04:15:20.899 INFO:tasks.workunit.client.1.smithi148.stdout:2/749: creat d0/d2b/da3/fe7 x:0 0 0 2024-09-21T04:15:20.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/750: write d0/d9/dc/d13/f69 [2971627,25608] 0 2024-09-21T04:15:20.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/751: write d0/d9/d38/fcc [1045589,3358] 0 2024-09-21T04:15:20.900 INFO:tasks.workunit.client.1.smithi148.stdout:1/617: write db/d24/d2e/d71/d94/d9a/f95 [82601,16656] 0 2024-09-21T04:15:20.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/752: symlink d0/d9/dc/d13/d4c/le8 0 2024-09-21T04:15:20.901 INFO:tasks.workunit.client.1.smithi148.stdout:2/753: stat d0/d2b/da3/da1/l83 0 2024-09-21T04:15:20.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/890: dread d4/d1c/d62/f90 [0,4194304] 0 2024-09-21T04:15:20.902 INFO:tasks.workunit.client.1.smithi148.stdout:7/627: getdents d5/d14 0 2024-09-21T04:15:20.903 INFO:tasks.workunit.client.1.smithi148.stdout:1/618: mknod db/d46/d4c/cc8 0 2024-09-21T04:15:20.904 INFO:tasks.workunit.client.1.smithi148.stdout:1/619: dread - db/d46/d4c/f86 zero size 2024-09-21T04:15:20.904 INFO:tasks.workunit.client.0.smithi110.stdout:0/891: mkdir d4/dd/d60/d10e 0 2024-09-21T04:15:20.907 INFO:tasks.workunit.client.1.smithi148.stdout:2/754: creat d0/d2b/da3/d8f/d9f/dc9/fe9 x:0 0 0 2024-09-21T04:15:20.908 INFO:tasks.workunit.client.0.smithi110.stdout:0/892: getdents d4/d1c/d62/d69 0 2024-09-21T04:15:20.908 INFO:tasks.workunit.client.0.smithi110.stdout:0/893: chown d4/dd/d10/d29/f5d 15499609 1 2024-09-21T04:15:20.908 INFO:tasks.workunit.client.0.smithi110.stdout:0/894: creat d4/dd/d10/d29/d45/f10f x:0 0 0 2024-09-21T04:15:20.910 INFO:tasks.workunit.client.1.smithi148.stdout:2/755: unlink d0/d9/dc/d13/d25/c91 0 2024-09-21T04:15:20.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/895: symlink d4/dd/d10/d8a/l110 0 2024-09-21T04:15:20.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/896: readlink d4/dd/d10/l7c 0 2024-09-21T04:15:20.914 INFO:tasks.workunit.client.1.smithi148.stdout:2/756: mkdir d0/d9/dc/d13/d25/dea 0 2024-09-21T04:15:20.915 INFO:tasks.workunit.client.1.smithi148.stdout:5/639: dwrite d3/d14/d1f/d26/d2e/d3d/f60 [0,4194304] 0 2024-09-21T04:15:20.919 INFO:tasks.workunit.client.1.smithi148.stdout:9/699: rename d5/d60/d69/ffb to d5/d60/d4c/dbd/ffc 0 2024-09-21T04:15:20.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/507: sync 2024-09-21T04:15:20.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/508: creat d7/d96/d99/d3c/d1f/d43/fb0 x:0 0 0 2024-09-21T04:15:20.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/509: fsync d7/d96/d99/d3c/d1f/d43/f6c 0 2024-09-21T04:15:20.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/640: rename d3/d24/l3b to d3/d88/ldc 0 2024-09-21T04:15:20.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/641: creat d3/d37/fdd x:0 0 0 2024-09-21T04:15:20.927 INFO:tasks.workunit.client.0.smithi110.stdout:0/897: read d4/f1f [214576,55204] 0 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:4/684: sync 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:0/635: sync 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:3/582: sync 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:8/596: sync 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:6/623: sync 2024-09-21T04:15:20.928 INFO:tasks.workunit.client.0.smithi110.stdout:0/898: write d4/d1c/fe6 [86337,49872] 0 2024-09-21T04:15:20.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/700: unlink d5/d2b/dbe/ca0 0 2024-09-21T04:15:20.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/701: readlink d5/l1c 0 2024-09-21T04:15:20.930 INFO:tasks.workunit.client.1.smithi148.stdout:5/642: mkdir d3/d14/d1f/d26/d2e/ddb/dde 0 2024-09-21T04:15:20.935 INFO:tasks.workunit.client.1.smithi148.stdout:4/685: creat d4/d12/da7/fcd x:0 0 0 2024-09-21T04:15:20.935 INFO:tasks.workunit.client.1.smithi148.stdout:1/620: dwrite f6 [0,4194304] 0 2024-09-21T04:15:20.935 INFO:tasks.workunit.client.0.smithi110.stdout:0/899: dread d4/dd/d10/f22 [0,4194304] 0 2024-09-21T04:15:20.936 INFO:tasks.workunit.client.0.smithi110.stdout:0/900: write d4/dd/f32 [790752,66011] 0 2024-09-21T04:15:20.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/757: dwrite d0/d9/d59/f65 [0,4194304] 0 2024-09-21T04:15:20.936 INFO:tasks.workunit.client.1.smithi148.stdout:8/597: link dc/d13/d76/d50/faa dc/d13/d28/d62/fb6 0 2024-09-21T04:15:20.937 INFO:tasks.workunit.client.0.smithi110.stdout:0/901: dread d4/dd/d10/d29/d45/ff5 [0,4194304] 0 2024-09-21T04:15:20.937 INFO:tasks.workunit.client.0.smithi110.stdout:0/902: fdatasync d4/dd/f1b 0 2024-09-21T04:15:20.937 INFO:tasks.workunit.client.0.smithi110.stdout:0/903: read - d4/dd/d53/d7b/ffe zero size 2024-09-21T04:15:20.940 INFO:tasks.workunit.client.1.smithi148.stdout:3/583: rename d3/d5/f7b to d3/d5/d6/d12/d43/fbb 0 2024-09-21T04:15:20.940 INFO:tasks.workunit.client.1.smithi148.stdout:3/584: write d3/d5/d6/d12/d43/d54/f71 [691499,18153] 0 2024-09-21T04:15:20.944 INFO:tasks.workunit.client.0.smithi110.stdout:0/904: symlink d4/d1c/d62/d91/l111 0 2024-09-21T04:15:20.946 INFO:tasks.workunit.client.1.smithi148.stdout:0/636: write d5/d6/d2a/d2b/f2c [3099344,128552] 0 2024-09-21T04:15:20.947 INFO:tasks.workunit.client.1.smithi148.stdout:1/621: mkdir db/dd/d21/d89/dc9 0 2024-09-21T04:15:20.949 INFO:tasks.workunit.client.0.smithi110.stdout:0/905: mknod d4/dd/d60/d72/c112 0 2024-09-21T04:15:20.949 INFO:tasks.workunit.client.0.smithi110.stdout:0/906: readlink d4/dd/d10/l26 0 2024-09-21T04:15:20.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/598: mkdir dc/d13/d3c/d81/db0/db7 0 2024-09-21T04:15:20.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/599: fsync dc/d13/d39/d6e/f6f 0 2024-09-21T04:15:20.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/624: rmdir d0/d8/d18 39 2024-09-21T04:15:20.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/625: write d0/d34/d38/d57/f75 [81840,120661] 0 2024-09-21T04:15:20.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/626: write d0/d8/d18/f25 [2768682,85331] 0 2024-09-21T04:15:20.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/627: write d0/d34/d97/f1f [1747768,13821] 0 2024-09-21T04:15:20.953 INFO:tasks.workunit.client.1.smithi148.stdout:0/637: dread d5/d6/d2a/d2b/d84/da5/f27 [0,4194304] 0 2024-09-21T04:15:20.953 INFO:tasks.workunit.client.1.smithi148.stdout:5/643: mknod d3/cdf 0 2024-09-21T04:15:20.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/622: mknod db/d24/d2e/daf/cca 0 2024-09-21T04:15:20.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/623: readlink db/dd/l3f 0 2024-09-21T04:15:20.957 INFO:tasks.workunit.client.1.smithi148.stdout:1/624: write db/f15 [5172295,46783] 0 2024-09-21T04:15:20.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/585: write d3/d1f/d4d/d82/d52/f59 [2520080,88020] 0 2024-09-21T04:15:20.960 INFO:tasks.workunit.client.1.smithi148.stdout:8/600: rename dc/d13/d76/f6d to dc/d13/d3c/d81/d85/da1/fb8 0 2024-09-21T04:15:20.960 INFO:tasks.workunit.client.1.smithi148.stdout:6/628: mkdir d0/d8/dba/dbc 0 2024-09-21T04:15:20.961 INFO:tasks.workunit.client.1.smithi148.stdout:0/638: fsync d5/d6/d2a/d2b/f61 0 2024-09-21T04:15:20.964 INFO:tasks.workunit.client.1.smithi148.stdout:8/601: rename dc/d3e/f70 to dc/d13/d3c/d81/d98/fb9 0 2024-09-21T04:15:20.966 INFO:tasks.workunit.client.1.smithi148.stdout:3/586: creat d3/d1f/d4d/d82/d23/d28/d65/fbc x:0 0 0 2024-09-21T04:15:20.968 INFO:tasks.workunit.client.1.smithi148.stdout:3/587: creat d3/d1f/d4d/d82/d21/fbd x:0 0 0 2024-09-21T04:15:20.970 INFO:tasks.workunit.client.0.smithi110.stdout:9/510: dwrite d7/d96/d99/d3c/d11/f81 [0,4194304] 0 2024-09-21T04:15:20.970 INFO:tasks.workunit.client.0.smithi110.stdout:0/907: write d4/dd/d10/d29/d45/f5b [719243,35830] 0 2024-09-21T04:15:20.974 INFO:tasks.workunit.client.1.smithi148.stdout:6/629: dread d0/d34/d97/f30 [0,4194304] 0 2024-09-21T04:15:20.974 INFO:tasks.workunit.client.1.smithi148.stdout:2/758: dwrite d0/d9/dc/d13/d25/f79 [0,4194304] 0 2024-09-21T04:15:20.977 INFO:tasks.workunit.client.1.smithi148.stdout:9/702: dwrite d5/d2b/d32/f82 [0,4194304] 0 2024-09-21T04:15:20.977 INFO:tasks.workunit.client.1.smithi148.stdout:2/759: symlink d0/d2b/da3/d95/leb 0 2024-09-21T04:15:20.978 INFO:tasks.workunit.client.1.smithi148.stdout:4/686: dwrite d4/f6 [0,4194304] 0 2024-09-21T04:15:20.979 INFO:tasks.workunit.client.1.smithi148.stdout:3/588: write d3/d1f/d4d/d82/d23/d28/d46/f5b [1824837,72977] 0 2024-09-21T04:15:20.979 INFO:tasks.workunit.client.1.smithi148.stdout:3/589: creat d3/d5/fbe x:0 0 0 2024-09-21T04:15:20.982 INFO:tasks.workunit.client.1.smithi148.stdout:2/760: mkdir d0/d9/dc/d13/dec 0 2024-09-21T04:15:20.982 INFO:tasks.workunit.client.1.smithi148.stdout:2/761: write d0/d2b/f7d [1887178,65652] 0 2024-09-21T04:15:20.983 INFO:tasks.workunit.client.1.smithi148.stdout:9/703: dread d5/d1e/d37/fb0 [0,4194304] 0 2024-09-21T04:15:20.983 INFO:tasks.workunit.client.1.smithi148.stdout:9/704: readlink d5/d2b/d4e/le0 0 2024-09-21T04:15:20.984 INFO:tasks.workunit.client.1.smithi148.stdout:5/644: dwrite d3/d14/d1f/d26/d2e/d56/f8a [0,4194304] 0 2024-09-21T04:15:20.984 INFO:tasks.workunit.client.1.smithi148.stdout:4/687: mkdir d4/d66/d7f/d9a/dce 0 2024-09-21T04:15:20.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/705: dread d5/d60/d28/f6a [0,4194304] 0 2024-09-21T04:15:20.988 INFO:tasks.workunit.client.1.smithi148.stdout:3/590: link d3/f49 d3/d1f/d4d/d82/d21/fbf 0 2024-09-21T04:15:20.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/762: unlink d0/d9/dc/d13/d25/f45 0 2024-09-21T04:15:20.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/763: stat d0/d2b/da3/d95/fa9 0 2024-09-21T04:15:20.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/764: stat d0/d2b/da3/d82 0 2024-09-21T04:15:20.990 INFO:tasks.workunit.client.1.smithi148.stdout:8/602: dwrite dc/f21 [0,4194304] 0 2024-09-21T04:15:20.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/591: dread d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:20.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/592: dread - d3/d1f/d4d/d82/d21/f63 zero size 2024-09-21T04:15:20.993 INFO:tasks.workunit.client.1.smithi148.stdout:4/688: rename d4/d12/f65 to d4/d87/d91/fcf 0 2024-09-21T04:15:20.994 INFO:tasks.workunit.client.1.smithi148.stdout:9/706: mknod d5/d2b/dbe/cfd 0 2024-09-21T04:15:20.996 INFO:tasks.workunit.client.1.smithi148.stdout:0/639: dwrite d5/d6/d7/f83 [0,4194304] 0 2024-09-21T04:15:20.996 INFO:tasks.workunit.client.1.smithi148.stdout:2/765: mkdir d0/d9/dc/d13/daf/db2/ded 0 2024-09-21T04:15:20.996 INFO:tasks.workunit.client.1.smithi148.stdout:2/766: fsync d0/d9/dc/f24 0 2024-09-21T04:15:20.997 INFO:tasks.workunit.client.1.smithi148.stdout:8/603: creat dc/d13/d39/d6e/fba x:0 0 0 2024-09-21T04:15:20.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/593: creat d3/fc0 x:0 0 0 2024-09-21T04:15:20.997 INFO:tasks.workunit.client.1.smithi148.stdout:8/604: chown dc/d13/l37 56 1 2024-09-21T04:15:20.998 INFO:tasks.workunit.client.0.smithi110.stdout:3/489: sync 2024-09-21T04:15:20.998 INFO:tasks.workunit.client.0.smithi110.stdout:3/490: fdatasync d0/d22/f46 0 2024-09-21T04:15:20.998 INFO:tasks.workunit.client.0.smithi110.stdout:3/491: write d0/d12/d16/f62 [219533,74400] 0 2024-09-21T04:15:20.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/689: mkdir d4/d87/d2d/dd0 0 2024-09-21T04:15:21.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/640: unlink d5/d6/d2a/d3f/d8b/c9c 0 2024-09-21T04:15:21.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/641: read d5/d6/d2a/d2b/f39 [826835,34283] 0 2024-09-21T04:15:21.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/642: chown d5/d6/d2a/d3f/d8b/ca6 0 1 2024-09-21T04:15:21.002 INFO:tasks.workunit.client.1.smithi148.stdout:0/643: chown d5/d6/d2a/d45/d70 10677 1 2024-09-21T04:15:21.002 INFO:tasks.workunit.client.1.smithi148.stdout:2/767: symlink d0/d9/da2/lee 0 2024-09-21T04:15:21.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/605: mknod dc/d13/d76/d50/cbb 0 2024-09-21T04:15:21.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/606: read dc/f18 [2711134,105814] 0 2024-09-21T04:15:21.004 INFO:tasks.workunit.client.1.smithi148.stdout:0/644: mknod d5/cce 0 2024-09-21T04:15:21.008 INFO:tasks.workunit.client.1.smithi148.stdout:3/594: write d3/d1f/d4d/d82/d23/d28/d65/f67 [8715755,433] 0 2024-09-21T04:15:21.009 INFO:tasks.workunit.client.1.smithi148.stdout:4/690: mkdir d4/d12/d16/d29/d61/d7e/dd1 0 2024-09-21T04:15:21.011 INFO:tasks.workunit.client.1.smithi148.stdout:8/607: rename dc/d13/d39/d6e/l73 to dc/d13/d3c/d81/db0/lbc 0 2024-09-21T04:15:21.011 INFO:tasks.workunit.client.1.smithi148.stdout:8/608: write dc/d13/d3c/d81/d85/f87 [618953,115386] 0 2024-09-21T04:15:21.011 INFO:tasks.workunit.client.1.smithi148.stdout:8/609: write dc/d13/d76/d5d/fa7 [236169,91294] 0 2024-09-21T04:15:21.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/492: dread d0/d12/d16/d1a/f3b [4194304,4194304] 0 2024-09-21T04:15:21.012 INFO:tasks.workunit.client.0.smithi110.stdout:3/493: creat d0/d12/d16/d1a/d5c/faa x:0 0 0 2024-09-21T04:15:21.012 INFO:tasks.workunit.client.1.smithi148.stdout:6/630: dread d0/d34/d97/f1f [0,4194304] 0 2024-09-21T04:15:21.014 INFO:tasks.workunit.client.1.smithi148.stdout:3/595: link d3/d1f/d4d/d82/d23/d28/d65/l90 d3/d1f/d4d/d82/d23/d28/d65/lc1 0 2024-09-21T04:15:21.014 INFO:tasks.workunit.client.1.smithi148.stdout:3/596: readlink d3/d1f/d4d/d82/d52/l88 0 2024-09-21T04:15:21.014 INFO:tasks.workunit.client.1.smithi148.stdout:3/597: fdatasync d3/d1f/f8e 0 2024-09-21T04:15:21.014 INFO:tasks.workunit.client.0.smithi110.stdout:0/908: dwrite d4/d47/fc9 [0,4194304] 0 2024-09-21T04:15:21.015 INFO:tasks.workunit.client.1.smithi148.stdout:6/631: symlink d0/d34/da0/lbd 0 2024-09-21T04:15:21.015 INFO:tasks.workunit.client.1.smithi148.stdout:6/632: chown d0/d1a/d23/f12 39721 1 2024-09-21T04:15:21.018 INFO:tasks.workunit.client.1.smithi148.stdout:3/598: mknod d3/d1f/d4d/d82/d23/db5/cc2 0 2024-09-21T04:15:21.019 INFO:tasks.workunit.client.0.smithi110.stdout:0/909: rename d4/dd/d10/d29/d45/f10f to d4/d1c/d62/d91/dd6/f113 0 2024-09-21T04:15:21.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/633: mkdir d0/d1a/db8/dbe 0 2024-09-21T04:15:21.022 INFO:tasks.workunit.client.1.smithi148.stdout:6/634: creat d0/d34/d38/d43/fbf x:0 0 0 2024-09-21T04:15:21.022 INFO:tasks.workunit.client.1.smithi148.stdout:6/635: readlink d0/d1a/d23/l2b 0 2024-09-21T04:15:21.025 INFO:tasks.workunit.client.1.smithi148.stdout:6/636: mkdir d0/d34/d38/d57/d58/dc0 0 2024-09-21T04:15:21.025 INFO:tasks.workunit.client.1.smithi148.stdout:8/610: dread f9 [0,4194304] 0 2024-09-21T04:15:21.025 INFO:tasks.workunit.client.1.smithi148.stdout:8/611: write dc/d13/f9b [548557,71321] 0 2024-09-21T04:15:21.026 INFO:tasks.workunit.client.1.smithi148.stdout:8/612: mkdir dc/d13/dbd 0 2024-09-21T04:15:21.028 INFO:tasks.workunit.client.1.smithi148.stdout:8/613: mknod dc/d13/d3c/d81/db0/cbe 0 2024-09-21T04:15:21.028 INFO:tasks.workunit.client.1.smithi148.stdout:8/614: readlink dc/d13/d76/d5d/lb2 0 2024-09-21T04:15:21.028 INFO:tasks.workunit.client.1.smithi148.stdout:8/615: write dc/d13/d39/d6e/dac/faf [293827,114941] 0 2024-09-21T04:15:21.032 INFO:tasks.workunit.client.0.smithi110.stdout:9/511: dwrite d7/d96/d99/d3c/d1f/d68/f6b [0,4194304] 0 2024-09-21T04:15:21.032 INFO:tasks.workunit.client.0.smithi110.stdout:9/512: dread - d7/d96/d99/d3c/d1f/d43/f92 zero size 2024-09-21T04:15:21.036 INFO:tasks.workunit.client.0.smithi110.stdout:9/513: dread f2 [0,4194304] 0 2024-09-21T04:15:21.037 INFO:tasks.workunit.client.0.smithi110.stdout:9/514: readlink d7/d96/d99/d3c/d1f/d25/l74 0 2024-09-21T04:15:21.039 INFO:tasks.workunit.client.1.smithi148.stdout:3/599: dread d3/d1f/d4d/d82/d21/fbf [0,4194304] 0 2024-09-21T04:15:21.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/600: rename d3/d1f/d4d/d82/d21/fbd to d3/d5/d5c/fc3 0 2024-09-21T04:15:21.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/601: write d3/d1f/d4d/d82/d21/fbf [7165,40994] 0 2024-09-21T04:15:21.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/602: truncate d3/d1f/d4d/d82/d23/d28/f42 2329434 0 2024-09-21T04:15:21.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/603: fsync d3/d5/d6/f17 0 2024-09-21T04:15:21.044 INFO:tasks.workunit.client.1.smithi148.stdout:9/707: dwrite d5/d1e/d37/f3b [0,4194304] 0 2024-09-21T04:15:21.045 INFO:tasks.workunit.client.1.smithi148.stdout:9/708: rename d5/d60/d28/d2f/dbf/dc2/cd9 to d5/d2b/dbe/cfe 0 2024-09-21T04:15:21.046 INFO:tasks.workunit.client.1.smithi148.stdout:9/709: mknod d5/d2b/cff 0 2024-09-21T04:15:21.047 INFO:tasks.workunit.client.1.smithi148.stdout:9/710: chown d5/d1f/d81 2 1 2024-09-21T04:15:21.048 INFO:tasks.workunit.client.1.smithi148.stdout:5/645: dwrite d3/d14/d1f/d26/d2e/d56/da7/fd9 [0,4194304] 0 2024-09-21T04:15:21.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/691: dwrite d4/d12/d83/fa6 [0,4194304] 0 2024-09-21T04:15:21.049 INFO:tasks.workunit.client.1.smithi148.stdout:5/646: creat d3/d4a/fe0 x:0 0 0 2024-09-21T04:15:21.054 INFO:tasks.workunit.client.0.smithi110.stdout:9/515: write d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a [2523895,83802] 0 2024-09-21T04:15:21.054 INFO:tasks.workunit.client.0.smithi110.stdout:9/516: truncate d7/d96/d99/d3c/d1f/d43/fb0 733225 0 2024-09-21T04:15:21.054 INFO:tasks.workunit.client.0.smithi110.stdout:9/517: chown d7/d96/d99/d3c/d35 558299523 1 2024-09-21T04:15:21.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/768: dwrite d0/d9/d16/fe1 [4194304,4194304] 0 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:2/769: chown d0/d9/d16/d55/d71/lc5 1993 1 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:2/770: creat d0/d2b/da3/d82/fef x:0 0 0 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:4/692: rename d4/d87/f57 to d4/d12/d78/fd2 0 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:4/693: stat d4/d12/f41 0 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/647: truncate d3/d14/d1f/d26/d2e/d3d/da0/fcc 2060245 0 2024-09-21T04:15:21.055 INFO:tasks.workunit.client.1.smithi148.stdout:0/645: dwrite d5/f85 [0,4194304] 0 2024-09-21T04:15:21.056 INFO:tasks.workunit.client.0.smithi110.stdout:3/494: dwrite d0/f7 [4194304,4194304] 0 2024-09-21T04:15:21.056 INFO:tasks.workunit.client.0.smithi110.stdout:9/518: creat d7/d26/fb1 x:0 0 0 2024-09-21T04:15:21.056 INFO:tasks.workunit.client.0.smithi110.stdout:9/519: dread - d7/d96/f9a zero size 2024-09-21T04:15:21.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/616: dwrite dc/d13/d76/d42/f4a [0,4194304] 0 2024-09-21T04:15:21.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/617: chown dc/d13/d76/f2c 107370 1 2024-09-21T04:15:21.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/618: write dc/d13/d76/d50/faa [582137,71867] 0 2024-09-21T04:15:21.058 INFO:tasks.workunit.client.1.smithi148.stdout:8/619: truncate dc/d13/d28/f2f 1810411 0 2024-09-21T04:15:21.058 INFO:tasks.workunit.client.1.smithi148.stdout:8/620: write dc/d13/d3c/f93 [823561,24952] 0 2024-09-21T04:15:21.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/637: dwrite d0/d34/d38/d43/f5e [4194304,4194304] 0 2024-09-21T04:15:21.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: pgmap v7: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 204 MiB/s rd, 242 MiB/s wr, 388 op/s 2024-09-21T04:15:21.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: Standby manager daemon smithi110.axmqzb started 2024-09-21T04:15:21.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/crt"}]: dispatch 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/key"}]: dispatch 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:20 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.060 INFO:tasks.workunit.client.1.smithi148.stdout:4/694: symlink d4/d87/dc1/ld3 0 2024-09-21T04:15:21.062 INFO:tasks.workunit.client.0.smithi110.stdout:3/495: mkdir d0/d12/d3c/d5a/d70/dab 0 2024-09-21T04:15:21.063 INFO:tasks.workunit.client.1.smithi148.stdout:5/648: rmdir d3/d14/d1f 39 2024-09-21T04:15:21.065 INFO:tasks.workunit.client.0.smithi110.stdout:3/496: dread d0/d51/f67 [0,4194304] 0 2024-09-21T04:15:21.067 INFO:tasks.workunit.client.1.smithi148.stdout:9/711: dwrite d5/d60/d28/d73/fc3 [0,4194304] 0 2024-09-21T04:15:21.068 INFO:tasks.workunit.client.0.smithi110.stdout:9/520: getdents d7 0 2024-09-21T04:15:21.069 INFO:tasks.workunit.client.0.smithi110.stdout:9/521: creat d7/d96/d99/d3c/d1f/d43/fb2 x:0 0 0 2024-09-21T04:15:21.075 INFO:tasks.workunit.client.0.smithi110.stdout:9/522: rename d7/d96/d99/d3c/d1f/d43/l8d to d7/d96/d99/d3c/d1f/d72/d59/lb3 0 2024-09-21T04:15:21.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/523: mknod d7/cb4 0 2024-09-21T04:15:21.079 INFO:tasks.workunit.client.0.smithi110.stdout:3/497: dread d0/d12/d16/d1a/f35 [0,4194304] 0 2024-09-21T04:15:21.082 INFO:tasks.workunit.client.0.smithi110.stdout:9/524: unlink d7/d96/d99/d3c/d4f/d9f/d6f/l95 0 2024-09-21T04:15:21.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/638: dwrite d0/d1a/d23/f19 [0,4194304] 0 2024-09-21T04:15:21.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/639: write d0/d1a/d23/d32/fad [319709,26307] 0 2024-09-21T04:15:21.088 INFO:tasks.workunit.client.1.smithi148.stdout:6/640: dread d0/d34/d38/f3a [0,4194304] 0 2024-09-21T04:15:21.089 INFO:tasks.workunit.client.1.smithi148.stdout:6/641: creat d0/d1a/d23/d53/fc1 x:0 0 0 2024-09-21T04:15:21.089 INFO:tasks.workunit.client.1.smithi148.stdout:3/604: dwrite d3/d1f/d4d/d82/d23/d28/d65/f67 [0,4194304] 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:8/621: link dc/d13/d3c/lae dc/d43/lbf 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:8/622: fdatasync dc/d13/f24 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:2/771: dwrite d0/d9/d16/d55/d6f/f3a [8388608,4194304] 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:3/605: write d3/d1f/f8e [627515,14702] 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:3/606: fdatasync d3/d1f/d4d/d82/d23/d28/d65/f8f 0 2024-09-21T04:15:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:3/607: write d3/f85 [757682,97806] 0 2024-09-21T04:15:21.091 INFO:tasks.workunit.client.1.smithi148.stdout:6/642: dread d0/d1a/d23/d32/d93/fa8 [0,4194304] 0 2024-09-21T04:15:21.094 INFO:tasks.workunit.client.1.smithi148.stdout:4/695: mknod d4/d12/d16/d29/d61/d7e/dd1/cd4 0 2024-09-21T04:15:21.095 INFO:tasks.workunit.client.1.smithi148.stdout:4/696: truncate d4/d66/d7f/d2a/fc7 298849 0 2024-09-21T04:15:21.095 INFO:tasks.workunit.client.1.smithi148.stdout:4/697: chown d4/d12/d16/d29/d61/d6c/d9d/daf/cb8 1476 1 2024-09-21T04:15:21.095 INFO:tasks.workunit.client.1.smithi148.stdout:4/698: fdatasync d4/d12/f63 0 2024-09-21T04:15:21.095 INFO:tasks.workunit.client.1.smithi148.stdout:4/699: chown d4/d87/d91/lc9 183 1 2024-09-21T04:15:21.095 INFO:tasks.workunit.client.1.smithi148.stdout:5/649: mknod d3/d14/d1f/d26/d2e/ddb/ce1 0 2024-09-21T04:15:21.098 INFO:tasks.workunit.client.1.smithi148.stdout:9/712: link d5/d60/d69/fbc d5/de3/de7/f100 0 2024-09-21T04:15:21.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/646: getdents d5/d6/d7/d11/d7b 0 2024-09-21T04:15:21.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/647: creat d5/d6/d7/d11/fcf x:0 0 0 2024-09-21T04:15:21.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/648: chown d5/d6/d7/d22/f46 3 1 2024-09-21T04:15:21.100 INFO:tasks.workunit.client.0.smithi110.stdout:9/525: dwrite d7/d96/d99/d3c/d1f/d43/f6e [0,4194304] 0 2024-09-21T04:15:21.101 INFO:tasks.workunit.client.0.smithi110.stdout:9/526: read d7/d96/d99/d3c/d1f/d72/f63 [68605,83723] 0 2024-09-21T04:15:21.101 INFO:tasks.workunit.client.1.smithi148.stdout:4/700: dread d4/d87/d91/fcf [0,4194304] 0 2024-09-21T04:15:21.103 INFO:tasks.workunit.client.1.smithi148.stdout:8/623: mknod dc/d43/d4c/d4f/cc0 0 2024-09-21T04:15:21.111 INFO:tasks.workunit.client.1.smithi148.stdout:2/772: unlink d0/d2b/d48/fab 0 2024-09-21T04:15:21.111 INFO:tasks.workunit.client.1.smithi148.stdout:2/773: write d0/d9/d16/d55/d6f/f7e [198880,93229] 0 2024-09-21T04:15:21.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/608: creat d3/d5/d5c/fc4 x:0 0 0 2024-09-21T04:15:21.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/609: creat d3/d5/d6/d2b/da2/da3/fc5 x:0 0 0 2024-09-21T04:15:21.115 INFO:tasks.workunit.client.1.smithi148.stdout:6/643: rename d0/d34/d38/d57/d58/dc0 to d0/d1a/d23/d27/dc2 0 2024-09-21T04:15:21.115 INFO:tasks.workunit.client.1.smithi148.stdout:6/644: write d0/d1a/d23/dc/d68/fa6 [1008579,32432] 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/650: link d3/d14/d1f/d26/d2e/d56/fc3 d3/d34/d3c/d5d/d75/fe2 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/651: fsync d3/f83 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/652: readlink d3/d24/l9d 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/653: chown d3/d14/l5a 3788731 1 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/654: chown d3/d34/f64 35083 1 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/655: write d3/d34/f47 [1021409,81428] 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/656: chown d3/d34/d3c/d5d/d75/la5 789864994 1 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/657: stat d3/d14/d1f/d26/d2e/d3d/da0 0 2024-09-21T04:15:21.117 INFO:tasks.workunit.client.1.smithi148.stdout:5/658: creat d3/d14/d1f/d26/d2e/d56/fe3 x:0 0 0 2024-09-21T04:15:21.118 INFO:tasks.workunit.client.1.smithi148.stdout:0/649: mkdir d5/d6/d7/dd0 0 2024-09-21T04:15:21.119 INFO:tasks.workunit.client.1.smithi148.stdout:8/624: getdents dc/d13/d76/d50/d80 0 2024-09-21T04:15:21.119 INFO:tasks.workunit.client.1.smithi148.stdout:8/625: write dc/d13/d28/f48 [1260021,38183] 0 2024-09-21T04:15:21.120 INFO:tasks.workunit.client.1.smithi148.stdout:8/626: getdents dc/d13/d76/d50/d80 0 2024-09-21T04:15:21.121 INFO:tasks.workunit.client.1.smithi148.stdout:2/774: write d0/d9/dc/d13/d25/f79 [2297816,112434] 0 2024-09-21T04:15:21.121 INFO:tasks.workunit.client.1.smithi148.stdout:2/775: creat d0/d2b/da3/d8f/d9f/ff0 x:0 0 0 2024-09-21T04:15:21.122 INFO:tasks.workunit.client.1.smithi148.stdout:3/610: creat d3/d5/d6/d12/d43/d54/fc6 x:0 0 0 2024-09-21T04:15:21.124 INFO:tasks.workunit.client.1.smithi148.stdout:4/701: rename d4/d66/d7f/f97 to d4/d87/fd5 0 2024-09-21T04:15:21.124 INFO:tasks.workunit.client.1.smithi148.stdout:4/702: fdatasync d4/f6 0 2024-09-21T04:15:21.126 INFO:tasks.workunit.client.1.smithi148.stdout:8/627: mkdir dc/d13/d3c/d81/d98/dc1 0 2024-09-21T04:15:21.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/628: chown dc/d13/d39/d6e/dac 40 1 2024-09-21T04:15:21.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/629: stat dc/d13/f35 0 2024-09-21T04:15:21.129 INFO:tasks.workunit.client.1.smithi148.stdout:2/776: symlink d0/d9/dc/d13/daf/db2/lf1 0 2024-09-21T04:15:21.129 INFO:tasks.workunit.client.1.smithi148.stdout:2/777: stat d0/d9/d16/d55/d71 0 2024-09-21T04:15:21.130 INFO:tasks.workunit.client.1.smithi148.stdout:2/778: write d0/d9/dc/d13/d4c/f8a [1312305,50227] 0 2024-09-21T04:15:21.130 INFO:tasks.workunit.client.1.smithi148.stdout:3/611: mknod d3/d1f/d4d/d82/d23/da1/cc7 0 2024-09-21T04:15:21.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/659: rename d3/d34/d3c/d5d/d75/f9b to d3/d37/fe4 0 2024-09-21T04:15:21.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/660: creat d3/d14/d1a/fe5 x:0 0 0 2024-09-21T04:15:21.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/661: chown d3/d14/d1f/db1/db8/dba/fc1 13929 1 2024-09-21T04:15:21.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/662: fdatasync d3/d14/d1f/f7e 0 2024-09-21T04:15:21.134 INFO:tasks.workunit.client.1.smithi148.stdout:6/645: dwrite d0/d1a/d23/d32/faa [4194304,4194304] 0 2024-09-21T04:15:21.134 INFO:tasks.workunit.client.1.smithi148.stdout:8/630: unlink dc/f17 0 2024-09-21T04:15:21.140 INFO:tasks.workunit.client.1.smithi148.stdout:5/663: creat d3/d24/dad/fe6 x:0 0 0 2024-09-21T04:15:21.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/646: creat d0/d1a/db8/dbe/fc3 x:0 0 0 2024-09-21T04:15:21.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/650: rename d5/d6/d2a/d45/f50 to d5/d6/d2a/d2b/fd1 0 2024-09-21T04:15:21.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/651: fdatasync d5/d6/d2a/d2b/fd1 0 2024-09-21T04:15:21.145 INFO:tasks.workunit.client.1.smithi148.stdout:6/647: mkdir d0/d1a/d23/d27/d62/dc4 0 2024-09-21T04:15:21.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/648: write d0/d8/d18/f25 [1545752,68587] 0 2024-09-21T04:15:21.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/649: stat d0/d8/l28 0 2024-09-21T04:15:21.148 INFO:tasks.workunit.client.1.smithi148.stdout:0/652: symlink d5/d6/d2a/d3f/d6d/da3/dcc/ld2 0 2024-09-21T04:15:21.149 INFO:tasks.workunit.client.1.smithi148.stdout:0/653: creat d5/d6/d2a/d2b/fd3 x:0 0 0 2024-09-21T04:15:21.149 INFO:tasks.workunit.client.1.smithi148.stdout:0/654: write d5/d6/f15 [3152277,93430] 0 2024-09-21T04:15:21.150 INFO:tasks.workunit.client.1.smithi148.stdout:0/655: write d5/d6/d2a/d45/d70/fc6 [1430376,64468] 0 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: pgmap v7: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 204 MiB/s rd, 242 MiB/s wr, 388 op/s 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: Standby manager daemon smithi110.axmqzb started 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/crt"}]: dispatch 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/key"}]: dispatch 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/4284468897' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:20 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:21.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/628: sync 2024-09-21T04:15:21.210 INFO:tasks.workunit.client.1.smithi148.stdout:7/629: write d5/d8/d6f/d7d/d90/faf [235055,52712] 0 2024-09-21T04:15:21.210 INFO:tasks.workunit.client.1.smithi148.stdout:6/650: dread d0/d8/f49 [0,4194304] 0 2024-09-21T04:15:21.210 INFO:tasks.workunit.client.1.smithi148.stdout:7/630: write d5/d8/f9 [1330112,15962] 0 2024-09-21T04:15:21.211 INFO:tasks.workunit.client.1.smithi148.stdout:7/631: stat d5/d8/d6f/d7d/d8a/ld3 0 2024-09-21T04:15:21.213 INFO:tasks.workunit.client.1.smithi148.stdout:7/632: rename d5/c2a to d5/d8/d6f/d7d/d90/d28/ce0 0 2024-09-21T04:15:21.218 INFO:tasks.workunit.client.1.smithi148.stdout:1/625: sync 2024-09-21T04:15:21.239 INFO:tasks.workunit.client.1.smithi148.stdout:3/612: dwrite d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:21.240 INFO:tasks.workunit.client.1.smithi148.stdout:0/656: dwrite d5/d6/f63 [0,4194304] 0 2024-09-21T04:15:21.240 INFO:tasks.workunit.client.1.smithi148.stdout:0/657: write d5/d6/d2a/d2b/f39 [689396,88666] 0 2024-09-21T04:15:21.240 INFO:tasks.workunit.client.1.smithi148.stdout:0/658: readlink d5/d6/d7/d11/l5f 0 2024-09-21T04:15:21.241 INFO:tasks.workunit.client.1.smithi148.stdout:3/613: creat d3/d5/d6/d2b/da2/fc8 x:0 0 0 2024-09-21T04:15:21.241 INFO:tasks.workunit.client.1.smithi148.stdout:3/614: write d3/d5/d6/d2b/da2/da3/fc5 [898164,122083] 0 2024-09-21T04:15:21.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/631: dwrite dc/d13/d3c/d81/d98/fb9 [0,4194304] 0 2024-09-21T04:15:21.247 INFO:tasks.workunit.client.0.smithi110.stdout:9/527: dwrite d7/d96/d99/d3c/d1f/d43/f6c [0,4194304] 0 2024-09-21T04:15:21.247 INFO:tasks.workunit.client.1.smithi148.stdout:4/703: dwrite d4/d12/d16/d29/f33 [0,4194304] 0 2024-09-21T04:15:21.248 INFO:tasks.workunit.client.0.smithi110.stdout:9/528: dread d7/d96/d99/f38 [0,4194304] 0 2024-09-21T04:15:21.248 INFO:tasks.workunit.client.1.smithi148.stdout:5/664: dwrite d3/f9 [0,4194304] 0 2024-09-21T04:15:21.248 INFO:tasks.workunit.client.1.smithi148.stdout:2/779: dwrite d0/d9/d16/fe1 [0,4194304] 0 2024-09-21T04:15:21.248 INFO:tasks.workunit.client.1.smithi148.stdout:2/780: write d0/f14 [4964995,6477] 0 2024-09-21T04:15:21.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/615: mkdir d3/d5/db4/dc9 0 2024-09-21T04:15:21.254 INFO:tasks.workunit.client.1.smithi148.stdout:6/651: dwrite d0/d1a/d23/d53/fc1 [0,4194304] 0 2024-09-21T04:15:21.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/616: dread - d3/d1f/d4d/d82/d21/f63 zero size 2024-09-21T04:15:21.254 INFO:tasks.workunit.client.0.smithi110.stdout:9/529: mkdir d7/d96/d99/d3c/d1f/d43/db5 0 2024-09-21T04:15:21.254 INFO:tasks.workunit.client.0.smithi110.stdout:9/530: chown d7/d96/d99/d3c/f5a 13 1 2024-09-21T04:15:21.255 INFO:tasks.workunit.client.0.smithi110.stdout:9/531: chown d7/d96/d99/d3c/lc 65957 1 2024-09-21T04:15:21.256 INFO:tasks.workunit.client.0.smithi110.stdout:9/532: truncate d7/d10/f12 284793 0 2024-09-21T04:15:21.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/633: dwrite d5/d8/d6f/d7d/f68 [0,4194304] 0 2024-09-21T04:15:21.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/634: write d5/d14/d25/d40/d8b/f99 [146089,92955] 0 2024-09-21T04:15:21.257 INFO:tasks.workunit.client.1.smithi148.stdout:4/704: truncate d4/d66/d7f/f2e 2904234 0 2024-09-21T04:15:21.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/652: dread d0/d34/d97/f30 [0,4194304] 0 2024-09-21T04:15:21.258 INFO:tasks.workunit.client.0.smithi110.stdout:9/533: rmdir d7/d96/d99/d3c/d1f/d68 39 2024-09-21T04:15:21.260 INFO:tasks.workunit.client.0.smithi110.stdout:9/534: write d7/d96/d99/d3c/d1f/d43/fb0 [34037,106699] 0 2024-09-21T04:15:21.264 INFO:tasks.workunit.client.1.smithi148.stdout:2/781: creat d0/d9/da2/ff2 x:0 0 0 2024-09-21T04:15:21.265 INFO:tasks.workunit.client.1.smithi148.stdout:0/659: getdents d5/d6/d2a/d3f/d8b 0 2024-09-21T04:15:21.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/535: creat d7/d96/d99/d3c/dae/fb6 x:0 0 0 2024-09-21T04:15:21.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/536: truncate d7/d96/d99/d3c/d11/f8c 690435 0 2024-09-21T04:15:21.269 INFO:tasks.workunit.client.1.smithi148.stdout:2/782: dread d0/d9/d16/d55/d71/fbd [0,4194304] 0 2024-09-21T04:15:21.269 INFO:tasks.workunit.client.0.smithi110.stdout:9/537: truncate d7/d96/d99/d3c/d35/f9c 1014707 0 2024-09-21T04:15:21.269 INFO:tasks.workunit.client.0.smithi110.stdout:9/538: fsync d7/d96/d99/f38 0 2024-09-21T04:15:21.272 INFO:tasks.workunit.client.0.smithi110.stdout:9/539: mkdir d7/d10/d88/d8f/db7 0 2024-09-21T04:15:21.273 INFO:tasks.workunit.client.1.smithi148.stdout:3/617: symlink d3/d1f/d4d/d82/d23/d28/lca 0 2024-09-21T04:15:21.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/632: dwrite dc/d13/d3c/d81/d85/f87 [0,4194304] 0 2024-09-21T04:15:21.277 INFO:tasks.workunit.client.1.smithi148.stdout:6/653: dwrite d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:21.277 INFO:tasks.workunit.client.1.smithi148.stdout:6/654: fdatasync d0/d8/d18/f60 0 2024-09-21T04:15:21.277 INFO:tasks.workunit.client.0.smithi110.stdout:9/540: unlink d7/d96/d99/d3c/d1f/d72/d59/c9d 0 2024-09-21T04:15:21.277 INFO:tasks.workunit.client.0.smithi110.stdout:9/541: stat d7/d96/d99/d3c/d1f/d43 0 2024-09-21T04:15:21.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/635: mknod d5/d8/d64/ce1 0 2024-09-21T04:15:21.282 INFO:tasks.workunit.client.1.smithi148.stdout:6/655: write d0/d1a/d23/dc/d68/fa6 [988965,21686] 0 2024-09-21T04:15:21.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/665: dwrite d3/d14/d1f/f42 [0,4194304] 0 2024-09-21T04:15:21.283 INFO:tasks.workunit.client.1.smithi148.stdout:2/783: write d0/d9/d16/d55/d6f/f3a [6398336,25387] 0 2024-09-21T04:15:21.283 INFO:tasks.workunit.client.1.smithi148.stdout:4/705: rename d4/d12/da7/fcd to d4/d87/d2d/dd0/fd6 0 2024-09-21T04:15:21.284 INFO:tasks.workunit.client.1.smithi148.stdout:4/706: chown d4/d12/d78/fd2 0 1 2024-09-21T04:15:21.290 INFO:tasks.workunit.client.0.smithi110.stdout:9/542: write d7/d96/d99/d3c/f5a [1009098,35195] 0 2024-09-21T04:15:21.291 INFO:tasks.workunit.client.1.smithi148.stdout:5/666: dread d3/d14/d1a/f4d [0,4194304] 0 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:8/595: sync 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:7/566: sync 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:7/567: chown dd/c4c 3437958 1 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:1/898: sync 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:2/415: sync 2024-09-21T04:15:21.293 INFO:tasks.workunit.client.0.smithi110.stdout:2/416: truncate d2/d1a/d1f/d29/f71 671162 0 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:2/417: write d2/d50/d56/f53 [6194,64095] 0 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:4/716: sync 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:6/642: sync 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:5/616: sync 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:6/643: readlink d4/d12/l2c 0 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:5/617: dread - d1/d49/d69/fca zero size 2024-09-21T04:15:21.294 INFO:tasks.workunit.client.0.smithi110.stdout:5/618: readlink d1/d49/d4d/l1f 0 2024-09-21T04:15:21.296 INFO:tasks.workunit.client.1.smithi148.stdout:7/636: creat d5/d14/d25/d79/fe2 x:0 0 0 2024-09-21T04:15:21.296 INFO:tasks.workunit.client.1.smithi148.stdout:6/656: truncate d0/d1a/d23/dc/d68/f50 679844 0 2024-09-21T04:15:21.296 INFO:tasks.workunit.client.0.smithi110.stdout:8/596: getdents d0/d7/d23/d82/d85 0 2024-09-21T04:15:21.299 INFO:tasks.workunit.client.0.smithi110.stdout:7/568: mkdir dd/db1 0 2024-09-21T04:15:21.303 INFO:tasks.workunit.client.0.smithi110.stdout:4/717: symlink d17/d1e/d64/lf0 0 2024-09-21T04:15:21.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/707: dwrite d4/f11 [0,4194304] 0 2024-09-21T04:15:21.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/708: chown d4/d12/d75/d9c/fc4 347739 1 2024-09-21T04:15:21.308 INFO:tasks.workunit.client.0.smithi110.stdout:8/597: unlink d0/d30/d54/f59 0 2024-09-21T04:15:21.308 INFO:tasks.workunit.client.0.smithi110.stdout:8/598: write d0/d7/d23/d52/d8b/faf [236253,28643] 0 2024-09-21T04:15:21.308 INFO:tasks.workunit.client.0.smithi110.stdout:8/599: dread - d0/d30/d34/d3e/d32/f73 zero size 2024-09-21T04:15:21.312 INFO:tasks.workunit.client.1.smithi148.stdout:2/784: mknod d0/d9/cf3 0 2024-09-21T04:15:21.313 INFO:tasks.workunit.client.0.smithi110.stdout:1/899: mkdir d2/d9/d23/d120 0 2024-09-21T04:15:21.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/633: dwrite dc/d43/f54 [0,4194304] 0 2024-09-21T04:15:21.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/634: readlink dc/d43/l6c 0 2024-09-21T04:15:21.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/635: write dc/f68 [2268147,17142] 0 2024-09-21T04:15:21.326 INFO:tasks.workunit.client.0.smithi110.stdout:2/418: dwrite d2/d1a/d1f/d29/d19/d62/f78 [0,4194304] 0 2024-09-21T04:15:21.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/419: creat d2/d50/d56/f7e x:0 0 0 2024-09-21T04:15:21.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/420: truncate d2/d1a/d1f/d65/f68 1488657 0 2024-09-21T04:15:21.327 INFO:tasks.workunit.client.0.smithi110.stdout:6/644: dwrite d4/d78/daa/f11 [0,4194304] 0 2024-09-21T04:15:21.341 INFO:tasks.workunit.client.0.smithi110.stdout:2/421: dwrite d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:21.341 INFO:tasks.workunit.client.0.smithi110.stdout:2/422: stat d2/d1a/d1f/d5f/l73 0 2024-09-21T04:15:21.342 INFO:tasks.workunit.client.0.smithi110.stdout:2/423: creat d2/d50/f7f x:0 0 0 2024-09-21T04:15:21.382 INFO:tasks.workunit.client.0.smithi110.stdout:7/569: rename dd/d1c/d39 to dd/d1c/dad/db2 0 2024-09-21T04:15:21.384 INFO:tasks.workunit.client.0.smithi110.stdout:1/900: unlink d2/d9/d49/d4f/d87/fd1 0 2024-09-21T04:15:21.385 INFO:tasks.workunit.client.0.smithi110.stdout:4/718: rmdir d17/d34/d3d/d57/db9 39 2024-09-21T04:15:21.385 INFO:tasks.workunit.client.0.smithi110.stdout:4/719: stat d17/d1e/d77/dcf/fe4 0 2024-09-21T04:15:21.397 INFO:tasks.workunit.client.0.smithi110.stdout:6/645: creat d4/d12/d16/d66/d7a/d83/fcf x:0 0 0 2024-09-21T04:15:21.399 INFO:tasks.workunit.client.1.smithi148.stdout:0/660: rmdir d5/d6 39 2024-09-21T04:15:21.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/424: truncate d2/d50/d56/f7a 1159549 0 2024-09-21T04:15:21.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/425: chown d2/d1a/c69 1 1 2024-09-21T04:15:21.403 INFO:tasks.workunit.client.0.smithi110.stdout:4/720: dwrite d17/d34/d3d/d20/d43/d51/fb5 [0,4194304] 0 2024-09-21T04:15:21.404 INFO:tasks.workunit.client.1.smithi148.stdout:3/618: rmdir d3/d1f/d4d/d82/d23 39 2024-09-21T04:15:21.407 INFO:tasks.workunit.client.1.smithi148.stdout:5/667: truncate d3/f8 1403787 0 2024-09-21T04:15:21.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/637: creat d5/d8/fe3 x:0 0 0 2024-09-21T04:15:21.409 INFO:tasks.workunit.client.0.smithi110.stdout:6/646: dread d4/d12/d17/f2a [0,4194304] 0 2024-09-21T04:15:21.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/657: truncate d0/d34/d97/f30 16984 0 2024-09-21T04:15:21.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/658: write d0/d1a/d23/dc/d68/f86 [2199459,33037] 0 2024-09-21T04:15:21.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/659: write d0/d1a/d23/d32/d93/f9c [755716,108761] 0 2024-09-21T04:15:21.412 INFO:tasks.workunit.client.0.smithi110.stdout:7/570: link dd/c4c dd/d1c/dad/db2/d4a/d52/cb3 0 2024-09-21T04:15:21.414 INFO:tasks.workunit.client.0.smithi110.stdout:1/901: rename d2/d9/d49/d7a/dee/f10e to d2/d9/d26/d3d/f121 0 2024-09-21T04:15:21.416 INFO:tasks.workunit.client.0.smithi110.stdout:2/426: rmdir d2/d1a/d1f/d29/d19 39 2024-09-21T04:15:21.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/785: symlink d0/d9/lf4 0 2024-09-21T04:15:21.421 INFO:tasks.workunit.client.0.smithi110.stdout:7/571: mknod dd/d1c/dad/db2/d4a/d52/cb4 0 2024-09-21T04:15:21.421 INFO:tasks.workunit.client.0.smithi110.stdout:1/902: write d2/d9/d23/d4e/d57/f10d [3420059,59102] 0 2024-09-21T04:15:21.422 INFO:tasks.workunit.client.0.smithi110.stdout:1/903: chown d2/d9/d26/d3d/d8f/l9b 10678819 1 2024-09-21T04:15:21.426 INFO:tasks.workunit.client.1.smithi148.stdout:8/636: unlink dc/d43/d4c/d4f/f8e 0 2024-09-21T04:15:21.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/427: rename d2/f54 to d2/d1a/d1f/d65/f80 0 2024-09-21T04:15:21.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/428: chown d2/d50/d56/f51 60420655 1 2024-09-21T04:15:21.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/429: truncate d2/d1a/d1f/d5f/f6f 93865 0 2024-09-21T04:15:21.430 INFO:tasks.workunit.client.0.smithi110.stdout:4/721: getdents d17/d34/d3d/d20/dca 0 2024-09-21T04:15:21.431 INFO:tasks.workunit.client.1.smithi148.stdout:0/661: symlink d5/d6/d2a/d3f/d6d/da3/dcc/ld4 0 2024-09-21T04:15:21.432 INFO:tasks.workunit.client.1.smithi148.stdout:0/662: readlink d5/l25 0 2024-09-21T04:15:21.432 INFO:tasks.workunit.client.0.smithi110.stdout:6/647: getdents d4/d86/d8f 0 2024-09-21T04:15:21.432 INFO:tasks.workunit.client.0.smithi110.stdout:6/648: chown d4/d12/d17/d9a/dba 6 1 2024-09-21T04:15:21.432 INFO:tasks.workunit.client.0.smithi110.stdout:6/649: chown d4/d78/daa/lb5 8155 1 2024-09-21T04:15:21.432 INFO:tasks.workunit.client.0.smithi110.stdout:2/430: dread d2/f3b [0,4194304] 0 2024-09-21T04:15:21.433 INFO:tasks.workunit.client.1.smithi148.stdout:3/619: symlink d3/d1f/d4d/d82/d52/lcb 0 2024-09-21T04:15:21.434 INFO:tasks.workunit.client.0.smithi110.stdout:1/904: mkdir d2/d9/d122 0 2024-09-21T04:15:21.435 INFO:tasks.workunit.client.1.smithi148.stdout:5/668: creat d3/fe7 x:0 0 0 2024-09-21T04:15:21.436 INFO:tasks.workunit.client.0.smithi110.stdout:6/650: mknod d4/d12/d17/d81/dbf/cd0 0 2024-09-21T04:15:21.438 INFO:tasks.workunit.client.0.smithi110.stdout:2/431: mkdir d2/d81 0 2024-09-21T04:15:21.438 INFO:tasks.workunit.client.1.smithi148.stdout:7/638: rmdir d5/d8/d64 39 2024-09-21T04:15:21.441 INFO:tasks.workunit.client.1.smithi148.stdout:6/660: rename d0/d8/le to d0/d8/d18/lc5 0 2024-09-21T04:15:21.443 INFO:tasks.workunit.client.0.smithi110.stdout:2/432: mknod d2/d1a/d1f/d2a/c82 0 2024-09-21T04:15:21.445 INFO:tasks.workunit.client.1.smithi148.stdout:4/709: getdents d4/d87/d2d 0 2024-09-21T04:15:21.447 INFO:tasks.workunit.client.1.smithi148.stdout:2/786: mkdir d0/d2b/da3/d8f/d9f/df5 0 2024-09-21T04:15:21.447 INFO:tasks.workunit.client.1.smithi148.stdout:2/787: chown d0/d2f 5826 1 2024-09-21T04:15:21.447 INFO:tasks.workunit.client.0.smithi110.stdout:4/722: write d17/f2a [3113201,58621] 0 2024-09-21T04:15:21.448 INFO:tasks.workunit.client.1.smithi148.stdout:0/663: symlink d5/d6/d2a/d2b/d84/daf/dc8/ld5 0 2024-09-21T04:15:21.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/620: rmdir d3/d1f/d4d/d82/d23 39 2024-09-21T04:15:21.452 INFO:tasks.workunit.client.1.smithi148.stdout:7/639: mknod d5/daa/ce4 0 2024-09-21T04:15:21.453 INFO:tasks.workunit.client.1.smithi148.stdout:4/710: rename d4/d87/d91/lb5 to d4/d66/d94/ld7 0 2024-09-21T04:15:21.455 INFO:tasks.workunit.client.0.smithi110.stdout:2/433: dread d2/d1a/d1f/d29/f45 [0,4194304] 0 2024-09-21T04:15:21.457 INFO:tasks.workunit.client.1.smithi148.stdout:2/788: mkdir d0/d2b/da3/d8f/d9f/df6 0 2024-09-21T04:15:21.459 INFO:tasks.workunit.client.0.smithi110.stdout:2/434: symlink d2/d1a/d60/l83 0 2024-09-21T04:15:21.460 INFO:tasks.workunit.client.1.smithi148.stdout:7/640: rmdir d5/d8/d6f/d7d/d8a 39 2024-09-21T04:15:21.461 INFO:tasks.workunit.client.1.smithi148.stdout:4/711: link d4/f10 d4/fd8 0 2024-09-21T04:15:21.461 INFO:tasks.workunit.client.0.smithi110.stdout:1/905: dread d2/d9/d26/f4d [0,4194304] 0 2024-09-21T04:15:21.463 INFO:tasks.workunit.client.0.smithi110.stdout:7/572: dwrite dd/f31 [0,4194304] 0 2024-09-21T04:15:21.463 INFO:tasks.workunit.client.0.smithi110.stdout:2/435: symlink d2/d1a/d1f/d29/d19/l84 0 2024-09-21T04:15:21.463 INFO:tasks.workunit.client.0.smithi110.stdout:1/906: mknod d2/d9/d23/d3b/d9c/c123 0 2024-09-21T04:15:21.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/712: symlink d4/d66/d7f/d9a/ld9 0 2024-09-21T04:15:21.468 INFO:tasks.workunit.client.0.smithi110.stdout:7/573: creat dd/d1c/dad/db2/d56/d66/da6/d98/fb5 x:0 0 0 2024-09-21T04:15:21.468 INFO:tasks.workunit.client.0.smithi110.stdout:7/574: fdatasync dd/d1c/f8c 0 2024-09-21T04:15:21.471 INFO:tasks.workunit.client.0.smithi110.stdout:1/907: mkdir d2/d9/d23/d3b/dca/d107/d124 0 2024-09-21T04:15:21.471 INFO:tasks.workunit.client.1.smithi148.stdout:5/669: dwrite d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:21.472 INFO:tasks.workunit.client.1.smithi148.stdout:5/670: readlink d3/d34/d78/l91 0 2024-09-21T04:15:21.472 INFO:tasks.workunit.client.1.smithi148.stdout:5/671: write d3/d14/d1f/d26/d2e/d56/fe3 [638485,96787] 0 2024-09-21T04:15:21.472 INFO:tasks.workunit.client.1.smithi148.stdout:8/637: dwrite dc/d13/d39/d6e/dac/faf [0,4194304] 0 2024-09-21T04:15:21.472 INFO:tasks.workunit.client.1.smithi148.stdout:6/661: dread d0/d1a/d23/dc/d68/f51 [0,4194304] 0 2024-09-21T04:15:21.474 INFO:tasks.workunit.client.0.smithi110.stdout:7/575: creat dd/d1c/dad/db2/d56/d66/fb6 x:0 0 0 2024-09-21T04:15:21.475 INFO:tasks.workunit.client.1.smithi148.stdout:5/672: truncate d3/d34/f8d 1324511 0 2024-09-21T04:15:21.476 INFO:tasks.workunit.client.0.smithi110.stdout:6/651: dwrite d4/d78/daa/fb4 [0,4194304] 0 2024-09-21T04:15:21.478 INFO:tasks.workunit.client.0.smithi110.stdout:1/908: symlink d2/d9/d23/d4e/d57/dc9/l125 0 2024-09-21T04:15:21.483 INFO:tasks.workunit.client.1.smithi148.stdout:8/638: creat dc/d13/d3c/d81/d98/dc1/fc2 x:0 0 0 2024-09-21T04:15:21.483 INFO:tasks.workunit.client.1.smithi148.stdout:8/639: write dc/d13/d76/d5d/d78/f9f [128181,105412] 0 2024-09-21T04:15:21.486 INFO:tasks.workunit.client.1.smithi148.stdout:6/662: mkdir d0/d34/d38/dc6 0 2024-09-21T04:15:21.486 INFO:tasks.workunit.client.1.smithi148.stdout:6/663: truncate d0/d1a/d23/d27/d62/d9a/fa3 286765 0 2024-09-21T04:15:21.487 INFO:tasks.workunit.client.0.smithi110.stdout:1/909: mkdir d2/d9/d49/d4f/d87/d126 0 2024-09-21T04:15:21.487 INFO:tasks.workunit.client.0.smithi110.stdout:1/910: getdents d2/df3/dfb/d11c 0 2024-09-21T04:15:21.487 INFO:tasks.workunit.client.0.smithi110.stdout:1/911: write d2/d9/d23/d4e/fe4 [217312,14779] 0 2024-09-21T04:15:21.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/576: read dd/d1c/dad/db2/f4f [2793833,20984] 0 2024-09-21T04:15:21.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/673: mknod d3/d34/ce8 0 2024-09-21T04:15:21.490 INFO:tasks.workunit.client.0.smithi110.stdout:4/723: dwrite d17/d1e/d64/f9d [0,4194304] 0 2024-09-21T04:15:21.490 INFO:tasks.workunit.client.0.smithi110.stdout:4/724: stat d17/f39 0 2024-09-21T04:15:21.490 INFO:tasks.workunit.client.0.smithi110.stdout:1/912: creat d2/d9/d49/d4f/d87/d126/f127 x:0 0 0 2024-09-21T04:15:21.490 INFO:tasks.workunit.client.0.smithi110.stdout:1/913: truncate d2/d9/d23/d3b/dca/f110 1014437 0 2024-09-21T04:15:21.491 INFO:tasks.workunit.client.1.smithi148.stdout:6/664: symlink d0/d8/dba/lc7 0 2024-09-21T04:15:21.492 INFO:tasks.workunit.client.1.smithi148.stdout:3/621: dwrite d3/d84/f95 [0,4194304] 0 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:3/498: sync 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/910: sync 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:3/499: read d0/d12/d16/f25 [661611,112002] 0 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/911: write d4/dd/d60/de5/f10b [24655,46645] 0 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/577: creat dd/d48/fb7 x:0 0 0 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/578: fsync dd/d4d/fa1 0 2024-09-21T04:15:21.493 INFO:tasks.workunit.client.1.smithi148.stdout:5/674: mknod d3/d34/d78/dbf/dc2/ce9 0 2024-09-21T04:15:21.494 INFO:tasks.workunit.client.0.smithi110.stdout:6/652: dread d4/d12/d16/d66/d7a/d83/f9f [0,4194304] 0 2024-09-21T04:15:21.496 INFO:tasks.workunit.client.1.smithi148.stdout:7/641: dwrite d5/d8/d6f/d7d/fdc [0,4194304] 0 2024-09-21T04:15:21.497 INFO:tasks.workunit.client.1.smithi148.stdout:8/640: mknod dc/d13/d39/cc3 0 2024-09-21T04:15:21.497 INFO:tasks.workunit.client.1.smithi148.stdout:0/664: dwrite d5/d6/d2a/d3f/d8b/dab/fc9 [0,4194304] 0 2024-09-21T04:15:21.497 INFO:tasks.workunit.client.1.smithi148.stdout:0/665: dread - d5/d6/d2a/d2b/fd3 zero size 2024-09-21T04:15:21.499 INFO:tasks.workunit.client.1.smithi148.stdout:3/622: dread d3/d5/d6/d12/d43/d54/f71 [0,4194304] 0 2024-09-21T04:15:21.500 INFO:tasks.workunit.client.0.smithi110.stdout:2/436: getdents d2/d1a/d1f/d65 0 2024-09-21T04:15:21.501 INFO:tasks.workunit.client.1.smithi148.stdout:4/713: dwrite d4/d3a/f53 [0,4194304] 0 2024-09-21T04:15:21.501 INFO:tasks.workunit.client.1.smithi148.stdout:4/714: readlink d4/d66/d94/ld7 0 2024-09-21T04:15:21.503 INFO:tasks.workunit.client.1.smithi148.stdout:2/789: dwrite d0/d9/d16/d55/d71/fa5 [0,4194304] 0 2024-09-21T04:15:21.504 INFO:tasks.workunit.client.1.smithi148.stdout:5/675: rename d3/d14/d1f/d26/c46 to d3/d14/d1f/d72/dc9/cea 0 2024-09-21T04:15:21.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/642: creat d5/d14/d25/d40/dab/dad/fe5 x:0 0 0 2024-09-21T04:15:21.510 INFO:tasks.workunit.client.1.smithi148.stdout:8/641: unlink dc/f31 0 2024-09-21T04:15:21.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/725: creat d17/d1e/d21/de1/ff1 x:0 0 0 2024-09-21T04:15:21.513 INFO:tasks.workunit.client.0.smithi110.stdout:1/914: mknod d2/d9/d23/d4e/d57/dc9/c128 0 2024-09-21T04:15:21.513 INFO:tasks.workunit.client.1.smithi148.stdout:5/676: dread d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:21.513 INFO:tasks.workunit.client.1.smithi148.stdout:5/677: write d3/d34/d78/fa1 [146219,1195] 0 2024-09-21T04:15:21.514 INFO:tasks.workunit.client.1.smithi148.stdout:5/678: write d3/d14/d1f/d26/d2e/d3d/fa2 [101751,22014] 0 2024-09-21T04:15:21.515 INFO:tasks.workunit.client.0.smithi110.stdout:3/500: mkdir d0/d12/d16/d1a/dac 0 2024-09-21T04:15:21.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/715: read d4/d12/d16/d29/f3d [7985359,106802] 0 2024-09-21T04:15:21.516 INFO:tasks.workunit.client.1.smithi148.stdout:0/666: mknod d5/d6/d7/dd0/cd6 0 2024-09-21T04:15:21.517 INFO:tasks.workunit.client.0.smithi110.stdout:0/912: mkdir d4/d1c/d62/d91/dd6/d114 0 2024-09-21T04:15:21.518 INFO:tasks.workunit.client.1.smithi148.stdout:3/623: mknod d3/d1f/d4d/d82/d23/d28/d65/ccc 0 2024-09-21T04:15:21.520 INFO:tasks.workunit.client.0.smithi110.stdout:7/579: mknod dd/d1c/cb8 0 2024-09-21T04:15:21.521 INFO:tasks.workunit.client.1.smithi148.stdout:5/679: write d3/d37/f7f [637622,109353] 0 2024-09-21T04:15:21.521 INFO:tasks.workunit.client.1.smithi148.stdout:6/665: dwrite d0/d1a/db8/dbe/fc3 [0,4194304] 0 2024-09-21T04:15:21.521 INFO:tasks.workunit.client.1.smithi148.stdout:6/666: chown d0/d1a/fab 0 1 2024-09-21T04:15:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:2/437: rename d2/d1a/d1f/d29/c5a to d2/d81/c85 0 2024-09-21T04:15:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:2/438: stat d2/d1a/c5e 0 2024-09-21T04:15:21.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/726: getdents d17/d34/d3d/d20/d43/d51/d7a 0 2024-09-21T04:15:21.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/727: fsync d17/d34/d3d/fa6 0 2024-09-21T04:15:21.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/790: mkdir d0/d9/d16/d55/d71/dc4/df7 0 2024-09-21T04:15:21.530 INFO:tasks.workunit.client.0.smithi110.stdout:1/915: mknod d2/d9/d26/d3d/d8f/de7/c129 0 2024-09-21T04:15:21.532 INFO:tasks.workunit.client.0.smithi110.stdout:2/439: write d2/d1a/d1f/d29/f4c [3295197,16625] 0 2024-09-21T04:15:21.533 INFO:tasks.workunit.client.1.smithi148.stdout:7/643: rename d5/d8/d6f/d7d/fac to d5/d8/d6f/dcd/fe6 0 2024-09-21T04:15:21.533 INFO:tasks.workunit.client.1.smithi148.stdout:7/644: read - d5/d14/d25/d40/db9/fd1 zero size 2024-09-21T04:15:21.533 INFO:tasks.workunit.client.0.smithi110.stdout:7/580: dread dd/d1c/dad/db2/d4a/d52/f60 [0,4194304] 0 2024-09-21T04:15:21.533 INFO:tasks.workunit.client.0.smithi110.stdout:3/501: mkdir d0/d12/d3c/dad 0 2024-09-21T04:15:21.534 INFO:tasks.workunit.client.0.smithi110.stdout:0/913: symlink d4/d1c/l115 0 2024-09-21T04:15:21.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/667: creat d5/d6/d2a/fd7 x:0 0 0 2024-09-21T04:15:21.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/668: chown d5/d82/l94 196315 1 2024-09-21T04:15:21.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/624: link d3/d5/d6/d2b/l64 d3/d5/dac/lcd 0 2024-09-21T04:15:21.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/728: link d17/d34/d3d/fa6 d17/d3e/dda/ff2 0 2024-09-21T04:15:21.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/729: write d17/d1e/f4c [883398,111065] 0 2024-09-21T04:15:21.540 INFO:tasks.workunit.client.1.smithi148.stdout:3/625: dread d3/d1f/d4d/d82/d23/f20 [0,4194304] 0 2024-09-21T04:15:21.541 INFO:tasks.workunit.client.1.smithi148.stdout:5/680: mkdir d3/d14/d1f/d26/d2e/ddb/deb 0 2024-09-21T04:15:21.542 INFO:tasks.workunit.client.1.smithi148.stdout:8/642: dwrite dc/d13/d3c/d81/d85/da1/fb8 [0,4194304] 0 2024-09-21T04:15:21.544 INFO:tasks.workunit.client.0.smithi110.stdout:1/916: dread d2/d9/d23/d3b/d9c/fa4 [0,4194304] 0 2024-09-21T04:15:21.545 INFO:tasks.workunit.client.1.smithi148.stdout:4/716: dwrite d4/d87/f5c [0,4194304] 0 2024-09-21T04:15:21.548 INFO:tasks.workunit.client.0.smithi110.stdout:2/440: mkdir d2/d1a/d1f/d29/d19/d62/d86 0 2024-09-21T04:15:21.548 INFO:tasks.workunit.client.0.smithi110.stdout:7/581: truncate dd/f1a 3083178 0 2024-09-21T04:15:21.548 INFO:tasks.workunit.client.0.smithi110.stdout:7/582: write dd/d1c/dad/db2/d56/d66/fb6 [647472,38190] 0 2024-09-21T04:15:21.548 INFO:tasks.workunit.client.0.smithi110.stdout:7/583: creat dd/d14/d17/fb9 x:0 0 0 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/667: mkdir d0/d1a/d23/d32/dc8 0 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/668: write d0/d34/d97/f52 [935983,90485] 0 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/669: dread - d0/d8/d18/fb7 zero size 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/670: truncate d0/d1a/d23/d27/f95 767640 0 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/671: write d0/d8/d18/f4d [4661264,75502] 0 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/672: chown d0/d1a/d23/d27/f95 813 1 2024-09-21T04:15:21.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/673: fsync d0/d1a/d23/d27/d62/d9a/f7a 0 2024-09-21T04:15:21.550 INFO:tasks.workunit.client.0.smithi110.stdout:3/502: creat d0/d12/d3c/dad/fae x:0 0 0 2024-09-21T04:15:21.551 INFO:tasks.workunit.client.1.smithi148.stdout:2/791: stat d0/d9/d16/c22 0 2024-09-21T04:15:21.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/914: symlink d4/dd/d10/d8a/db5/dd7/l116 0 2024-09-21T04:15:21.553 INFO:tasks.workunit.client.1.smithi148.stdout:7/645: mknod d5/d8/d64/ce7 0 2024-09-21T04:15:21.557 INFO:tasks.workunit.client.1.smithi148.stdout:0/669: rename d5/f9a to d5/fd8 0 2024-09-21T04:15:21.558 INFO:tasks.workunit.client.0.smithi110.stdout:1/917: creat d2/d9/d23/d120/f12a x:0 0 0 2024-09-21T04:15:21.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/626: creat d3/d1f/d4d/d82/d23/d28/d65/fce x:0 0 0 2024-09-21T04:15:21.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/627: truncate d3/d1f/d4d/d82/d23/fba 832252 0 2024-09-21T04:15:21.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/674: dread d0/d1a/d23/d27/d80/fb2 [0,4194304] 0 2024-09-21T04:15:21.560 INFO:tasks.workunit.client.0.smithi110.stdout:7/584: symlink dd/d1c/dad/db2/d4a/d52/lba 0 2024-09-21T04:15:21.561 INFO:tasks.workunit.client.0.smithi110.stdout:2/441: dread d2/f16 [0,4194304] 0 2024-09-21T04:15:21.562 INFO:tasks.workunit.client.0.smithi110.stdout:3/503: link d0/d12/c15 d0/d12/d3c/d5a/d70/dab/caf 0 2024-09-21T04:15:21.562 INFO:tasks.workunit.client.0.smithi110.stdout:3/504: stat d0/d12/d2a 0 2024-09-21T04:15:21.563 INFO:tasks.workunit.client.0.smithi110.stdout:0/915: symlink d4/d1c/l117 0 2024-09-21T04:15:21.565 INFO:tasks.workunit.client.1.smithi148.stdout:4/717: unlink d4/d12/da7/lc0 0 2024-09-21T04:15:21.565 INFO:tasks.workunit.client.0.smithi110.stdout:1/918: truncate d2/d9/d23/de0/f9e 581827 0 2024-09-21T04:15:21.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/585: creat dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/fbb x:0 0 0 2024-09-21T04:15:21.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/586: dread - dd/d1c/dad/db2/d56/fac zero size 2024-09-21T04:15:21.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/587: readlink dd/d1c/dad/db2/d4a/l79 0 2024-09-21T04:15:21.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/588: write dd/d1c/dad/db2/d56/d66/da6/d7a/f97 [773654,105743] 0 2024-09-21T04:15:21.570 INFO:tasks.workunit.client.0.smithi110.stdout:2/442: truncate d2/d1a/d1f/d29/f71 388470 0 2024-09-21T04:15:21.570 INFO:tasks.workunit.client.0.smithi110.stdout:2/443: chown d2/d50/d56/c33 0 1 2024-09-21T04:15:21.571 INFO:tasks.workunit.client.0.smithi110.stdout:3/505: mkdir d0/d12/d2a/d80/da2/db0 0 2024-09-21T04:15:21.572 INFO:tasks.workunit.client.0.smithi110.stdout:3/506: stat d0/d22/d29/f8b 0 2024-09-21T04:15:21.572 INFO:tasks.workunit.client.0.smithi110.stdout:5/619: sync 2024-09-21T04:15:21.572 INFO:tasks.workunit.client.0.smithi110.stdout:9/543: sync 2024-09-21T04:15:21.572 INFO:tasks.workunit.client.0.smithi110.stdout:8/600: sync 2024-09-21T04:15:21.573 INFO:tasks.workunit.client.0.smithi110.stdout:9/544: stat d7/d96/d99/d3c/d1f/d72/c83 0 2024-09-21T04:15:21.573 INFO:tasks.workunit.client.0.smithi110.stdout:5/620: creat d1/d49/d6f/fd0 x:0 0 0 2024-09-21T04:15:21.573 INFO:tasks.workunit.client.0.smithi110.stdout:5/621: creat d1/d22/d51/fd1 x:0 0 0 2024-09-21T04:15:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/646: symlink d5/daa/dc6/le8 0 2024-09-21T04:15:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/647: write d5/d8/d6f/d7d/d90/f50 [710020,33333] 0 2024-09-21T04:15:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/648: chown d5/d14/d25/d40/ldb 10518 1 2024-09-21T04:15:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/649: write d5/d8/d6f/d7d/f7f [5235783,56675] 0 2024-09-21T04:15:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/650: truncate d5/daa/fc5 642520 0 2024-09-21T04:15:21.575 INFO:tasks.workunit.client.0.smithi110.stdout:1/919: creat d2/d9/d49/d4f/f12b x:0 0 0 2024-09-21T04:15:21.576 INFO:tasks.workunit.client.1.smithi148.stdout:3/628: mknod d3/d1f/d4d/ccf 0 2024-09-21T04:15:21.576 INFO:tasks.workunit.client.1.smithi148.stdout:3/629: write d3/d5/d6/d12/d43/d54/f71 [1604516,30685] 0 2024-09-21T04:15:21.577 INFO:tasks.workunit.client.0.smithi110.stdout:7/589: mkdir dd/d1c/dad/db2/d6d/dbc 0 2024-09-21T04:15:21.578 INFO:tasks.workunit.client.0.smithi110.stdout:3/507: rmdir d0/d12/d3c/d5a/d70 39 2024-09-21T04:15:21.580 INFO:tasks.workunit.client.0.smithi110.stdout:8/601: mknod d0/d30/ccf 0 2024-09-21T04:15:21.580 INFO:tasks.workunit.client.0.smithi110.stdout:8/602: dread - d0/d7/d6f/d86/fc0 zero size 2024-09-21T04:15:21.580 INFO:tasks.workunit.client.0.smithi110.stdout:2/444: write d2/d1a/d1f/d29/f45 [8004078,33639] 0 2024-09-21T04:15:21.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/718: creat d4/d12/d16/d29/d61/fda x:0 0 0 2024-09-21T04:15:21.584 INFO:tasks.workunit.client.1.smithi148.stdout:2/792: mknod d0/d9/d16/d55/d6f/da7/cf8 0 2024-09-21T04:15:21.587 INFO:tasks.workunit.client.0.smithi110.stdout:2/445: dread d2/f44 [0,4194304] 0 2024-09-21T04:15:21.587 INFO:tasks.workunit.client.0.smithi110.stdout:3/508: read d0/fe [1701143,92898] 0 2024-09-21T04:15:21.587 INFO:tasks.workunit.client.0.smithi110.stdout:2/446: stat d2 0 2024-09-21T04:15:21.587 INFO:tasks.workunit.client.0.smithi110.stdout:2/447: readlink d2/l23 0 2024-09-21T04:15:21.588 INFO:tasks.workunit.client.0.smithi110.stdout:3/509: creat d0/d22/d29/d76/fb1 x:0 0 0 2024-09-21T04:15:21.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/630: truncate d3/d5/d6/d12/d43/d54/f6a 1390767 0 2024-09-21T04:15:21.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/590: write dd/d14/f36 [3750102,60187] 0 2024-09-21T04:15:21.589 INFO:tasks.workunit.client.0.smithi110.stdout:1/920: mkdir d2/d9/d49/d7a/d12c 0 2024-09-21T04:15:21.590 INFO:tasks.workunit.client.1.smithi148.stdout:6/675: getdents d0/d1a/d23/d27/d62 0 2024-09-21T04:15:21.591 INFO:tasks.workunit.client.0.smithi110.stdout:8/603: creat d0/d7/d23/d52/fd0 x:0 0 0 2024-09-21T04:15:21.591 INFO:tasks.workunit.client.0.smithi110.stdout:8/604: chown d0/d30/d34/d3e/cc8 25 1 2024-09-21T04:15:21.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/719: symlink d4/d12/d16/d29/dbd/ldb 0 2024-09-21T04:15:21.595 INFO:tasks.workunit.client.1.smithi148.stdout:2/793: creat d0/d9/dc/d13/daf/db2/ff9 x:0 0 0 2024-09-21T04:15:21.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/651: rename d5/d8/f22 to d5/d14/d25/d40/d8b/fe9 0 2024-09-21T04:15:21.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/626: sync 2024-09-21T04:15:21.602 INFO:tasks.workunit.client.1.smithi148.stdout:9/713: sync 2024-09-21T04:15:21.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/720: dread d4/d66/d7f/f4a [0,4194304] 0 2024-09-21T04:15:21.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/721: fdatasync d4/d12/d16/f19 0 2024-09-21T04:15:21.604 INFO:tasks.workunit.client.0.smithi110.stdout:7/591: mkdir dd/d1c/dad/db2/d56/d66/da6/dbd 0 2024-09-21T04:15:21.604 INFO:tasks.workunit.client.0.smithi110.stdout:7/592: chown dd/d1c/dad/db2/d4a/d52/cb3 366397 1 2024-09-21T04:15:21.605 INFO:tasks.workunit.client.0.smithi110.stdout:1/921: unlink d2/d9/d23/d4e/f61 0 2024-09-21T04:15:21.605 INFO:tasks.workunit.client.0.smithi110.stdout:1/922: fsync d2/d9/d23/d3b/d9c/fa4 0 2024-09-21T04:15:21.605 INFO:tasks.workunit.client.0.smithi110.stdout:1/923: creat d2/d9/d49/d4f/d87/f12d x:0 0 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/605: symlink d0/d7/d13/d46/ld1 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/606: fdatasync d0/d7/d13/d4b/f17 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/607: write d0/d30/d34/d3e/d32/f73 [137158,97487] 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/608: fdatasync d0/d30/f58 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.1.smithi148.stdout:6/676: unlink d0/d1a/d23/d32/c5a 0 2024-09-21T04:15:21.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/794: dread d0/d9/d38/f41 [0,4194304] 0 2024-09-21T04:15:21.607 INFO:tasks.workunit.client.1.smithi148.stdout:2/795: truncate d0/d9/d16/d55/d71/dc4/fe2 797986 0 2024-09-21T04:15:21.607 INFO:tasks.workunit.client.0.smithi110.stdout:4/730: rename d17/d3e/dda to d17/d34/d3d/d20/dca/df3 0 2024-09-21T04:15:21.607 INFO:tasks.workunit.client.1.smithi148.stdout:3/631: dwrite d3/d1f/d4d/d82/d74/f91 [0,4194304] 0 2024-09-21T04:15:21.608 INFO:tasks.workunit.client.0.smithi110.stdout:1/924: creat d2/d9/d122/f12e x:0 0 0 2024-09-21T04:15:21.608 INFO:tasks.workunit.client.0.smithi110.stdout:1/925: fdatasync d2/d9/d26/d3d/d8f/f90 0 2024-09-21T04:15:21.609 INFO:tasks.workunit.client.1.smithi148.stdout:9/714: creat d5/d60/de1/f101 x:0 0 0 2024-09-21T04:15:21.610 INFO:tasks.workunit.client.0.smithi110.stdout:7/593: dread dd/d1c/dad/db2/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:21.610 INFO:tasks.workunit.client.0.smithi110.stdout:8/609: getdents d0/d7/d23 0 2024-09-21T04:15:21.610 INFO:tasks.workunit.client.0.smithi110.stdout:8/610: readlink d0/l6 0 2024-09-21T04:15:21.611 INFO:tasks.workunit.client.1.smithi148.stdout:1/627: creat db/d24/fcb x:0 0 0 2024-09-21T04:15:21.611 INFO:tasks.workunit.client.1.smithi148.stdout:1/628: stat db/dd/l3f 0 2024-09-21T04:15:21.612 INFO:tasks.workunit.client.0.smithi110.stdout:4/731: creat d17/d34/d3d/d20/d43/ff4 x:0 0 0 2024-09-21T04:15:21.612 INFO:tasks.workunit.client.1.smithi148.stdout:9/715: dread d5/d1e/f65 [0,4194304] 0 2024-09-21T04:15:21.613 INFO:tasks.workunit.client.1.smithi148.stdout:9/716: read - d5/d60/d28/d73/fcd zero size 2024-09-21T04:15:21.613 INFO:tasks.workunit.client.1.smithi148.stdout:4/722: symlink d4/d66/d7f/d2a/ldc 0 2024-09-21T04:15:21.615 INFO:tasks.workunit.client.0.smithi110.stdout:1/926: creat d2/d9/d23/d3b/dca/d107/d124/f12f x:0 0 0 2024-09-21T04:15:21.616 INFO:tasks.workunit.client.1.smithi148.stdout:6/677: mkdir d0/d34/d38/dc6/dc9 0 2024-09-21T04:15:21.617 INFO:tasks.workunit.client.0.smithi110.stdout:7/594: unlink dd/d1c/dad/db2/d56/d66/da6/d7a/d82/laa 0 2024-09-21T04:15:21.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/796: rmdir d0/d2b/da3/d95 39 2024-09-21T04:15:21.620 INFO:tasks.workunit.client.0.smithi110.stdout:8/611: creat d0/d7/d13/d46/fd2 x:0 0 0 2024-09-21T04:15:21.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/670: dread d5/d6/d2a/d2b/d84/db4/fb5 [0,4194304] 0 2024-09-21T04:15:21.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/632: link d3/d5/d6/d12/d15/d73/f7c d3/d1f/d4d/d82/d23/db5/fd0 0 2024-09-21T04:15:21.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/633: truncate d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 947030 0 2024-09-21T04:15:21.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/634: chown d3/l45 466135 1 2024-09-21T04:15:21.622 INFO:tasks.workunit.client.0.smithi110.stdout:4/732: mkdir d17/d34/df5 0 2024-09-21T04:15:21.624 INFO:tasks.workunit.client.1.smithi148.stdout:1/629: mknod db/dd/d21/d89/ccc 0 2024-09-21T04:15:21.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/723: unlink d4/d12/d78/fd2 0 2024-09-21T04:15:21.629 INFO:tasks.workunit.client.0.smithi110.stdout:7/595: getdents dd/d1c/dad/db2/d4a/d52 0 2024-09-21T04:15:21.632 INFO:tasks.workunit.client.1.smithi148.stdout:6/678: unlink d0/d1a/c8b 0 2024-09-21T04:15:21.633 INFO:tasks.workunit.client.0.smithi110.stdout:3/510: dread d0/d12/d16/d1a/f35 [0,4194304] 0 2024-09-21T04:15:21.636 INFO:tasks.workunit.client.0.smithi110.stdout:8/612: symlink d0/d7/d23/ld3 0 2024-09-21T04:15:21.636 INFO:tasks.workunit.client.0.smithi110.stdout:8/613: chown d0/d7/d13/d1c/f1d 69854154 1 2024-09-21T04:15:21.636 INFO:tasks.workunit.client.0.smithi110.stdout:8/614: write d0/d7/d23/f69 [262944,100259] 0 2024-09-21T04:15:21.637 INFO:tasks.workunit.client.0.smithi110.stdout:9/545: dread d7/d96/d99/d3c/f20 [0,4194304] 0 2024-09-21T04:15:21.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/916: dwrite d4/dd/fcb [0,4194304] 0 2024-09-21T04:15:21.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/917: fsync d4/dd/d53/d66/fc1 0 2024-09-21T04:15:21.642 INFO:tasks.workunit.client.1.smithi148.stdout:4/724: read d4/d12/d83/fa6 [231622,52348] 0 2024-09-21T04:15:21.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/652: dwrite d5/d14/d25/d40/d7b/f84 [4194304,4194304] 0 2024-09-21T04:15:21.643 INFO:tasks.workunit.client.0.smithi110.stdout:8/615: rename d0/d7/d23/d52/cc9 to d0/d30/d34/d3e/d5d/cd4 0 2024-09-21T04:15:21.643 INFO:tasks.workunit.client.0.smithi110.stdout:8/616: read d0/d30/d34/d3e/d32/f73 [7569,28503] 0 2024-09-21T04:15:21.644 INFO:tasks.workunit.client.0.smithi110.stdout:8/617: stat d0/d30/d34/d3e/d32/fb9 0 2024-09-21T04:15:21.644 INFO:tasks.workunit.client.0.smithi110.stdout:8/618: stat d0/d7/c7a 0 2024-09-21T04:15:21.644 INFO:tasks.workunit.client.0.smithi110.stdout:2/448: truncate d2/d1a/d1f/d29/f71 467951 0 2024-09-21T04:15:21.645 INFO:tasks.workunit.client.0.smithi110.stdout:2/449: write d2/d1a/d1f/d5f/f6f [202377,121625] 0 2024-09-21T04:15:21.645 INFO:tasks.workunit.client.0.smithi110.stdout:3/511: dread d0/d12/d2a/f4d [0,4194304] 0 2024-09-21T04:15:21.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/681: truncate d3/d24/f43 544613 0 2024-09-21T04:15:21.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/682: write d3/d34/d3c/d5d/d75/fe2 [136453,69275] 0 2024-09-21T04:15:21.649 INFO:tasks.workunit.client.0.smithi110.stdout:4/733: dwrite d17/d1e/d7b/f93 [0,4194304] 0 2024-09-21T04:15:21.654 INFO:tasks.workunit.client.1.smithi148.stdout:2/797: creat d0/d2b/da3/da1/dcd/ffa x:0 0 0 2024-09-21T04:15:21.654 INFO:tasks.workunit.client.1.smithi148.stdout:2/798: fsync d0/d9/d16/d55/d71/dc2/dcb/fd8 0 2024-09-21T04:15:21.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/671: creat d5/d6/d2a/d3f/d6d/fd9 x:0 0 0 2024-09-21T04:15:21.657 INFO:tasks.workunit.client.0.smithi110.stdout:8/619: creat d0/d30/d9d/fd5 x:0 0 0 2024-09-21T04:15:21.658 INFO:tasks.workunit.client.0.smithi110.stdout:8/620: getdents d0/d7/d13/dc5 0 2024-09-21T04:15:21.658 INFO:tasks.workunit.client.0.smithi110.stdout:8/621: chown d0/d7/d13/f5a 0 1 2024-09-21T04:15:21.658 INFO:tasks.workunit.client.0.smithi110.stdout:8/622: write d0/d7/d6f/d86/fc0 [412297,17594] 0 2024-09-21T04:15:21.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/635: symlink d3/d5/d6/d12/d43/ld1 0 2024-09-21T04:15:21.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/636: creat d3/d1f/d4d/d82/d21/fd2 x:0 0 0 2024-09-21T04:15:21.659 INFO:tasks.workunit.client.1.smithi148.stdout:1/630: creat db/dd/d21/d89/dc9/fcd x:0 0 0 2024-09-21T04:15:21.660 INFO:tasks.workunit.client.0.smithi110.stdout:2/450: symlink d2/d50/d7b/l87 0 2024-09-21T04:15:21.663 INFO:tasks.workunit.client.1.smithi148.stdout:5/683: write d3/d34/f99 [3207433,93180] 0 2024-09-21T04:15:21.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/725: symlink d4/d66/d7f/d9a/dc2/ldd 0 2024-09-21T04:15:21.666 INFO:tasks.workunit.client.1.smithi148.stdout:4/726: creat d4/d12/d75/d9c/fde x:0 0 0 2024-09-21T04:15:21.666 INFO:tasks.workunit.client.0.smithi110.stdout:4/734: symlink d17/d34/d3d/d57/db9/lf6 0 2024-09-21T04:15:21.669 INFO:tasks.workunit.client.0.smithi110.stdout:8/623: mknod d0/d7/d23/d52/cd6 0 2024-09-21T04:15:21.669 INFO:tasks.workunit.client.0.smithi110.stdout:8/624: chown d0/d7/d13/d1c 4 1 2024-09-21T04:15:21.669 INFO:tasks.workunit.client.0.smithi110.stdout:8/625: stat d0/d30/d54 0 2024-09-21T04:15:21.669 INFO:tasks.workunit.client.0.smithi110.stdout:8/626: readlink d0/d30/d34/d3e/d32/l3f 0 2024-09-21T04:15:21.671 INFO:tasks.workunit.client.0.smithi110.stdout:8/627: rmdir d0 39 2024-09-21T04:15:21.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/799: creat d0/d9/d16/ffb x:0 0 0 2024-09-21T04:15:21.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/800: creat d0/d9/d16/d55/ffc x:0 0 0 2024-09-21T04:15:21.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/801: truncate d0/d9/d16/d55/d6f/f7e 908102 0 2024-09-21T04:15:21.673 INFO:tasks.workunit.client.0.smithi110.stdout:8/628: mkdir d0/d30/d34/d3e/d5d/dd7 0 2024-09-21T04:15:21.675 INFO:tasks.workunit.client.1.smithi148.stdout:0/672: rename d5/d6/d7/d11/fbc to d5/d6/d2a/d3f/d6d/da3/fda 0 2024-09-21T04:15:21.675 INFO:tasks.workunit.client.1.smithi148.stdout:0/673: fsync d5/d6/d2a/f79 0 2024-09-21T04:15:21.675 INFO:tasks.workunit.client.1.smithi148.stdout:6/679: dwrite d0/d8/d18/f25 [4194304,4194304] 0 2024-09-21T04:15:21.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/637: truncate d3/d5/d6/d2b/da2/da3/fc5 180684 0 2024-09-21T04:15:21.678 INFO:tasks.workunit.client.1.smithi148.stdout:5/684: read d3/d14/d1a/f9a [2946892,110170] 0 2024-09-21T04:15:21.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/631: symlink db/dd/d21/d89/lce 0 2024-09-21T04:15:21.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/632: truncate db/d24/d2a/da4/fbc 565960 0 2024-09-21T04:15:21.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/633: stat db/d24/d2e/d71/d94/d9a/c8a 0 2024-09-21T04:15:21.682 INFO:tasks.workunit.client.1.smithi148.stdout:4/727: creat d4/d12/d75/db4/fdf x:0 0 0 2024-09-21T04:15:21.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/653: link d5/d8/d6f/dcd/fe6 d5/d8/fea 0 2024-09-21T04:15:21.683 INFO:tasks.workunit.client.1.smithi148.stdout:7/654: chown d5/d8/d6f/d7d/d90/d28/f87 25510973 1 2024-09-21T04:15:21.685 INFO:tasks.workunit.client.1.smithi148.stdout:0/674: symlink d5/ldb 0 2024-09-21T04:15:21.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/680: creat d0/d1a/d23/d32/d93/fca x:0 0 0 2024-09-21T04:15:21.686 INFO:tasks.workunit.client.1.smithi148.stdout:5/685: write d3/d14/d1f/d26/d2e/f8f [88483,122678] 0 2024-09-21T04:15:21.687 INFO:tasks.workunit.client.0.smithi110.stdout:4/735: dread d17/d34/d3d/d20/d43/d51/d52/d98/f99 [0,4194304] 0 2024-09-21T04:15:21.687 INFO:tasks.workunit.client.1.smithi148.stdout:0/675: link d5/d6/d7/d11/fcf d5/d6/fdc 0 2024-09-21T04:15:21.688 INFO:tasks.workunit.client.1.smithi148.stdout:0/676: fdatasync d5/d6/fdc 0 2024-09-21T04:15:21.688 INFO:tasks.workunit.client.1.smithi148.stdout:0/677: write d5/d6/d7/d11/d7b/fb8 [545460,62280] 0 2024-09-21T04:15:21.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/736: unlink d17/d34/d3d/d20/l25 0 2024-09-21T04:15:21.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/737: truncate d17/d3e/dbd/fe6 296929 0 2024-09-21T04:15:21.692 INFO:tasks.workunit.client.1.smithi148.stdout:5/686: rename d3/d14/d1f/f23 to d3/d34/d3c/d5d/d75/fec 0 2024-09-21T04:15:21.693 INFO:tasks.workunit.client.1.smithi148.stdout:0/678: mknod d5/d6/d7/d11/d7b/cdd 0 2024-09-21T04:15:21.695 INFO:tasks.workunit.client.0.smithi110.stdout:1/927: rmdir d2/d9/d23/d4e 39 2024-09-21T04:15:21.696 INFO:tasks.workunit.client.0.smithi110.stdout:1/928: stat d2/d9/d26/d3d/d8f/fbc 0 2024-09-21T04:15:21.696 INFO:tasks.workunit.client.0.smithi110.stdout:1/929: chown d2/d9/d49/d4f/d87 4 1 2024-09-21T04:15:21.696 INFO:tasks.workunit.client.0.smithi110.stdout:1/930: getdents d2/d9/d49/d7a/da8 0 2024-09-21T04:15:21.696 INFO:tasks.workunit.client.1.smithi148.stdout:6/681: dread d0/d34/d38/d43/f7b [0,4194304] 0 2024-09-21T04:15:21.698 INFO:tasks.workunit.client.0.smithi110.stdout:1/931: symlink d2/d9/d23/d4e/d57/dc9/l130 0 2024-09-21T04:15:21.698 INFO:tasks.workunit.client.0.smithi110.stdout:1/932: chown d2/d9/l111 1886842902 1 2024-09-21T04:15:21.699 INFO:tasks.workunit.client.1.smithi148.stdout:6/682: creat d0/d1a/d23/d27/d62/fcb x:0 0 0 2024-09-21T04:15:21.700 INFO:tasks.workunit.client.0.smithi110.stdout:1/933: getdents d2/d9/d26/d3d/d8f 0 2024-09-21T04:15:21.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/802: dread d0/d2b/f76 [0,4194304] 0 2024-09-21T04:15:21.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/803: creat d0/d9/d16/d55/ffd x:0 0 0 2024-09-21T04:15:21.708 INFO:tasks.workunit.client.1.smithi148.stdout:7/655: dread d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:21.710 INFO:tasks.workunit.client.1.smithi148.stdout:2/804: mknod d0/d9/d59/dda/cfe 0 2024-09-21T04:15:21.712 INFO:tasks.workunit.client.1.smithi148.stdout:2/805: creat d0/fff x:0 0 0 2024-09-21T04:15:21.715 INFO:tasks.workunit.client.1.smithi148.stdout:2/806: mkdir d0/d9/d100 0 2024-09-21T04:15:21.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/807: fsync d0/d2b/f4b 0 2024-09-21T04:15:21.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/808: write d0/d9/f17 [3896029,82094] 0 2024-09-21T04:15:21.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/809: write d0/f23 [2900031,95009] 0 2024-09-21T04:15:21.718 INFO:tasks.workunit.client.0.smithi110.stdout:7/596: dwrite dd/d1c/dad/db2/f95 [0,4194304] 0 2024-09-21T04:15:21.719 INFO:tasks.workunit.client.0.smithi110.stdout:9/546: dwrite d7/d96/d99/d3c/d14/d31/f6a [0,4194304] 0 2024-09-21T04:15:21.720 INFO:tasks.workunit.client.1.smithi148.stdout:2/810: rmdir d0/d9/dc/d13/d50 39 2024-09-21T04:15:21.727 INFO:tasks.workunit.client.1.smithi148.stdout:5/687: dwrite d3/d34/d3c/d5d/fcd [0,4194304] 0 2024-09-21T04:15:21.729 INFO:tasks.workunit.client.1.smithi148.stdout:7/656: write d5/d8/d6f/dcd/fe6 [786801,103475] 0 2024-09-21T04:15:21.729 INFO:tasks.workunit.client.1.smithi148.stdout:7/657: readlink d5/d8/d64/la6 0 2024-09-21T04:15:21.730 INFO:tasks.workunit.client.1.smithi148.stdout:7/658: chown d5/d14/l65 354722 1 2024-09-21T04:15:21.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/688: unlink d3/d37/c3e 0 2024-09-21T04:15:21.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/689: creat d3/d37/fed x:0 0 0 2024-09-21T04:15:21.730 INFO:tasks.workunit.client.1.smithi148.stdout:5/690: creat d3/d14/d1a/fee x:0 0 0 2024-09-21T04:15:21.734 INFO:tasks.workunit.client.1.smithi148.stdout:5/691: symlink d3/d14/d1f/db1/db8/dba/lef 0 2024-09-21T04:15:21.734 INFO:tasks.workunit.client.1.smithi148.stdout:7/659: getdents d5/d8/d6f/d7d/d90 0 2024-09-21T04:15:21.735 INFO:tasks.workunit.client.1.smithi148.stdout:2/811: dread d0/d2b/d48/f58 [0,4194304] 0 2024-09-21T04:15:21.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/597: write dd/d1c/dad/db2/d56/f94 [1434240,43253] 0 2024-09-21T04:15:21.736 INFO:tasks.workunit.client.1.smithi148.stdout:5/692: rename d3/d14/d1f/f42 to d3/d14/d1f/ff0 0 2024-09-21T04:15:21.736 INFO:tasks.workunit.client.1.smithi148.stdout:7/660: creat d5/d8/d64/feb x:0 0 0 2024-09-21T04:15:21.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/728: dwrite d4/d12/d16/d29/f46 [0,4194304] 0 2024-09-21T04:15:21.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/729: read - d4/d66/fbe zero size 2024-09-21T04:15:21.737 INFO:tasks.workunit.client.0.smithi110.stdout:8/629: dwrite d0/d7/d13/d4b/f7d [4194304,4194304] 0 2024-09-21T04:15:21.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/812: symlink d0/d9/d16/d9e/l101 0 2024-09-21T04:15:21.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/813: truncate d0/d9/d16/d55/d71/dc2/dcb/fd6 707446 0 2024-09-21T04:15:21.738 INFO:tasks.workunit.client.0.smithi110.stdout:7/598: symlink dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/lbe 0 2024-09-21T04:15:21.738 INFO:tasks.workunit.client.0.smithi110.stdout:7/599: fdatasync dd/d14/d17/fb9 0 2024-09-21T04:15:21.739 INFO:tasks.workunit.client.1.smithi148.stdout:5/693: unlink d3/d14/d1f/d26/d2e/d56/c7b 0 2024-09-21T04:15:21.739 INFO:tasks.workunit.client.1.smithi148.stdout:7/661: rmdir d5/d14/d25/d40 39 2024-09-21T04:15:21.741 INFO:tasks.workunit.client.0.smithi110.stdout:7/600: mknod dd/d1c/dad/db2/d4a/da5/cbf 0 2024-09-21T04:15:21.742 INFO:tasks.workunit.client.1.smithi148.stdout:2/814: symlink d0/d9/d100/l102 0 2024-09-21T04:15:21.742 INFO:tasks.workunit.client.0.smithi110.stdout:7/601: mknod dd/d1c/dad/cc0 0 2024-09-21T04:15:21.742 INFO:tasks.workunit.client.0.smithi110.stdout:7/602: chown dd/d1c/l64 226959 1 2024-09-21T04:15:21.743 INFO:tasks.workunit.client.1.smithi148.stdout:7/662: creat d5/d8/d6f/fec x:0 0 0 2024-09-21T04:15:21.743 INFO:tasks.workunit.client.0.smithi110.stdout:7/603: mkdir dd/d4d/dc1 0 2024-09-21T04:15:21.744 INFO:tasks.workunit.client.1.smithi148.stdout:4/730: getdents d4/d12/da7 0 2024-09-21T04:15:21.745 INFO:tasks.workunit.client.0.smithi110.stdout:0/918: dwrite d4/f1f [0,4194304] 0 2024-09-21T04:15:21.745 INFO:tasks.workunit.client.0.smithi110.stdout:0/919: chown d4/dd/d53/d66/fc1 484279609 1 2024-09-21T04:15:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:0/679: dwrite d5/d6/f1f [0,4194304] 0 2024-09-21T04:15:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:0/680: chown d5/d6/l14 2320 1 2024-09-21T04:15:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:0/681: write d5/d6/d2a/d2b/f61 [4348316,78713] 0 2024-09-21T04:15:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:0/682: read d5/d6/d2a/d2b/d84/da5/fbf [232922,27187] 0 2024-09-21T04:15:21.751 INFO:tasks.workunit.client.0.smithi110.stdout:2/451: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:21.752 INFO:tasks.workunit.client.0.smithi110.stdout:3/512: dwrite d0/d12/d16/d1a/f3b [4194304,4194304] 0 2024-09-21T04:15:21.752 INFO:tasks.workunit.client.0.smithi110.stdout:2/452: chown d2/d1a/d1f/d29/d19/d62/d86 61056712 1 2024-09-21T04:15:21.752 INFO:tasks.workunit.client.1.smithi148.stdout:1/634: dwrite db/dd/f23 [0,4194304] 0 2024-09-21T04:15:21.752 INFO:tasks.workunit.client.1.smithi148.stdout:1/635: chown db/d24/d2e/d71 85 1 2024-09-21T04:15:21.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/636: truncate db/d24/d2e/d71/f8e 198420 0 2024-09-21T04:15:21.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/637: stat db/f64 0 2024-09-21T04:15:21.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/638: chown db/d24/d2a/da4/fb3 5309 1 2024-09-21T04:15:21.757 INFO:tasks.workunit.client.0.smithi110.stdout:1/934: dwrite d2/d9/d26/d3d/f11a [0,4194304] 0 2024-09-21T04:15:21.757 INFO:tasks.workunit.client.0.smithi110.stdout:1/935: dread - d2/d9/d122/f12e zero size 2024-09-21T04:15:21.757 INFO:tasks.workunit.client.0.smithi110.stdout:1/936: truncate d2/d9/d26/f7b 983610 0 2024-09-21T04:15:21.762 INFO:tasks.workunit.client.0.smithi110.stdout:7/604: dread dd/d1c/f75 [0,4194304] 0 2024-09-21T04:15:21.763 INFO:tasks.workunit.client.0.smithi110.stdout:8/630: dread d0/d30/d54/d6b/f75 [0,4194304] 0 2024-09-21T04:15:21.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/683: dwrite d0/d34/d38/d43/f78 [0,4194304] 0 2024-09-21T04:15:21.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/684: truncate d0/d34/da0/fa9 4204523 0 2024-09-21T04:15:21.767 INFO:tasks.workunit.client.1.smithi148.stdout:5/694: getdents d3/d14/d1f/d26/d59 0 2024-09-21T04:15:21.767 INFO:tasks.workunit.client.1.smithi148.stdout:5/695: chown d3/d34/d3c/l74 138 1 2024-09-21T04:15:21.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/663: dread d5/d8/d6f/d7d/d90/d28/f29 [0,4194304] 0 2024-09-21T04:15:21.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/664: creat d5/d14/d25/d79/fed x:0 0 0 2024-09-21T04:15:21.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/453: dread d2/d1a/d1f/f38 [0,4194304] 0 2024-09-21T04:15:21.773 INFO:tasks.workunit.client.1.smithi148.stdout:4/731: rmdir d4/d12/d16/db3 0 2024-09-21T04:15:21.774 INFO:tasks.workunit.client.0.smithi110.stdout:3/513: symlink d0/d12/d3c/d5a/d65/lb2 0 2024-09-21T04:15:21.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/815: dwrite d0/d2b/da3/fe7 [0,4194304] 0 2024-09-21T04:15:21.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/816: chown d0/d9/cf3 34 1 2024-09-21T04:15:21.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/817: write d0/d9/f6e [1717232,13292] 0 2024-09-21T04:15:21.776 INFO:tasks.workunit.client.1.smithi148.stdout:0/683: dwrite d5/d6/d2a/d2b/d84/daf/f9f [0,4194304] 0 2024-09-21T04:15:21.776 INFO:tasks.workunit.client.0.smithi110.stdout:7/605: read dd/d1c/f38 [1893700,34124] 0 2024-09-21T04:15:21.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/639: mknod db/d24/d2e/d59/da7/dc1/ccf 0 2024-09-21T04:15:21.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/640: fsync db/dd/d21/f93 0 2024-09-21T04:15:21.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/641: stat db/d46/d4c/d82/cc4 0 2024-09-21T04:15:21.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/642: creat db/d24/fd0 x:0 0 0 2024-09-21T04:15:21.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/643: chown db/d24/d2e/f63 37442 1 2024-09-21T04:15:21.780 INFO:tasks.workunit.client.0.smithi110.stdout:1/937: mkdir d2/d9/d49/d4f/d131 0 2024-09-21T04:15:21.780 INFO:tasks.workunit.client.0.smithi110.stdout:1/938: truncate d2/d9/d49/d4f/d53/fd4 946666 0 2024-09-21T04:15:21.780 INFO:tasks.workunit.client.0.smithi110.stdout:1/939: chown d2/d9/d23/d4e/d57/dc9/c128 2897629 1 2024-09-21T04:15:21.780 INFO:tasks.workunit.client.0.smithi110.stdout:1/940: write d2/d9/d49/d4f/f106 [265878,20922] 0 2024-09-21T04:15:21.785 INFO:tasks.workunit.client.0.smithi110.stdout:0/920: getdents d4/dd/d10/d29/d5f/dc3 0 2024-09-21T04:15:21.785 INFO:tasks.workunit.client.0.smithi110.stdout:0/921: dread - d4/dd/d10/d29/fe4 zero size 2024-09-21T04:15:21.787 INFO:tasks.workunit.client.0.smithi110.stdout:2/454: dread d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:21.788 INFO:tasks.workunit.client.0.smithi110.stdout:7/606: creat dd/d1c/dad/db2/d6d/dbc/fc2 x:0 0 0 2024-09-21T04:15:21.792 INFO:tasks.workunit.client.0.smithi110.stdout:0/922: truncate d4/dd/d10/d29/fec 2514459 0 2024-09-21T04:15:21.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/631: rmdir d0/d30/d34/d3e/d5d 39 2024-09-21T04:15:21.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/632: write d0/d7/d23/d52/d8b/faf [755590,24410] 0 2024-09-21T04:15:21.795 INFO:tasks.workunit.client.0.smithi110.stdout:2/455: creat d2/d1a/f88 x:0 0 0 2024-09-21T04:15:21.797 INFO:tasks.workunit.client.0.smithi110.stdout:7/607: link dd/d1c/f38 dd/d14/d17/fc3 0 2024-09-21T04:15:21.799 INFO:tasks.workunit.client.0.smithi110.stdout:0/923: write d4/d1c/d62/d69/f77 [3873288,82642] 0 2024-09-21T04:15:21.800 INFO:tasks.workunit.client.0.smithi110.stdout:1/941: getdents d2/d9/d49/d4f/d87/d126 0 2024-09-21T04:15:21.802 INFO:tasks.workunit.client.0.smithi110.stdout:8/633: mknod d0/d7/d23/dc1/cd8 0 2024-09-21T04:15:21.804 INFO:tasks.workunit.client.0.smithi110.stdout:2/456: rename d2/d1a/d1f/c37 to d2/d1a/d1f/d65/c89 0 2024-09-21T04:15:21.807 INFO:tasks.workunit.client.0.smithi110.stdout:7/608: rmdir dd/d1c/dad/db2 39 2024-09-21T04:15:21.810 INFO:tasks.workunit.client.0.smithi110.stdout:0/924: creat d4/dd/d10/d29/d5f/dc3/f118 x:0 0 0 2024-09-21T04:15:21.811 INFO:tasks.workunit.client.0.smithi110.stdout:1/942: truncate d2/d9/d49/d5c/d91/fa5 3189530 0 2024-09-21T04:15:21.812 INFO:tasks.workunit.client.0.smithi110.stdout:2/457: read d2/d50/d56/f51 [463138,64909] 0 2024-09-21T04:15:21.814 INFO:tasks.workunit.client.0.smithi110.stdout:8/634: mknod d0/d30/d54/d6b/da8/cd9 0 2024-09-21T04:15:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:7/609: creat dd/d1c/dad/db2/d56/fc4 x:0 0 0 2024-09-21T04:15:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:7/610: chown dd/d1c/dad/db2/d56/l78 4 1 2024-09-21T04:15:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:3/514: dwrite d0/d12/d16/f6a [0,4194304] 0 2024-09-21T04:15:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:3/515: creat d0/d12/fb3 x:0 0 0 2024-09-21T04:15:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:4/738: write f4 [2095499,34906] 0 2024-09-21T04:15:21.818 INFO:tasks.workunit.client.1.smithi148.stdout:6/685: dwrite d0/d1a/f83 [0,4194304] 0 2024-09-21T04:15:21.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/644: dwrite f6 [4194304,4194304] 0 2024-09-21T04:15:21.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/645: fsync db/d24/d2a/da4/fbc 0 2024-09-21T04:15:21.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/646: chown db/d46/d4c/d82/f90 2573 1 2024-09-21T04:15:21.818 INFO:tasks.workunit.client.1.smithi148.stdout:5/696: dwrite d3/f36 [0,4194304] 0 2024-09-21T04:15:21.824 INFO:tasks.workunit.client.0.smithi110.stdout:8/635: mkdir d0/d30/d34/d3e/d5d/dd7/dda 0 2024-09-21T04:15:21.824 INFO:tasks.workunit.client.0.smithi110.stdout:8/636: fdatasync d0/d7/d13/d4b/f35 0 2024-09-21T04:15:21.826 INFO:tasks.workunit.client.0.smithi110.stdout:7/611: symlink dd/lc5 0 2024-09-21T04:15:21.826 INFO:tasks.workunit.client.0.smithi110.stdout:7/612: stat dd/d1c/dad/c9e 0 2024-09-21T04:15:21.831 INFO:tasks.workunit.client.1.smithi148.stdout:7/665: mkdir d5/d8/d6f/d7d/d90/dee 0 2024-09-21T04:15:21.832 INFO:tasks.workunit.client.1.smithi148.stdout:4/732: symlink d4/le0 0 2024-09-21T04:15:21.832 INFO:tasks.workunit.client.1.smithi148.stdout:4/733: dread - d4/d12/d75/d9c/fde zero size 2024-09-21T04:15:21.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/818: symlink d0/d9/d59/l103 0 2024-09-21T04:15:21.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/819: write d0/d2b/da3/d8f/d9f/fe0 [807148,38350] 0 2024-09-21T04:15:21.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/820: write d0/d9/dc/d13/f21 [1157142,76742] 0 2024-09-21T04:15:21.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/821: readlink d0/d2b/l64 0 2024-09-21T04:15:21.845 INFO:tasks.workunit.client.1.smithi148.stdout:0/684: creat d5/d6/d2a/d2b/d84/da5/fde x:0 0 0 2024-09-21T04:15:21.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/925: dwrite d4/dd/d53/d66/fc1 [0,4194304] 0 2024-09-21T04:15:21.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/516: unlink d0/d12/d2a/f4d 0 2024-09-21T04:15:21.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/517: truncate d0/f88 1027039 0 2024-09-21T04:15:21.849 INFO:tasks.workunit.client.0.smithi110.stdout:2/458: getdents d2/d1a/d1f/d29 0 2024-09-21T04:15:21.849 INFO:tasks.workunit.client.0.smithi110.stdout:2/459: chown d2/d1a/d1f/d29/c3d 0 1 2024-09-21T04:15:21.851 INFO:tasks.workunit.client.1.smithi148.stdout:5/697: symlink d3/d88/lf1 0 2024-09-21T04:15:21.852 INFO:tasks.workunit.client.0.smithi110.stdout:8/637: symlink d0/d7/d6f/ldb 0 2024-09-21T04:15:21.853 INFO:tasks.workunit.client.1.smithi148.stdout:2/822: mknod d0/d9/d38/c104 0 2024-09-21T04:15:21.853 INFO:tasks.workunit.client.0.smithi110.stdout:7/613: rmdir dd/d1c/dad/db2/d56/d66/da6/dbd 0 2024-09-21T04:15:21.855 INFO:tasks.workunit.client.1.smithi148.stdout:0/685: creat d5/d6/d2a/d3f/d6d/da3/dcc/fdf x:0 0 0 2024-09-21T04:15:21.857 INFO:tasks.workunit.client.1.smithi148.stdout:2/823: getdents d0/d2f/db1 0 2024-09-21T04:15:21.857 INFO:tasks.workunit.client.0.smithi110.stdout:0/926: mknod d4/dd/d60/d10e/c119 0 2024-09-21T04:15:21.858 INFO:tasks.workunit.client.0.smithi110.stdout:0/927: getdents d4/dd/d10/d29/d75/dd2 0 2024-09-21T04:15:21.859 INFO:tasks.workunit.client.1.smithi148.stdout:0/686: creat d5/d6/d7/d11/fe0 x:0 0 0 2024-09-21T04:15:21.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/518: mkdir d0/d12/d2a/db4 0 2024-09-21T04:15:21.862 INFO:tasks.workunit.client.1.smithi148.stdout:2/824: rename d0/d9/d16/lce to d0/d9/d16/d55/d71/l105 0 2024-09-21T04:15:21.864 INFO:tasks.workunit.client.1.smithi148.stdout:0/687: creat d5/d6/d2a/d2b/d84/db4/fe1 x:0 0 0 2024-09-21T04:15:21.865 INFO:tasks.workunit.client.1.smithi148.stdout:6/686: dwrite d0/d34/d38/d57/f75 [0,4194304] 0 2024-09-21T04:15:21.865 INFO:tasks.workunit.client.1.smithi148.stdout:1/647: dwrite db/f15 [0,4194304] 0 2024-09-21T04:15:21.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/460: truncate d2/d1a/d1f/d29/f48 1601768 0 2024-09-21T04:15:21.866 INFO:tasks.workunit.client.0.smithi110.stdout:0/928: rmdir d4/dd/d10/d29/d45 39 2024-09-21T04:15:21.866 INFO:tasks.workunit.client.0.smithi110.stdout:0/929: write d4/d1c/f3c [1022140,2073] 0 2024-09-21T04:15:21.867 INFO:tasks.workunit.client.1.smithi148.stdout:7/666: dwrite d5/d14/f4b [0,4194304] 0 2024-09-21T04:15:21.867 INFO:tasks.workunit.client.1.smithi148.stdout:7/667: dread - d5/d14/d25/fdd zero size 2024-09-21T04:15:21.868 INFO:tasks.workunit.client.0.smithi110.stdout:3/519: getdents d0/d12/d3c/d5a/d70/d81 0 2024-09-21T04:15:21.868 INFO:tasks.workunit.client.0.smithi110.stdout:3/520: chown d0/d12/d16/d1a/la9 3747 1 2024-09-21T04:15:21.870 INFO:tasks.workunit.client.1.smithi148.stdout:4/734: dwrite d4/d87/fd5 [0,4194304] 0 2024-09-21T04:15:21.871 INFO:tasks.workunit.client.0.smithi110.stdout:2/461: creat d2/d1a/d60/f8a x:0 0 0 2024-09-21T04:15:21.873 INFO:tasks.workunit.client.0.smithi110.stdout:0/930: truncate d4/d1c/d62/f36 2943345 0 2024-09-21T04:15:21.873 INFO:tasks.workunit.client.0.smithi110.stdout:0/931: fsync d4/dd/d53/f5a 0 2024-09-21T04:15:21.877 INFO:tasks.workunit.client.0.smithi110.stdout:3/521: creat d0/d12/d16/d1a/dac/fb5 x:0 0 0 2024-09-21T04:15:21.878 INFO:tasks.workunit.client.1.smithi148.stdout:7/668: mknod d5/d8/d6f/d7d/cef 0 2024-09-21T04:15:21.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/687: dread d0/d1a/d23/d27/d62/d9a/f7a [0,4194304] 0 2024-09-21T04:15:21.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/688: write d0/d1a/d23/d53/f9f [4946891,115983] 0 2024-09-21T04:15:21.880 INFO:tasks.workunit.client.0.smithi110.stdout:2/462: getdents d2/d50/d7b 0 2024-09-21T04:15:21.882 INFO:tasks.workunit.client.0.smithi110.stdout:0/932: dread d4/dd/d60/fa4 [0,4194304] 0 2024-09-21T04:15:21.884 INFO:tasks.workunit.client.0.smithi110.stdout:0/933: symlink d4/d1c/d62/d91/l11a 0 2024-09-21T04:15:21.887 INFO:tasks.workunit.client.0.smithi110.stdout:3/522: rmdir d0/d22/d29/d87 0 2024-09-21T04:15:21.888 INFO:tasks.workunit.client.0.smithi110.stdout:0/934: truncate d4/dd/d60/fa4 2227780 0 2024-09-21T04:15:21.891 INFO:tasks.workunit.client.0.smithi110.stdout:0/935: rmdir d4/d47/d2c/d3a 39 2024-09-21T04:15:21.893 INFO:tasks.workunit.client.0.smithi110.stdout:0/936: mkdir d4/dd/d60/de5/d11b 0 2024-09-21T04:15:21.894 INFO:tasks.workunit.client.0.smithi110.stdout:1/943: write d2/db7/d86/f89 [3484810,75144] 0 2024-09-21T04:15:21.896 INFO:tasks.workunit.client.1.smithi148.stdout:4/735: creat d4/d66/d7f/d9a/dc2/fe1 x:0 0 0 2024-09-21T04:15:21.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/463: write d2/d1a/d1f/d5f/f58 [654691,109640] 0 2024-09-21T04:15:21.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/464: read d2/f16 [1077396,59514] 0 2024-09-21T04:15:21.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/465: fdatasync d2/d50/d56/f7a 0 2024-09-21T04:15:21.897 INFO:tasks.workunit.client.0.smithi110.stdout:1/944: creat d2/d9/d23/d3b/dca/d107/f132 x:0 0 0 2024-09-21T04:15:21.897 INFO:tasks.workunit.client.0.smithi110.stdout:1/945: dread - d2/d9/d49/d4f/f12b zero size 2024-09-21T04:15:21.898 INFO:tasks.workunit.client.0.smithi110.stdout:1/946: readlink d2/d9/d26/d3d/d8f/l98 0 2024-09-21T04:15:21.898 INFO:tasks.workunit.client.0.smithi110.stdout:1/947: dread - d2/d9/d23/de0/f11d zero size 2024-09-21T04:15:21.900 INFO:tasks.workunit.client.1.smithi148.stdout:1/648: dwrite db/f1b [0,4194304] 0 2024-09-21T04:15:21.900 INFO:tasks.workunit.client.1.smithi148.stdout:1/649: readlink db/l49 0 2024-09-21T04:15:21.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/825: dwrite d0/d2b/da3/d8f/fb7 [0,4194304] 0 2024-09-21T04:15:21.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/826: write d0/d9/d16/d55/d71/dc4/fe2 [1212848,62318] 0 2024-09-21T04:15:21.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/466: truncate d2/d1a/d1f/d29/f45 4140964 0 2024-09-21T04:15:21.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/467: chown d2/d1a/d1f/f38 3511562 1 2024-09-21T04:15:21.902 INFO:tasks.workunit.client.0.smithi110.stdout:7/614: dwrite dd/f57 [0,4194304] 0 2024-09-21T04:15:21.902 INFO:tasks.workunit.client.0.smithi110.stdout:7/615: creat dd/fc6 x:0 0 0 2024-09-21T04:15:21.903 INFO:tasks.workunit.client.1.smithi148.stdout:5/698: dwrite d3/d4a/fae [0,4194304] 0 2024-09-21T04:15:21.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/689: mknod d0/d1a/d23/ccc 0 2024-09-21T04:15:21.904 INFO:tasks.workunit.client.0.smithi110.stdout:3/523: write d0/d12/f6c [2900253,88336] 0 2024-09-21T04:15:21.904 INFO:tasks.workunit.client.0.smithi110.stdout:8/638: dwrite d0/d30/d54/d6b/fc3 [0,4194304] 0 2024-09-21T04:15:21.904 INFO:tasks.workunit.client.0.smithi110.stdout:8/639: readlink d0/d7/d23/l65 0 2024-09-21T04:15:21.904 INFO:tasks.workunit.client.0.smithi110.stdout:3/524: truncate d0/d12/f9d 1401076 0 2024-09-21T04:15:21.905 INFO:tasks.workunit.client.0.smithi110.stdout:1/948: rename d2/d9/d23/lec to d2/d9/d26/d3d/l133 0 2024-09-21T04:15:21.905 INFO:tasks.workunit.client.0.smithi110.stdout:1/949: fdatasync d2/f45 0 2024-09-21T04:15:21.905 INFO:tasks.workunit.client.0.smithi110.stdout:1/950: creat d2/d9/d49/d4f/d87/f134 x:0 0 0 2024-09-21T04:15:21.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/669: unlink d5/d8/l11 0 2024-09-21T04:15:21.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/670: fdatasync d5/d8/d6f/d7d/d90/f19 0 2024-09-21T04:15:21.914 INFO:tasks.workunit.client.1.smithi148.stdout:2/827: creat d0/d9/dc/d13/d50/dc0/f106 x:0 0 0 2024-09-21T04:15:21.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/736: link d4/d87/d91/fb2 d4/d12/d16/d29/d61/fe2 0 2024-09-21T04:15:21.918 INFO:tasks.workunit.client.1.smithi148.stdout:2/828: creat d0/d9/d16/d9e/f107 x:0 0 0 2024-09-21T04:15:21.918 INFO:tasks.workunit.client.1.smithi148.stdout:2/829: chown d0/d9/dc/d13/f69 230 1 2024-09-21T04:15:21.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/640: rename d0/d7/d13/d4b/d91 to d0/d7/d13/d4b/ddc 0 2024-09-21T04:15:21.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/641: chown d0/d7/d13/d1c/d41/f43 211006 1 2024-09-21T04:15:21.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/642: creat d0/d30/d34/fdd x:0 0 0 2024-09-21T04:15:21.923 INFO:tasks.workunit.client.0.smithi110.stdout:7/616: rename dd/d1c/dad/db2/d56/d66/da6/l83 to dd/d48/d9f/lc7 0 2024-09-21T04:15:21.924 INFO:tasks.workunit.client.0.smithi110.stdout:7/617: creat dd/d48/fc8 x:0 0 0 2024-09-21T04:15:21.924 INFO:tasks.workunit.client.1.smithi148.stdout:7/671: read d5/d14/d25/d40/db9/fbd [4185605,44164] 0 2024-09-21T04:15:21.924 INFO:tasks.workunit.client.1.smithi148.stdout:7/672: write d5/d14/d25/fdd [176644,103079] 0 2024-09-21T04:15:21.926 INFO:tasks.workunit.client.0.smithi110.stdout:8/643: link d0/d30/d54/d6b/fcc d0/d30/d34/d3e/d5d/fde 0 2024-09-21T04:15:21.926 INFO:tasks.workunit.client.0.smithi110.stdout:8/644: write d0/f3d [4967302,76995] 0 2024-09-21T04:15:21.926 INFO:tasks.workunit.client.0.smithi110.stdout:8/645: chown d0/d7/d6f/d86/fa6 791 1 2024-09-21T04:15:21.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/673: symlink d5/d14/d25/d40/lf0 0 2024-09-21T04:15:21.929 INFO:tasks.workunit.client.0.smithi110.stdout:7/618: mknod dd/d1c/dad/db2/cc9 0 2024-09-21T04:15:21.929 INFO:tasks.workunit.client.0.smithi110.stdout:7/619: write dd/d1c/dad/db2/d56/d66/da6/d7a/f91 [519036,127932] 0 2024-09-21T04:15:21.929 INFO:tasks.workunit.client.0.smithi110.stdout:7/620: write dd/d1c/dad/db2/f4f [1282788,12204] 0 2024-09-21T04:15:21.931 INFO:tasks.workunit.client.1.smithi148.stdout:7/674: rmdir d5/d8/d6f/d7d/d90/d9b 39 2024-09-21T04:15:21.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/737: write d4/d12/d16/d29/f46 [2790429,73186] 0 2024-09-21T04:15:21.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/738: read d4/d66/d7f/d2a/f36 [2521862,58456] 0 2024-09-21T04:15:21.935 INFO:tasks.workunit.client.0.smithi110.stdout:7/621: rename dd/d1c/dad/db2/f95 to dd/d4d/dc1/fca 0 2024-09-21T04:15:21.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/675: rename d5/d8/c20 to d5/d14/d25/d40/d7b/d93/cf1 0 2024-09-21T04:15:21.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/676: write d5/d14/d25/d40/d7b/fd5 [775486,57769] 0 2024-09-21T04:15:21.937 INFO:tasks.workunit.client.1.smithi148.stdout:2/830: dread d0/d9/dc/d13/f8b [0,4194304] 0 2024-09-21T04:15:21.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/622: creat dd/d14/fcb x:0 0 0 2024-09-21T04:15:21.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/623: stat dd/d1c/dad/db2/d4a/d52/lba 0 2024-09-21T04:15:21.939 INFO:tasks.workunit.client.1.smithi148.stdout:4/739: rename d4/d66/d7f/f2e to d4/d87/d2d/dd0/fe3 0 2024-09-21T04:15:21.941 INFO:tasks.workunit.client.1.smithi148.stdout:2/831: dread d0/d9/d16/d55/d6f/f7e [0,4194304] 0 2024-09-21T04:15:21.944 INFO:tasks.workunit.client.0.smithi110.stdout:7/624: rmdir dd/d1c/dad/db2/d56/d66/da6/d7a/d82 0 2024-09-21T04:15:21.945 INFO:tasks.workunit.client.0.smithi110.stdout:7/625: unlink dd/d14/c74 0 2024-09-21T04:15:21.947 INFO:tasks.workunit.client.0.smithi110.stdout:7/626: symlink dd/d1c/dad/db2/d4a/lcc 0 2024-09-21T04:15:21.947 INFO:tasks.workunit.client.0.smithi110.stdout:7/627: creat dd/d1c/dad/db2/d56/d66/fcd x:0 0 0 2024-09-21T04:15:21.947 INFO:tasks.workunit.client.0.smithi110.stdout:7/628: readlink dd/d1c/lb0 0 2024-09-21T04:15:21.947 INFO:tasks.workunit.client.0.smithi110.stdout:7/629: write dd/d48/f4b [961699,61529] 0 2024-09-21T04:15:21.950 INFO:tasks.workunit.client.0.smithi110.stdout:7/630: unlink dd/d1c/dad/db2/d6d/c81 0 2024-09-21T04:15:21.951 INFO:tasks.workunit.client.0.smithi110.stdout:8/646: dread d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:21.953 INFO:tasks.workunit.client.0.smithi110.stdout:8/647: symlink d0/d30/d34/d3e/d60/ldf 0 2024-09-21T04:15:21.954 INFO:tasks.workunit.client.0.smithi110.stdout:8/648: creat d0/d30/d34/d3e/d5d/dd7/fe0 x:0 0 0 2024-09-21T04:15:21.955 INFO:tasks.workunit.client.0.smithi110.stdout:8/649: symlink d0/d30/d34/d3e/d5d/le1 0 2024-09-21T04:15:21.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/468: dwrite d2/d1a/d1f/f5d [0,4194304] 0 2024-09-21T04:15:21.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/469: truncate d2/f16 4408429 0 2024-09-21T04:15:21.957 INFO:tasks.workunit.client.0.smithi110.stdout:1/951: dwrite d2/db7/d86/f95 [0,4194304] 0 2024-09-21T04:15:21.957 INFO:tasks.workunit.client.0.smithi110.stdout:1/952: write d2/d9/d49/d7a/dee/fcf [1014222,36717] 0 2024-09-21T04:15:21.957 INFO:tasks.workunit.client.1.smithi148.stdout:5/699: dwrite d3/d14/d1f/d26/d2e/d56/fe3 [0,4194304] 0 2024-09-21T04:15:21.957 INFO:tasks.workunit.client.1.smithi148.stdout:1/650: dwrite db/dd/f12 [0,4194304] 0 2024-09-21T04:15:21.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/740: write d4/d87/fd5 [3229654,115085] 0 2024-09-21T04:15:21.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/741: fdatasync d4/d12/d16/d29/f59 0 2024-09-21T04:15:21.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/742: readlink d4/d12/d16/d29/d61/d6c/l84 0 2024-09-21T04:15:21.958 INFO:tasks.workunit.client.1.smithi148.stdout:0/688: dwrite d5/d6/d2a/d2b/f61 [0,4194304] 0 2024-09-21T04:15:21.958 INFO:tasks.workunit.client.0.smithi110.stdout:8/650: mkdir d0/d7/d13/dc5/de2 0 2024-09-21T04:15:21.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/677: dwrite d5/d14/d25/f4c [0,4194304] 0 2024-09-21T04:15:21.960 INFO:tasks.workunit.client.0.smithi110.stdout:7/631: write dd/f31 [1405055,20729] 0 2024-09-21T04:15:21.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/470: unlink d2/d1a/d1f/f38 0 2024-09-21T04:15:21.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/471: truncate d2/d1a/f88 373263 0 2024-09-21T04:15:21.963 INFO:tasks.workunit.client.1.smithi148.stdout:5/700: mknod d3/d14/d1f/d72/dc9/dd8/cf2 0 2024-09-21T04:15:21.968 INFO:tasks.workunit.client.1.smithi148.stdout:1/651: creat db/dd/d21/d89/fd1 x:0 0 0 2024-09-21T04:15:21.968 INFO:tasks.workunit.client.1.smithi148.stdout:1/652: fsync db/d24/d2e/d59/fa2 0 2024-09-21T04:15:21.974 INFO:tasks.workunit.client.1.smithi148.stdout:1/653: dread db/d46/f66 [0,4194304] 0 2024-09-21T04:15:21.975 INFO:tasks.workunit.client.1.smithi148.stdout:4/743: mkdir d4/d12/d78/de4 0 2024-09-21T04:15:21.975 INFO:tasks.workunit.client.1.smithi148.stdout:4/744: creat d4/d12/d16/d29/d61/d7e/fe5 x:0 0 0 2024-09-21T04:15:21.976 INFO:tasks.workunit.client.1.smithi148.stdout:0/689: unlink d5/d6/d7/f43 0 2024-09-21T04:15:21.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/690: creat d5/d6/d2a/fe2 x:0 0 0 2024-09-21T04:15:21.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/691: dread - d5/d6/d2a/fd7 zero size 2024-09-21T04:15:21.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/692: dread - d5/d6/d2a/d45/f9b zero size 2024-09-21T04:15:21.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/693: readlink d5/ldb 0 2024-09-21T04:15:21.982 INFO:tasks.workunit.client.0.smithi110.stdout:8/651: dwrite d0/d30/d34/fa5 [0,4194304] 0 2024-09-21T04:15:21.982 INFO:tasks.workunit.client.0.smithi110.stdout:1/953: mkdir d2/d9/d23/d120/d135 0 2024-09-21T04:15:21.983 INFO:tasks.workunit.client.0.smithi110.stdout:2/472: dwrite d2/d1a/d1f/d65/f75 [0,4194304] 0 2024-09-21T04:15:21.983 INFO:tasks.workunit.client.0.smithi110.stdout:8/652: dread d0/d30/d34/d3e/d32/da3/fa9 [0,4194304] 0 2024-09-21T04:15:21.984 INFO:tasks.workunit.client.1.smithi148.stdout:7/678: truncate d5/d8/d6f/d7d/d90/f5d 97629 0 2024-09-21T04:15:21.984 INFO:tasks.workunit.client.1.smithi148.stdout:7/679: truncate d5/d14/d25/d79/d86/fd9 177962 0 2024-09-21T04:15:21.984 INFO:tasks.workunit.client.1.smithi148.stdout:7/680: chown d5/d8/d6f/d7d/d90/d28/fda 53892646 1 2024-09-21T04:15:21.986 INFO:tasks.workunit.client.1.smithi148.stdout:5/701: mknod d3/d14/d1f/d26/d2e/ddb/cf3 0 2024-09-21T04:15:21.987 INFO:tasks.workunit.client.0.smithi110.stdout:7/632: symlink dd/d48/d9f/lce 0 2024-09-21T04:15:21.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/654: mkdir db/d24/d2e/d59/da7/dc1/dd2 0 2024-09-21T04:15:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:9/717: sync 2024-09-21T04:15:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:9/718: creat d5/d60/d28/d73/f102 x:0 0 0 2024-09-21T04:15:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/638: sync 2024-09-21T04:15:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:8/643: sync 2024-09-21T04:15:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:8/644: truncate dc/d13/d76/f2c 5216090 0 2024-09-21T04:15:21.995 INFO:tasks.workunit.client.1.smithi148.stdout:4/745: symlink d4/d12/d16/d29/d61/d6c/le6 0 2024-09-21T04:15:21.996 INFO:tasks.workunit.client.1.smithi148.stdout:7/681: dread d5/d8/d6f/fc9 [0,4194304] 0 2024-09-21T04:15:21.998 INFO:tasks.workunit.client.1.smithi148.stdout:0/694: stat d5/d6/d2a/d2b/f5b 0 2024-09-21T04:15:21.998 INFO:tasks.workunit.client.1.smithi148.stdout:0/695: fsync d5/fd8 0 2024-09-21T04:15:21.998 INFO:tasks.workunit.client.1.smithi148.stdout:0/696: chown d5/d6/d2a/d3f/d8b 143 1 2024-09-21T04:15:21.998 INFO:tasks.workunit.client.1.smithi148.stdout:0/697: fdatasync d5/d6/d7/d11/fe0 0 2024-09-21T04:15:22.002 INFO:tasks.workunit.client.1.smithi148.stdout:5/702: creat d3/d14/d1f/db1/ff4 x:0 0 0 2024-09-21T04:15:22.003 INFO:tasks.workunit.client.1.smithi148.stdout:9/719: write d5/d60/d28/d73/fc3 [1126025,47563] 0 2024-09-21T04:15:22.003 INFO:tasks.workunit.client.1.smithi148.stdout:9/720: write d5/d2b/d4e/f97 [1243319,102821] 0 2024-09-21T04:15:22.003 INFO:tasks.workunit.client.0.smithi110.stdout:2/473: getdents d2/d1a 0 2024-09-21T04:15:22.006 INFO:tasks.workunit.client.1.smithi148.stdout:1/655: rename db/d46/f6a to db/d46/d76/fd3 0 2024-09-21T04:15:22.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/653: dwrite d0/d30/d9d/fd5 [0,4194304] 0 2024-09-21T04:15:22.008 INFO:tasks.workunit.client.0.smithi110.stdout:2/474: mkdir d2/d1a/d8b 0 2024-09-21T04:15:22.012 INFO:tasks.workunit.client.1.smithi148.stdout:8/645: symlink dc/d13/d76/d5d/d78/lc4 0 2024-09-21T04:15:22.013 INFO:tasks.workunit.client.1.smithi148.stdout:9/721: dread d5/d1e/f49 [0,4194304] 0 2024-09-21T04:15:22.014 INFO:tasks.workunit.client.1.smithi148.stdout:7/682: unlink d5/d8/d6f/d7d/d90/d28/f57 0 2024-09-21T04:15:22.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/654: symlink d0/d30/d34/d3e/d5d/le3 0 2024-09-21T04:15:22.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/655: chown d0/d30/d34/d3e/d60/cc2 31 1 2024-09-21T04:15:22.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/698: creat d5/d6/d2a/d2b/d84/daf/dc8/fe3 x:0 0 0 2024-09-21T04:15:22.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/699: readlink d5/d6/d7/d11/d48/l4a 0 2024-09-21T04:15:22.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/700: chown d5/d6/d7/d22/c5e 63540 1 2024-09-21T04:15:22.017 INFO:tasks.workunit.client.1.smithi148.stdout:5/703: link d3/d34/d78/dbf/dc2/ce9 d3/d14/d1f/d26/d59/cf5 0 2024-09-21T04:15:22.018 INFO:tasks.workunit.client.1.smithi148.stdout:4/746: dwrite d4/d12/d16/d29/f5a [4194304,4194304] 0 2024-09-21T04:15:22.018 INFO:tasks.workunit.client.1.smithi148.stdout:3/639: dwrite d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:22.019 INFO:tasks.workunit.client.0.smithi110.stdout:2/475: mknod d2/d1a/c8c 0 2024-09-21T04:15:22.019 INFO:tasks.workunit.client.0.smithi110.stdout:2/476: fdatasync d2/f3b 0 2024-09-21T04:15:22.020 INFO:tasks.workunit.client.1.smithi148.stdout:1/656: symlink db/dd/d21/d9b/ld4 0 2024-09-21T04:15:22.021 INFO:tasks.workunit.client.1.smithi148.stdout:8/646: mkdir dc/d43/d4c/d4f/dc5 0 2024-09-21T04:15:22.021 INFO:tasks.workunit.client.1.smithi148.stdout:5/704: dread d3/d14/d1f/d26/f77 [0,4194304] 0 2024-09-21T04:15:22.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/722: mkdir d5/d2b/d32/d103 0 2024-09-21T04:15:22.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/723: dread - d5/d2b/f5d zero size 2024-09-21T04:15:22.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/724: stat d5/f63 0 2024-09-21T04:15:22.024 INFO:tasks.workunit.client.1.smithi148.stdout:0/701: link d5/d6/d2a/d2b/d99/db7/fc7 d5/dcb/fe4 0 2024-09-21T04:15:22.026 INFO:tasks.workunit.client.1.smithi148.stdout:4/747: rename d4/c5 to d4/d66/d94/ce7 0 2024-09-21T04:15:22.026 INFO:tasks.workunit.client.1.smithi148.stdout:4/748: truncate d4/d12/d16/d29/d61/d6c/d9d/fb0 1037962 0 2024-09-21T04:15:22.026 INFO:tasks.workunit.client.1.smithi148.stdout:1/657: truncate db/dd/f13 7946232 0 2024-09-21T04:15:22.027 INFO:tasks.workunit.client.1.smithi148.stdout:5/705: rmdir d3/d14/d1f 39 2024-09-21T04:15:22.027 INFO:tasks.workunit.client.1.smithi148.stdout:7/683: rmdir d5/d14/d25/d79/d86 39 2024-09-21T04:15:22.029 INFO:tasks.workunit.client.0.smithi110.stdout:6/653: sync 2024-09-21T04:15:22.029 INFO:tasks.workunit.client.0.smithi110.stdout:5/622: sync 2024-09-21T04:15:22.029 INFO:tasks.workunit.client.0.smithi110.stdout:6/654: chown d4/d12/d17/d2e/d38/l8b 351063 1 2024-09-21T04:15:22.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/702: creat d5/d6/d2a/d45/fe5 x:0 0 0 2024-09-21T04:15:22.031 INFO:tasks.workunit.client.0.smithi110.stdout:2/477: dread d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:22.031 INFO:tasks.workunit.client.0.smithi110.stdout:5/623: symlink d1/d49/d4d/d1e/d4a/ld2 0 2024-09-21T04:15:22.034 INFO:tasks.workunit.client.1.smithi148.stdout:1/658: mknod db/d24/cd5 0 2024-09-21T04:15:22.035 INFO:tasks.workunit.client.1.smithi148.stdout:3/640: dwrite d3/d5/d6/d12/d15/d73/f9a [0,4194304] 0 2024-09-21T04:15:22.036 INFO:tasks.workunit.client.0.smithi110.stdout:5/624: rename d1/f8a to d1/d7/d18/d8f/fd3 0 2024-09-21T04:15:22.036 INFO:tasks.workunit.client.0.smithi110.stdout:5/625: truncate d1/d22/fb9 303257 0 2024-09-21T04:15:22.036 INFO:tasks.workunit.client.0.smithi110.stdout:5/626: fsync d1/d49/d4d/f24 0 2024-09-21T04:15:22.037 INFO:tasks.workunit.client.1.smithi148.stdout:5/706: symlink d3/d14/d1f/d26/d2e/lf6 0 2024-09-21T04:15:22.039 INFO:tasks.workunit.client.0.smithi110.stdout:2/478: unlink d2/d1a/d1f/c63 0 2024-09-21T04:15:22.041 INFO:tasks.workunit.client.1.smithi148.stdout:7/684: truncate d5/d8/d6f/d7d/d90/faf 3507257 0 2024-09-21T04:15:22.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/655: write d4/d78/daa/f11 [3578319,87854] 0 2024-09-21T04:15:22.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/656: creat d4/d78/daa/fd1 x:0 0 0 2024-09-21T04:15:22.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/647: rename dc/d13/d3c/d81/d98/dc1/fc2 to dc/d13/d39/fc6 0 2024-09-21T04:15:22.044 INFO:tasks.workunit.client.0.smithi110.stdout:5/627: write d1/d22/f4f [2370269,129993] 0 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/703: dwrite d5/d6/d2a/d2b/f39 [0,4194304] 0 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/704: write d5/d6/d2a/d3f/d8b/fc2 [2518058,126359] 0 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/705: read - d5/d82/fc4 zero size 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/706: dread - d5/d6/d2a/d2b/d84/daf/dc8/fe3 zero size 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.1.smithi148.stdout:0/707: readlink d5/d6/l14 0 2024-09-21T04:15:22.045 INFO:tasks.workunit.client.0.smithi110.stdout:2/479: mknod d2/d81/c8d 0 2024-09-21T04:15:22.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/749: getdents d4/d12/d75/d9c 0 2024-09-21T04:15:22.049 INFO:tasks.workunit.client.1.smithi148.stdout:1/659: truncate db/d24/f26 448494 0 2024-09-21T04:15:22.052 INFO:tasks.workunit.client.0.smithi110.stdout:6/657: write d4/f13 [4684836,119121] 0 2024-09-21T04:15:22.053 INFO:tasks.workunit.client.0.smithi110.stdout:6/658: truncate d4/f6a 151421 0 2024-09-21T04:15:22.053 INFO:tasks.workunit.client.0.smithi110.stdout:5/628: mkdir d1/d49/d4d/d1e/d4a/dd4 0 2024-09-21T04:15:22.057 INFO:tasks.workunit.client.1.smithi148.stdout:3/641: creat d3/d1f/d4d/d82/d23/daf/fd3 x:0 0 0 2024-09-21T04:15:22.062 INFO:tasks.workunit.client.1.smithi148.stdout:1/660: write db/d24/d2e/f7b [3505664,86100] 0 2024-09-21T04:15:22.062 INFO:tasks.workunit.client.1.smithi148.stdout:7/685: creat d5/d14/ff2 x:0 0 0 2024-09-21T04:15:22.065 INFO:tasks.workunit.client.1.smithi148.stdout:9/725: rename d5/d60/d4c/dbd to d5/d2b/d32/d67/d80/d104 0 2024-09-21T04:15:22.065 INFO:tasks.workunit.client.0.smithi110.stdout:2/480: dwrite d2/f16 [4194304,4194304] 0 2024-09-21T04:15:22.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/659: symlink d4/d12/d16/d66/d7a/d54/ld2 0 2024-09-21T04:15:22.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/660: fdatasync d4/d12/d17/d2e/d38/f98 0 2024-09-21T04:15:22.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/661: stat d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca 0 2024-09-21T04:15:22.067 INFO:tasks.workunit.client.1.smithi148.stdout:0/708: dwrite d5/d6/d2a/d2b/d99/db7/fc7 [0,4194304] 0 2024-09-21T04:15:22.067 INFO:tasks.workunit.client.1.smithi148.stdout:0/709: fdatasync d5/d6/d2a/d2b/f2c 0 2024-09-21T04:15:22.070 INFO:tasks.workunit.client.1.smithi148.stdout:8/648: mknod dc/d43/d4c/d4f/cc7 0 2024-09-21T04:15:22.070 INFO:tasks.workunit.client.1.smithi148.stdout:8/649: dread - dc/d13/d3c/d81/d9d/f95 zero size 2024-09-21T04:15:22.070 INFO:tasks.workunit.client.1.smithi148.stdout:8/650: fsync dc/d13/d76/d5d/d78/f9f 0 2024-09-21T04:15:22.071 INFO:tasks.workunit.client.0.smithi110.stdout:2/481: rmdir d2/d1a/d1f/d65 39 2024-09-21T04:15:22.072 INFO:tasks.workunit.client.1.smithi148.stdout:4/750: symlink d4/d12/d75/le8 0 2024-09-21T04:15:22.073 INFO:tasks.workunit.client.0.smithi110.stdout:6/662: mknod d4/d86/d8f/cd3 0 2024-09-21T04:15:22.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/663: read d4/d12/d16/f56 [426575,96163] 0 2024-09-21T04:15:22.074 INFO:tasks.workunit.client.0.smithi110.stdout:5/629: dwrite d1/d7/d18/d3f/f93 [0,4194304] 0 2024-09-21T04:15:22.078 INFO:tasks.workunit.client.1.smithi148.stdout:5/707: rmdir d3/d37 39 2024-09-21T04:15:22.078 INFO:tasks.workunit.client.1.smithi148.stdout:5/708: write d3/fa [753145,45000] 0 2024-09-21T04:15:22.079 INFO:tasks.workunit.client.1.smithi148.stdout:0/710: write d5/d6/d2a/d2b/f39 [2945648,22991] 0 2024-09-21T04:15:22.082 INFO:tasks.workunit.client.0.smithi110.stdout:2/482: rename d2/d1a/d1f/d29/d19/l6a to d2/d81/l8e 0 2024-09-21T04:15:22.083 INFO:tasks.workunit.client.1.smithi148.stdout:3/642: dwrite d3/d1f/d4d/d82/d23/f26 [0,4194304] 0 2024-09-21T04:15:22.086 INFO:tasks.workunit.client.0.smithi110.stdout:2/483: truncate d2/d1a/d1f/d29/f4f 2989907 0 2024-09-21T04:15:22.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/630: write d1/d49/d4d/d1e/d5a/f67 [2205654,9104] 0 2024-09-21T04:15:22.100 INFO:tasks.workunit.client.0.smithi110.stdout:6/664: dwrite d4/d78/daa/f1f [0,4194304] 0 2024-09-21T04:15:22.100 INFO:tasks.workunit.client.1.smithi148.stdout:7/686: dread d5/d8/f1e [0,4194304] 0 2024-09-21T04:15:22.100 INFO:tasks.workunit.client.1.smithi148.stdout:7/687: fsync d5/d8/d64/f96 0 2024-09-21T04:15:22.100 INFO:tasks.workunit.client.1.smithi148.stdout:7/688: fsync d5/d8/d6f/d7d/d90/d28/f87 0 2024-09-21T04:15:22.101 INFO:tasks.workunit.client.1.smithi148.stdout:7/689: fsync d5/d8/d6f/d7d/d90/d28/f29 0 2024-09-21T04:15:22.101 INFO:tasks.workunit.client.1.smithi148.stdout:7/690: creat d5/d8/d6f/ff3 x:0 0 0 2024-09-21T04:15:22.105 INFO:tasks.workunit.client.0.smithi110.stdout:6/665: symlink d4/d78/daa/ld4 0 2024-09-21T04:15:22.105 INFO:tasks.workunit.client.0.smithi110.stdout:6/666: fsync d4/d78/daa/fcd 0 2024-09-21T04:15:22.105 INFO:tasks.workunit.client.0.smithi110.stdout:6/667: chown d4/d12/d17/d81 618848 1 2024-09-21T04:15:22.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/651: creat dc/fc8 x:0 0 0 2024-09-21T04:15:22.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/652: chown dc/d13/d3c/f5f 5719 1 2024-09-21T04:15:22.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/653: write dc/d13/d3c/f9c [300400,24911] 0 2024-09-21T04:15:22.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/654: write f5 [5639032,101676] 0 2024-09-21T04:15:22.107 INFO:tasks.workunit.client.1.smithi148.stdout:4/751: symlink d4/d12/d16/le9 0 2024-09-21T04:15:22.107 INFO:tasks.workunit.client.1.smithi148.stdout:8/655: dread dc/d13/f34 [0,4194304] 0 2024-09-21T04:15:22.108 INFO:tasks.workunit.client.0.smithi110.stdout:6/668: link d4/d12/d17/d81/fcc d4/d12/d16/d66/fd5 0 2024-09-21T04:15:22.109 INFO:tasks.workunit.client.0.smithi110.stdout:6/669: dread d4/d78/daa/f84 [0,4194304] 0 2024-09-21T04:15:22.110 INFO:tasks.workunit.client.1.smithi148.stdout:5/709: unlink d3/d14/d1f/d26/d2e/d3d/f61 0 2024-09-21T04:15:22.112 INFO:tasks.workunit.client.1.smithi148.stdout:1/661: unlink db/dd/l5e 0 2024-09-21T04:15:22.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/643: dwrite d3/d1f/d4d/d82/d23/f26 [4194304,4194304] 0 2024-09-21T04:15:22.114 INFO:tasks.workunit.client.0.smithi110.stdout:6/670: symlink d4/d12/d16/d6d/ld6 0 2024-09-21T04:15:22.114 INFO:tasks.workunit.client.0.smithi110.stdout:6/671: fdatasync d4/d12/d16/d66/d7a/d83/f9f 0 2024-09-21T04:15:22.114 INFO:tasks.workunit.client.0.smithi110.stdout:2/484: dwrite d2/d1a/d1f/d5f/f58 [0,4194304] 0 2024-09-21T04:15:22.115 INFO:tasks.workunit.client.0.smithi110.stdout:6/672: symlink d4/d12/d17/d81/d90/dc3/ld7 0 2024-09-21T04:15:22.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/726: getdents d5/d39/dba 0 2024-09-21T04:15:22.119 INFO:tasks.workunit.client.0.smithi110.stdout:2/485: unlink d2/d1a/d1f/d65/c89 0 2024-09-21T04:15:22.119 INFO:tasks.workunit.client.0.smithi110.stdout:2/486: fsync d2/d1a/d1f/d2a/f61 0 2024-09-21T04:15:22.121 INFO:tasks.workunit.client.1.smithi148.stdout:7/691: mknod d5/d8/d6f/d7d/cf4 0 2024-09-21T04:15:22.132 INFO:tasks.workunit.client.0.smithi110.stdout:2/487: write d2/d1a/d1f/d5f/f24 [2724087,109396] 0 2024-09-21T04:15:22.133 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:21 smithi148 ceph-mon[34957]: mgrmap e26: smithi148.zjrahi(active, since 9s), standbys: smithi110.axmqzb 2024-09-21T04:15:22.133 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:21 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:15:22.145 INFO:tasks.workunit.client.1.smithi148.stdout:4/752: creat d4/d66/fea x:0 0 0 2024-09-21T04:15:22.146 INFO:tasks.workunit.client.0.smithi110.stdout:5/631: dwrite d1/d22/f27 [0,4194304] 0 2024-09-21T04:15:22.146 INFO:tasks.workunit.client.1.smithi148.stdout:8/656: symlink dc/d13/d3c/d81/db0/lc9 0 2024-09-21T04:15:22.149 INFO:tasks.workunit.client.1.smithi148.stdout:1/662: mknod db/d46/cd6 0 2024-09-21T04:15:22.149 INFO:tasks.workunit.client.1.smithi148.stdout:1/663: stat db/dd/d21/db0 0 2024-09-21T04:15:22.150 INFO:tasks.workunit.client.1.smithi148.stdout:7/692: dwrite d5/d8/d6f/d7d/d90/f10 [4194304,4194304] 0 2024-09-21T04:15:22.150 INFO:tasks.workunit.client.0.smithi110.stdout:2/488: chown d2/d1a/f88 53118592 1 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/710: read d3/d34/d3c/d5d/f93 [1984677,127134] 0 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/711: chown d3/d14/d1f/d26/d2e/d56/f8a 16461 1 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/712: chown d3/d4a 221345 1 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/713: getdents d3/d14/d1f/d26/d59/d97 0 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:3/644: symlink d3/d5/d6/d2b/d96/ld4 0 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:3/645: readlink d3/d1f/d4d/d82/d52/lcb 0 2024-09-21T04:15:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:3/646: fsync d3/d1f/d4d/d82/d23/d28/d46/f5b 0 2024-09-21T04:15:22.152 INFO:tasks.workunit.client.0.smithi110.stdout:2/489: dread d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:22.152 INFO:tasks.workunit.client.0.smithi110.stdout:2/490: read d2/d1a/f88 [144102,57712] 0 2024-09-21T04:15:22.152 INFO:tasks.workunit.client.1.smithi148.stdout:4/753: symlink d4/d87/leb 0 2024-09-21T04:15:22.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/647: dread d3/d1f/d4d/d82/d74/fa0 [0,4194304] 0 2024-09-21T04:15:22.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/648: truncate d3/f85 1699423 0 2024-09-21T04:15:22.156 INFO:tasks.workunit.client.1.smithi148.stdout:9/727: write d5/d1e/d37/f64 [1095162,65641] 0 2024-09-21T04:15:22.156 INFO:tasks.workunit.client.1.smithi148.stdout:9/728: chown d5/d1e/d37/d46/fac 975333 1 2024-09-21T04:15:22.156 INFO:tasks.workunit.client.0.smithi110.stdout:6/673: dwrite d4/d12/d17/d2e/d38/f4f [0,4194304] 0 2024-09-21T04:15:22.157 INFO:tasks.workunit.client.1.smithi148.stdout:7/693: mkdir d5/d8/d6f/d7d/df5 0 2024-09-21T04:15:22.158 INFO:tasks.workunit.client.0.smithi110.stdout:5/632: mkdir d1/d49/d4d/d1e/d4a/dd4/dd5 0 2024-09-21T04:15:22.158 INFO:tasks.workunit.client.0.smithi110.stdout:5/633: fdatasync d1/d49/d6f/fa8 0 2024-09-21T04:15:22.158 INFO:tasks.workunit.client.1.smithi148.stdout:1/664: write db/f15 [2452668,3579] 0 2024-09-21T04:15:22.164 INFO:tasks.workunit.client.0.smithi110.stdout:6/674: creat d4/d12/d17/d2e/d40/fd8 x:0 0 0 2024-09-21T04:15:22.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:21 smithi110 ceph-mon[27723]: mgrmap e26: smithi148.zjrahi(active, since 9s), standbys: smithi110.axmqzb 2024-09-21T04:15:22.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:21 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:15:22.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/634: rmdir d1/d39 39 2024-09-21T04:15:22.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/635: fsync d1/d22/f97 0 2024-09-21T04:15:22.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/636: write d1/d7/d18/d3f/f86 [751439,58462] 0 2024-09-21T04:15:22.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/637: fdatasync d1/d49/d4d/d1e/d5a/f7a 0 2024-09-21T04:15:22.169 INFO:tasks.workunit.client.1.smithi148.stdout:7/694: read d5/d14/d25/f52 [2895709,80252] 0 2024-09-21T04:15:22.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/754: getdents d4/d66/d7f 0 2024-09-21T04:15:22.173 INFO:tasks.workunit.client.1.smithi148.stdout:3/649: mknod d3/d1f/d4d/d82/cd5 0 2024-09-21T04:15:22.173 INFO:tasks.workunit.client.1.smithi148.stdout:3/650: read - d3/d5/d6/d2b/da2/da3/fb2 zero size 2024-09-21T04:15:22.175 INFO:tasks.workunit.client.1.smithi148.stdout:4/755: mkdir d4/d12/d16/d29/d61/d6c/d9d/daf/dec 0 2024-09-21T04:15:22.175 INFO:tasks.workunit.client.1.smithi148.stdout:3/651: dread d3/d1f/d4d/d82/f11 [0,4194304] 0 2024-09-21T04:15:22.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/652: mkdir d3/d5/d6/d12/d43/d54/d9b/dd6 0 2024-09-21T04:15:22.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/653: stat d3/d1f/d4d/d82 0 2024-09-21T04:15:22.177 INFO:tasks.workunit.client.1.smithi148.stdout:3/654: fdatasync d3/fc0 0 2024-09-21T04:15:22.180 INFO:tasks.workunit.client.1.smithi148.stdout:8/657: write dc/d13/d76/f2c [6186216,87265] 0 2024-09-21T04:15:22.183 INFO:tasks.workunit.client.1.smithi148.stdout:5/714: write d3/d14/d1f/d26/d2e/d56/f73 [1569432,70067] 0 2024-09-21T04:15:22.183 INFO:tasks.workunit.client.0.smithi110.stdout:6/675: write d4/d12/d17/d2e/f47 [3606941,39306] 0 2024-09-21T04:15:22.184 INFO:tasks.workunit.client.1.smithi148.stdout:8/658: truncate dc/f46 6649153 0 2024-09-21T04:15:22.184 INFO:tasks.workunit.client.1.smithi148.stdout:5/715: creat d3/d34/ff7 x:0 0 0 2024-09-21T04:15:22.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/659: creat dc/d13/d3c/d81/d85/da1/fca x:0 0 0 2024-09-21T04:15:22.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/660: chown dc/d43/c64 259151018 1 2024-09-21T04:15:22.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/661: getdents dc/d13/d76/d50/d80 0 2024-09-21T04:15:22.187 INFO:tasks.workunit.client.1.smithi148.stdout:5/716: rename d3/d14/d1f/c51 to d3/d37/cf8 0 2024-09-21T04:15:22.187 INFO:tasks.workunit.client.1.smithi148.stdout:5/717: write d3/d34/f64 [310055,53995] 0 2024-09-21T04:15:22.187 INFO:tasks.workunit.client.1.smithi148.stdout:3/655: read d3/d1f/d4d/d82/d23/d28/d46/f9f [768471,69650] 0 2024-09-21T04:15:22.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/656: chown d3/d5/d6/d2b/l56 7540 1 2024-09-21T04:15:22.188 INFO:tasks.workunit.client.1.smithi148.stdout:5/718: symlink d3/d34/d3c/d5d/d75/lf9 0 2024-09-21T04:15:22.191 INFO:tasks.workunit.client.0.smithi110.stdout:6/676: write d4/d12/f2b [67553,598] 0 2024-09-21T04:15:22.196 INFO:tasks.workunit.client.1.smithi148.stdout:8/662: dread dc/d43/f54 [0,4194304] 0 2024-09-21T04:15:22.197 INFO:tasks.workunit.client.1.smithi148.stdout:5/719: dread d3/d34/d78/dbf/f4b [0,4194304] 0 2024-09-21T04:15:22.201 INFO:tasks.workunit.client.1.smithi148.stdout:9/729: dwrite d5/f63 [0,4194304] 0 2024-09-21T04:15:22.201 INFO:tasks.workunit.client.1.smithi148.stdout:9/730: write d5/d2b/dce/fdd [767112,65229] 0 2024-09-21T04:15:22.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/665: dwrite db/d46/d8f/fbb [0,4194304] 0 2024-09-21T04:15:22.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/756: dwrite d4/d87/f5c [0,4194304] 0 2024-09-21T04:15:22.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/757: fdatasync d4/d12/d78/f8c 0 2024-09-21T04:15:22.202 INFO:tasks.workunit.client.0.smithi110.stdout:2/491: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:22.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/720: dread d3/d34/d3c/d5d/f93 [0,4194304] 0 2024-09-21T04:15:22.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/721: write d3/d14/d1a/f4d [1717371,96916] 0 2024-09-21T04:15:22.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/722: fsync d3/d14/d1f/d26/d2e/d56/fc3 0 2024-09-21T04:15:22.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/723: read d3/d34/d78/fa1 [67670,112946] 0 2024-09-21T04:15:22.221 INFO:tasks.workunit.client.0.smithi110.stdout:2/492: dwrite d2/d1a/d1f/d5f/f24 [4194304,4194304] 0 2024-09-21T04:15:22.248 INFO:tasks.workunit.client.0.smithi110.stdout:9/547: sync 2024-09-21T04:15:22.248 INFO:tasks.workunit.client.0.smithi110.stdout:9/548: write d7/d96/d99/d3c/d1f/f23 [469047,99538] 0 2024-09-21T04:15:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/657: truncate d3/d1f/d4d/d82/d23/db5/fd0 2118600 0 2024-09-21T04:15:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/658: chown d3/d5/db4 424 1 2024-09-21T04:15:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/659: chown d3/f85 102012 1 2024-09-21T04:15:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/660: fsync d3/d1f/f47 0 2024-09-21T04:15:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/661: read - d3/d5/d6/d2b/da2/fc8 zero size 2024-09-21T04:15:22.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/663: symlink dc/d43/d4c/lcb 0 2024-09-21T04:15:22.264 INFO:tasks.workunit.client.0.smithi110.stdout:6/677: creat d4/d12/d17/d81/d90/fd9 x:0 0 0 2024-09-21T04:15:22.264 INFO:tasks.workunit.client.0.smithi110.stdout:6/678: fsync d4/d12/d17/d33/d3e/f53 0 2024-09-21T04:15:22.267 INFO:tasks.workunit.client.1.smithi148.stdout:9/731: unlink d5/d2b/dce/fdd 0 2024-09-21T04:15:22.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/493: symlink d2/d50/d7b/l8f 0 2024-09-21T04:15:22.272 INFO:tasks.workunit.client.1.smithi148.stdout:6/690: sync 2024-09-21T04:15:22.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/711: sync 2024-09-21T04:15:22.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/832: sync 2024-09-21T04:15:22.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/712: truncate d5/d6/d2a/d2b/d84/db4/fe1 980513 0 2024-09-21T04:15:22.273 INFO:tasks.workunit.client.1.smithi148.stdout:6/691: write d0/d8/f10 [173983,128739] 0 2024-09-21T04:15:22.273 INFO:tasks.workunit.client.1.smithi148.stdout:6/692: write d0/d1a/d23/d32/d93/d96/fae [188114,43213] 0 2024-09-21T04:15:22.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/666: rename db/dd/l2c to db/d46/da0/ld7 0 2024-09-21T04:15:22.282 INFO:tasks.workunit.client.0.smithi110.stdout:9/549: creat d7/d10/d88/d8f/fb8 x:0 0 0 2024-09-21T04:15:22.283 INFO:tasks.workunit.client.0.smithi110.stdout:9/550: dread d7/d96/d99/d3c/d4f/f64 [0,4194304] 0 2024-09-21T04:15:22.284 INFO:tasks.workunit.client.0.smithi110.stdout:6/679: rename d4/d86/d8f/f9b to d4/d78/daa/fda 0 2024-09-21T04:15:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:9/551: creat d7/d96/d99/d3c/d4f/d9f/d6f/d78/fb9 x:0 0 0 2024-09-21T04:15:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:9/552: write d7/d96/d99/d3c/d35/f3a [2081951,130930] 0 2024-09-21T04:15:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:9/553: readlink d7/d96/d99/d3c/d1f/l48 0 2024-09-21T04:15:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:9/554: stat d7/d10/d88/fac 0 2024-09-21T04:15:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:9/555: readlink d7/d96/d99/d3c/d14/la5 0 2024-09-21T04:15:22.293 INFO:tasks.workunit.client.1.smithi148.stdout:3/662: link d3/d5/d5c/fc4 d3/d5/d6/d12/d15/da7/fd7 0 2024-09-21T04:15:22.294 INFO:tasks.workunit.client.1.smithi148.stdout:8/664: creat dc/d13/d3c/d81/d85/fcc x:0 0 0 2024-09-21T04:15:22.294 INFO:tasks.workunit.client.1.smithi148.stdout:2/833: mknod d0/d9/dc/d13/daf/db2/ded/c108 0 2024-09-21T04:15:22.296 INFO:tasks.workunit.client.1.smithi148.stdout:6/693: symlink d0/d8/dba/lcd 0 2024-09-21T04:15:22.296 INFO:tasks.workunit.client.1.smithi148.stdout:1/667: rename db/d46/d4c/d82/f90 to db/d24/d2e/d59/da7/fd8 0 2024-09-21T04:15:22.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/834: read d0/f23 [4529030,85931] 0 2024-09-21T04:15:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:4/758: getdents d4/d66 0 2024-09-21T04:15:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:5/724: rmdir d3/d24/dad 39 2024-09-21T04:15:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:4/759: chown d4/d12/d75/le8 158 1 2024-09-21T04:15:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:8/665: creat dc/d43/d4c/fcd x:0 0 0 2024-09-21T04:15:22.300 INFO:tasks.workunit.client.1.smithi148.stdout:9/732: dwrite d5/f23 [4194304,4194304] 0 2024-09-21T04:15:22.300 INFO:tasks.workunit.client.1.smithi148.stdout:9/733: chown d5/d39/c84 99942 1 2024-09-21T04:15:22.307 INFO:tasks.workunit.client.1.smithi148.stdout:0/713: dwrite d5/d6/d2a/f79 [0,4194304] 0 2024-09-21T04:15:22.307 INFO:tasks.workunit.client.1.smithi148.stdout:0/714: dread - d5/d6/d2a/fd7 zero size 2024-09-21T04:15:22.307 INFO:tasks.workunit.client.1.smithi148.stdout:0/715: write d5/d6/d2a/d2b/d84/db4/fb5 [2284184,21764] 0 2024-09-21T04:15:22.308 INFO:tasks.workunit.client.0.smithi110.stdout:9/556: dwrite d7/d96/d99/d3c/d1f/d43/f6e [0,4194304] 0 2024-09-21T04:15:22.308 INFO:tasks.workunit.client.0.smithi110.stdout:9/557: write d7/d96/d99/d3c/f5a [84912,58185] 0 2024-09-21T04:15:22.308 INFO:tasks.workunit.client.0.smithi110.stdout:2/494: dwrite d2/d1a/d1f/d5f/f6f [0,4194304] 0 2024-09-21T04:15:22.312 INFO:tasks.workunit.client.0.smithi110.stdout:9/558: creat d7/d96/d99/d3c/d1f/d43/db5/fba x:0 0 0 2024-09-21T04:15:22.312 INFO:tasks.workunit.client.0.smithi110.stdout:9/559: write d7/d96/d99/d3c/d1f/d43/fb2 [596969,39098] 0 2024-09-21T04:15:22.312 INFO:tasks.workunit.client.0.smithi110.stdout:9/560: read - d7/d96/d99/d3c/d4f/fa1 zero size 2024-09-21T04:15:22.313 INFO:tasks.workunit.client.1.smithi148.stdout:1/668: dread db/d24/f8b [0,4194304] 0 2024-09-21T04:15:22.313 INFO:tasks.workunit.client.1.smithi148.stdout:1/669: stat db/d24/cd5 0 2024-09-21T04:15:22.313 INFO:tasks.workunit.client.1.smithi148.stdout:1/670: truncate db/d46/f48 4467033 0 2024-09-21T04:15:22.316 INFO:tasks.workunit.client.1.smithi148.stdout:0/716: unlink d5/f85 0 2024-09-21T04:15:22.316 INFO:tasks.workunit.client.1.smithi148.stdout:1/671: link db/d24/c79 db/d46/d4c/d82/cd9 0 2024-09-21T04:15:22.319 INFO:tasks.workunit.client.0.smithi110.stdout:2/495: unlink d2/d50/d56/f53 0 2024-09-21T04:15:22.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/717: mkdir d5/dcb/de6 0 2024-09-21T04:15:22.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/672: truncate db/dd/d21/f7a 2256088 0 2024-09-21T04:15:22.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/673: write db/ff [750911,129328] 0 2024-09-21T04:15:22.321 INFO:tasks.workunit.client.0.smithi110.stdout:2/496: mkdir d2/d1a/d90 0 2024-09-21T04:15:22.322 INFO:tasks.workunit.client.1.smithi148.stdout:0/718: symlink d5/d6/d7/dd0/le7 0 2024-09-21T04:15:22.323 INFO:tasks.workunit.client.1.smithi148.stdout:1/674: unlink db/c52 0 2024-09-21T04:15:22.323 INFO:tasks.workunit.client.1.smithi148.stdout:4/760: dread d4/d3a/f53 [0,4194304] 0 2024-09-21T04:15:22.323 INFO:tasks.workunit.client.1.smithi148.stdout:0/719: rmdir d5/dcb 39 2024-09-21T04:15:22.325 INFO:tasks.workunit.client.1.smithi148.stdout:4/761: creat d4/d12/fed x:0 0 0 2024-09-21T04:15:22.325 INFO:tasks.workunit.client.1.smithi148.stdout:4/762: chown d4/d87/fd5 76580 1 2024-09-21T04:15:22.326 INFO:tasks.workunit.client.1.smithi148.stdout:0/720: mkdir d5/d6/d2a/d2b/d84/daf/dc8/de8 0 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.1.smithi148.stdout:4/763: symlink d4/d12/d16/lee 0 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.1.smithi148.stdout:4/764: write d4/d12/d75/d85/f99 [423600,59432] 0 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.1.smithi148.stdout:4/765: fdatasync d4/d87/d91/fcf 0 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.1.smithi148.stdout:4/766: chown d4/d12/c18 4880 1 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/721: unlink d5/d6/d7/c10 0 2024-09-21T04:15:22.328 INFO:tasks.workunit.client.0.smithi110.stdout:0/937: sync 2024-09-21T04:15:22.329 INFO:tasks.workunit.client.0.smithi110.stdout:4/739: sync 2024-09-21T04:15:22.329 INFO:tasks.workunit.client.0.smithi110.stdout:0/938: chown d4/d1c 0 1 2024-09-21T04:15:22.330 INFO:tasks.workunit.client.0.smithi110.stdout:0/939: rename d4/f11 to d4/dd/d53/f11c 0 2024-09-21T04:15:22.332 INFO:tasks.workunit.client.1.smithi148.stdout:5/725: write d3/f12 [1748566,112215] 0 2024-09-21T04:15:22.332 INFO:tasks.workunit.client.1.smithi148.stdout:8/666: write dc/d13/f35 [1824103,19290] 0 2024-09-21T04:15:22.333 INFO:tasks.workunit.client.1.smithi148.stdout:4/767: rename d4/d12/d16/d29/d61/d6c/le6 to d4/d12/lef 0 2024-09-21T04:15:22.334 INFO:tasks.workunit.client.0.smithi110.stdout:0/940: mknod d4/dd/d53/df4/c11d 0 2024-09-21T04:15:22.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/726: creat d3/d14/d1f/d72/ffa x:0 0 0 2024-09-21T04:15:22.338 INFO:tasks.workunit.client.1.smithi148.stdout:4/768: rename d4/d12/d16/d29/d61/d6c/d9d/daf to d4/d12/d16/df0 0 2024-09-21T04:15:22.339 INFO:tasks.workunit.client.0.smithi110.stdout:0/941: mknod d4/d47/c11e 0 2024-09-21T04:15:22.341 INFO:tasks.workunit.client.1.smithi148.stdout:5/727: rename d3/f36 to d3/d14/d1f/d26/d2e/ffb 0 2024-09-21T04:15:22.341 INFO:tasks.workunit.client.1.smithi148.stdout:5/728: stat d3/d14/d1f/c2c 0 2024-09-21T04:15:22.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/695: sync 2024-09-21T04:15:22.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/696: creat d5/d14/d25/d40/db9/ff6 x:0 0 0 2024-09-21T04:15:22.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/697: write d5/d8/ff [2123284,19878] 0 2024-09-21T04:15:22.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/729: mkdir d3/d14/d1f/dfc 0 2024-09-21T04:15:22.345 INFO:tasks.workunit.client.1.smithi148.stdout:5/730: unlink d3/fe7 0 2024-09-21T04:15:22.345 INFO:tasks.workunit.client.1.smithi148.stdout:5/731: write d3/d14/d1f/d26/d2e/d56/fc3 [849779,70161] 0 2024-09-21T04:15:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:4/769: read d4/d87/f89 [3704727,36946] 0 2024-09-21T04:15:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:4/770: chown d4/d12/d83/fa6 135648281 1 2024-09-21T04:15:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:4/771: chown d4/d12/d16/d29/dbd/f8a 4760238 1 2024-09-21T04:15:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:5/732: truncate d3/d14/d1f/d26/d2e/d56/f15 1229465 0 2024-09-21T04:15:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:5/733: creat d3/d24/ffd x:0 0 0 2024-09-21T04:15:22.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/734: creat d3/d14/d1f/d26/d2e/ddb/ffe x:0 0 0 2024-09-21T04:15:22.372 INFO:tasks.workunit.client.1.smithi148.stdout:8/667: dread dc/d13/d3c/f49 [0,4194304] 0 2024-09-21T04:15:22.380 INFO:tasks.workunit.client.1.smithi148.stdout:3/663: dwrite d3/d5/d5c/fc3 [0,4194304] 0 2024-09-21T04:15:22.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/664: mknod d3/d5/d6/d12/d43/d54/d9b/dd6/cd8 0 2024-09-21T04:15:22.386 INFO:tasks.workunit.client.1.smithi148.stdout:3/665: creat d3/d1f/d4d/d82/d23/d28/fd9 x:0 0 0 2024-09-21T04:15:22.389 INFO:tasks.workunit.client.1.smithi148.stdout:3/666: creat d3/d5/d6/d12/d43/d54/d9b/fda x:0 0 0 2024-09-21T04:15:22.389 INFO:tasks.workunit.client.1.smithi148.stdout:3/667: stat d3/d1f/d4d/d82/d23/da1/cc7 0 2024-09-21T04:15:22.391 INFO:tasks.workunit.client.1.smithi148.stdout:5/735: dread d3/d24/f76 [0,4194304] 0 2024-09-21T04:15:22.391 INFO:tasks.workunit.client.0.smithi110.stdout:4/740: dread d17/d1e/d64/f9d [0,4194304] 0 2024-09-21T04:15:22.394 INFO:tasks.workunit.client.1.smithi148.stdout:5/736: rename d3/d14/d1f/d26/d2e/d56/f17 to d3/d24/dad/fff 0 2024-09-21T04:15:22.398 INFO:tasks.workunit.client.0.smithi110.stdout:9/561: dwrite d7/d96/d99/d3c/d4f/fa1 [0,4194304] 0 2024-09-21T04:15:22.402 INFO:tasks.workunit.client.0.smithi110.stdout:9/562: write d7/d96/d99/f2a [4940863,36370] 0 2024-09-21T04:15:22.402 INFO:tasks.workunit.client.0.smithi110.stdout:4/741: dread d17/d34/d3d/d57/f5a [0,4194304] 0 2024-09-21T04:15:22.402 INFO:tasks.workunit.client.0.smithi110.stdout:4/742: rename d17/d34 to d17/d34/d3d/d57/df7 22 2024-09-21T04:15:22.403 INFO:tasks.workunit.client.0.smithi110.stdout:9/563: mknod d7/d96/d99/d3c/d35/d71/cbb 0 2024-09-21T04:15:22.405 INFO:tasks.workunit.client.0.smithi110.stdout:4/743: mknod d17/d3e/d6b/dc7/cf8 0 2024-09-21T04:15:22.407 INFO:tasks.workunit.client.0.smithi110.stdout:9/564: creat d7/d96/d99/d3c/d1f/d72/d59/fbc x:0 0 0 2024-09-21T04:15:22.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/772: dwrite d4/d12/d83/fa6 [4194304,4194304] 0 2024-09-21T04:15:22.408 INFO:tasks.workunit.client.0.smithi110.stdout:9/565: symlink d7/d10/d88/d8f/db7/lbd 0 2024-09-21T04:15:22.408 INFO:tasks.workunit.client.0.smithi110.stdout:9/566: fsync f2 0 2024-09-21T04:15:22.420 INFO:tasks.workunit.client.1.smithi148.stdout:6/694: dwrite d0/d1a/d23/dc/d68/f86 [4194304,4194304] 0 2024-09-21T04:15:22.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/695: creat d0/d34/d38/dc6/fce x:0 0 0 2024-09-21T04:15:22.423 INFO:tasks.workunit.client.1.smithi148.stdout:6/696: write d0/d1a/d23/d32/d93/d96/fae [536981,93836] 0 2024-09-21T04:15:22.425 INFO:tasks.workunit.client.0.smithi110.stdout:2/497: dwrite d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:22.427 INFO:tasks.workunit.client.1.smithi148.stdout:6/697: dread d0/d1a/d23/d27/d80/f94 [0,4194304] 0 2024-09-21T04:15:22.427 INFO:tasks.workunit.client.1.smithi148.stdout:1/675: dwrite db/d24/d2e/fa1 [0,4194304] 0 2024-09-21T04:15:22.427 INFO:tasks.workunit.client.1.smithi148.stdout:1/676: stat db/dd/d21/f7a 0 2024-09-21T04:15:22.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/498: creat d2/d50/d7b/f91 x:0 0 0 2024-09-21T04:15:22.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/499: dread - d2/d50/d7b/f91 zero size 2024-09-21T04:15:22.429 INFO:tasks.workunit.client.0.smithi110.stdout:2/500: mknod d2/d1a/d1f/c92 0 2024-09-21T04:15:22.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/668: dwrite dc/d13/d76/d42/f71 [0,4194304] 0 2024-09-21T04:15:22.440 INFO:tasks.workunit.client.0.smithi110.stdout:0/942: dwrite d4/dd/d10/d29/d5f/dc3/f10a [0,4194304] 0 2024-09-21T04:15:22.440 INFO:tasks.workunit.client.1.smithi148.stdout:7/698: dwrite d5/d14/d25/f54 [0,4194304] 0 2024-09-21T04:15:22.440 INFO:tasks.workunit.client.1.smithi148.stdout:7/699: readlink d5/d8/d6f/d7d/l4d 0 2024-09-21T04:15:22.442 INFO:tasks.workunit.client.1.smithi148.stdout:8/669: creat dc/d43/d4c/d4f/fce x:0 0 0 2024-09-21T04:15:22.444 INFO:tasks.workunit.client.1.smithi148.stdout:8/670: creat dc/d43/d4c/d4f/dc5/fcf x:0 0 0 2024-09-21T04:15:22.445 INFO:tasks.workunit.client.1.smithi148.stdout:8/671: mkdir dc/d13/d76/d50/dd0 0 2024-09-21T04:15:22.447 INFO:tasks.workunit.client.1.smithi148.stdout:1/677: write db/dd/f23 [3489583,84205] 0 2024-09-21T04:15:22.448 INFO:tasks.workunit.client.1.smithi148.stdout:1/678: mknod db/d46/da0/cda 0 2024-09-21T04:15:22.449 INFO:tasks.workunit.client.1.smithi148.stdout:6/698: read d0/d1a/d23/f12 [2763655,119305] 0 2024-09-21T04:15:22.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/679: rename db/dd/c4a to db/dd/d21/d89/dc9/cdb 0 2024-09-21T04:15:22.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/680: truncate db/dd/fc3 484849 0 2024-09-21T04:15:22.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/681: read - db/d24/fd0 zero size 2024-09-21T04:15:22.450 INFO:tasks.workunit.client.1.smithi148.stdout:1/682: write db/d46/f53 [1503382,123350] 0 2024-09-21T04:15:22.451 INFO:tasks.workunit.client.1.smithi148.stdout:1/683: fsync db/d24/d2e/d71/f8e 0 2024-09-21T04:15:22.451 INFO:tasks.workunit.client.1.smithi148.stdout:1/684: mkdir db/dd/db9/ddc 0 2024-09-21T04:15:22.453 INFO:tasks.workunit.client.1.smithi148.stdout:5/737: dwrite d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:22.453 INFO:tasks.workunit.client.1.smithi148.stdout:5/738: truncate d3/d34/d78/fa1 890897 0 2024-09-21T04:15:22.454 INFO:tasks.workunit.client.1.smithi148.stdout:4/773: dwrite d4/d12/d16/d29/d61/d7e/fa0 [0,4194304] 0 2024-09-21T04:15:22.456 INFO:tasks.workunit.client.1.smithi148.stdout:5/739: rename d3/d14/l3a to d3/d14/d1f/db1/l100 0 2024-09-21T04:15:22.456 INFO:tasks.workunit.client.0.smithi110.stdout:2/501: dwrite d2/d50/f7f [0,4194304] 0 2024-09-21T04:15:22.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/744: dwrite d17/d34/d3d/d20/d43/f8c [0,4194304] 0 2024-09-21T04:15:22.460 INFO:tasks.workunit.client.1.smithi148.stdout:4/774: mkdir d4/d12/d16/d29/d61/d6c/df1 0 2024-09-21T04:15:22.460 INFO:tasks.workunit.client.1.smithi148.stdout:4/775: write d4/d12/d16/d29/f46 [4579069,64523] 0 2024-09-21T04:15:22.460 INFO:tasks.workunit.client.1.smithi148.stdout:4/776: truncate d4/d12/d16/fcb 822625 0 2024-09-21T04:15:22.462 INFO:tasks.workunit.client.1.smithi148.stdout:3/668: dwrite d3/d1f/d4d/d82/d21/fd2 [0,4194304] 0 2024-09-21T04:15:22.465 INFO:tasks.workunit.client.0.smithi110.stdout:9/567: dwrite d7/d96/d99/d3c/d11/f4e [0,4194304] 0 2024-09-21T04:15:22.465 INFO:tasks.workunit.client.1.smithi148.stdout:7/700: read d5/d8/f1e [1629379,50960] 0 2024-09-21T04:15:22.465 INFO:tasks.workunit.client.0.smithi110.stdout:2/502: dread d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:22.474 INFO:tasks.workunit.client.0.smithi110.stdout:4/745: symlink d17/d34/d3d/d20/d43/d51/d7a/d86/lf9 0 2024-09-21T04:15:22.478 INFO:tasks.workunit.client.1.smithi148.stdout:4/777: dread d4/d66/d7f/d2a/f36 [0,4194304] 0 2024-09-21T04:15:22.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/740: dread d3/f58 [0,4194304] 0 2024-09-21T04:15:22.478 INFO:tasks.workunit.client.1.smithi148.stdout:4/778: dread d4/d12/d16/fcb [0,4194304] 0 2024-09-21T04:15:22.478 INFO:tasks.workunit.client.1.smithi148.stdout:4/779: write d4/d87/fba [464472,75805] 0 2024-09-21T04:15:22.479 INFO:tasks.workunit.client.0.smithi110.stdout:2/503: rename d2/l70 to d2/l93 0 2024-09-21T04:15:22.484 INFO:tasks.workunit.client.0.smithi110.stdout:4/746: creat d17/d34/d3d/d20/d43/d51/d7a/d96/ddf/ffa x:0 0 0 2024-09-21T04:15:22.488 INFO:tasks.workunit.client.0.smithi110.stdout:4/747: rmdir d17/d1e 39 2024-09-21T04:15:22.488 INFO:tasks.workunit.client.0.smithi110.stdout:4/748: truncate d17/d34/d3d/d20/d43/d51/d7a/d96/fdb 803295 0 2024-09-21T04:15:22.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/701: getdents d5/d14/d25/d79 0 2024-09-21T04:15:22.497 INFO:tasks.workunit.client.1.smithi148.stdout:6/699: dwrite d0/d34/d38/d57/f75 [0,4194304] 0 2024-09-21T04:15:22.500 INFO:tasks.workunit.client.1.smithi148.stdout:6/700: getdents d0/d34/d38/d57/d58 0 2024-09-21T04:15:22.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/701: symlink d0/d1a/d23/d27/lcf 0 2024-09-21T04:15:22.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/702: stat d0/d1a/d23/d27/d62/dc4 0 2024-09-21T04:15:22.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/703: chown d0/d59 24035 1 2024-09-21T04:15:22.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/704: chown d0/d8/d18/f25 48 1 2024-09-21T04:15:22.502 INFO:tasks.workunit.client.1.smithi148.stdout:6/705: symlink d0/d1a/d23/d27/dc2/ld0 0 2024-09-21T04:15:22.505 INFO:tasks.workunit.client.0.smithi110.stdout:3/525: sync 2024-09-21T04:15:22.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/685: dwrite db/d24/d2a/f45 [0,4194304] 0 2024-09-21T04:15:22.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/686: write db/d46/f8d [590530,104377] 0 2024-09-21T04:15:22.506 INFO:tasks.workunit.client.0.smithi110.stdout:0/943: dwrite d4/dd/d10/f13 [0,4194304] 0 2024-09-21T04:15:22.506 INFO:tasks.workunit.client.0.smithi110.stdout:0/944: chown d4/dd/d53/feb 37719 1 2024-09-21T04:15:22.506 INFO:tasks.workunit.client.0.smithi110.stdout:0/945: readlink d4/dd/l17 0 2024-09-21T04:15:22.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/706: dread d0/d34/d97/d4f/f8c [0,4194304] 0 2024-09-21T04:15:22.511 INFO:tasks.workunit.client.1.smithi148.stdout:5/741: getdents d3/d14 0 2024-09-21T04:15:22.511 INFO:tasks.workunit.client.1.smithi148.stdout:5/742: truncate d3/d14/d1f/d26/d59/fb2 1818612 0 2024-09-21T04:15:22.512 INFO:tasks.workunit.client.0.smithi110.stdout:3/526: getdents d0/d51 0 2024-09-21T04:15:22.513 INFO:tasks.workunit.client.1.smithi148.stdout:1/687: mkdir db/dd/ddd 0 2024-09-21T04:15:22.513 INFO:tasks.workunit.client.0.smithi110.stdout:0/946: dread d4/d1c/f3c [0,4194304] 0 2024-09-21T04:15:22.514 INFO:tasks.workunit.client.0.smithi110.stdout:0/947: write d4/dd/f8e [2407868,32578] 0 2024-09-21T04:15:22.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/743: unlink d3/d14/d1f/d26/d2e/d56/lcf 0 2024-09-21T04:15:22.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/744: write d3/d14/d1a/f9a [4791810,89035] 0 2024-09-21T04:15:22.516 INFO:tasks.workunit.client.0.smithi110.stdout:0/948: unlink d4/dd/d60/lf6 0 2024-09-21T04:15:22.516 INFO:tasks.workunit.client.0.smithi110.stdout:0/949: chown d4/dd/d10/d29/fec 2123 1 2024-09-21T04:15:22.517 INFO:tasks.workunit.client.0.smithi110.stdout:0/950: mkdir d4/dd/d10/d29/d11f 0 2024-09-21T04:15:22.526 INFO:tasks.workunit.client.1.smithi148.stdout:4/780: dwrite d4/d87/d91/fb2 [0,4194304] 0 2024-09-21T04:15:22.526 INFO:tasks.workunit.client.1.smithi148.stdout:7/702: dwrite d5/f3d [0,4194304] 0 2024-09-21T04:15:22.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/749: dwrite d17/d34/d3d/d57/db9/fd5 [0,4194304] 0 2024-09-21T04:15:22.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/750: truncate d17/d1e/f48 4660998 0 2024-09-21T04:15:22.532 INFO:tasks.workunit.client.0.smithi110.stdout:4/751: symlink d17/d34/d3d/d20/d43/d51/d7a/d96/lfb 0 2024-09-21T04:15:22.532 INFO:tasks.workunit.client.0.smithi110.stdout:4/752: creat d17/d34/d3d/d20/d43/d51/d7a/d96/ddf/ffc x:0 0 0 2024-09-21T04:15:22.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/753: getdents d17/d1e/d64 0 2024-09-21T04:15:22.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/754: dread - d17/d1e/d21/de1/ff1 zero size 2024-09-21T04:15:22.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/755: dread - d17/d1e/d21/fbe zero size 2024-09-21T04:15:22.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/756: creat d17/d3e/ffd x:0 0 0 2024-09-21T04:15:22.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/757: creat d17/d3e/ffe x:0 0 0 2024-09-21T04:15:22.541 INFO:tasks.workunit.client.0.smithi110.stdout:4/758: creat d17/d1e/d7b/daf/fff x:0 0 0 2024-09-21T04:15:22.541 INFO:tasks.workunit.client.0.smithi110.stdout:4/759: chown d17/d1e/d21/de1 89031881 1 2024-09-21T04:15:22.542 INFO:tasks.workunit.client.0.smithi110.stdout:4/760: symlink d17/d34/l100 0 2024-09-21T04:15:22.542 INFO:tasks.workunit.client.0.smithi110.stdout:4/761: fsync d17/d34/d3d/d20/d43/d51/d7a/d96/ddf/ffa 0 2024-09-21T04:15:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:0/951: write d4/dd/f1b [2539156,84459] 0 2024-09-21T04:15:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:9/568: dwrite d7/d10/f3f [0,4194304] 0 2024-09-21T04:15:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:2/504: dwrite d2/d1a/f88 [0,4194304] 0 2024-09-21T04:15:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:2/505: creat d2/d50/d56/f94 x:0 0 0 2024-09-21T04:15:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:4/762: dread d17/d34/d3d/d20/d43/d51/d52/fea [0,4194304] 0 2024-09-21T04:15:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:9/569: mkdir d7/d96/d99/d3c/d35/dbe 0 2024-09-21T04:15:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:9/570: stat d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a 0 2024-09-21T04:15:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:9/571: dread - d7/d96/d99/d3c/d1f/d43/f7b zero size 2024-09-21T04:15:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:2/506: creat d2/d1a/d1f/d5f/f95 x:0 0 0 2024-09-21T04:15:22.553 INFO:tasks.workunit.client.0.smithi110.stdout:4/763: creat d17/d34/d3d/d20/d43/d51/f101 x:0 0 0 2024-09-21T04:15:22.554 INFO:tasks.workunit.client.0.smithi110.stdout:2/507: symlink d2/d1a/d1f/l96 0 2024-09-21T04:15:22.554 INFO:tasks.workunit.client.0.smithi110.stdout:2/508: write d2/d1a/d1f/d29/f45 [368375,112823] 0 2024-09-21T04:15:22.555 INFO:tasks.workunit.client.0.smithi110.stdout:4/764: mknod d17/d1e/d21/c102 0 2024-09-21T04:15:22.556 INFO:tasks.workunit.client.0.smithi110.stdout:2/509: truncate d2/f44 3303452 0 2024-09-21T04:15:22.556 INFO:tasks.workunit.client.0.smithi110.stdout:1/954: sync 2024-09-21T04:15:22.556 INFO:tasks.workunit.client.0.smithi110.stdout:1/955: write d2/d9/d49/d7a/f7c [1518009,92523] 0 2024-09-21T04:15:22.557 INFO:tasks.workunit.client.0.smithi110.stdout:4/765: truncate d17/d34/d3d/d20/f26 6882225 0 2024-09-21T04:15:22.558 INFO:tasks.workunit.client.0.smithi110.stdout:2/510: mkdir d2/d50/d56/d97 0 2024-09-21T04:15:22.558 INFO:tasks.workunit.client.0.smithi110.stdout:0/952: dwrite d4/dd/d10/f50 [0,4194304] 0 2024-09-21T04:15:22.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/956: truncate d2/d9/d23/d3b/d9c/fa4 910624 0 2024-09-21T04:15:22.559 INFO:tasks.workunit.client.0.smithi110.stdout:1/957: creat d2/d9/d23/d3b/dca/f136 x:0 0 0 2024-09-21T04:15:22.560 INFO:tasks.workunit.client.0.smithi110.stdout:3/527: dwrite d0/d12/d3c/d5a/d70/f97 [0,4194304] 0 2024-09-21T04:15:22.560 INFO:tasks.workunit.client.0.smithi110.stdout:3/528: fdatasync d0/d12/d16/d1a/d5c/f85 0 2024-09-21T04:15:22.560 INFO:tasks.workunit.client.0.smithi110.stdout:0/953: symlink d4/l120 0 2024-09-21T04:15:22.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/745: dwrite d3/d14/d1f/f7e [4194304,4194304] 0 2024-09-21T04:15:22.563 INFO:tasks.workunit.client.1.smithi148.stdout:1/688: dwrite db/d24/d2a/da4/fb3 [0,4194304] 0 2024-09-21T04:15:22.563 INFO:tasks.workunit.client.1.smithi148.stdout:7/703: dwrite d5/d8/d6f/d7d/f98 [0,4194304] 0 2024-09-21T04:15:22.564 INFO:tasks.workunit.client.0.smithi110.stdout:0/954: rename d4/d1c/d101/c102 to d4/dd/d60/c121 0 2024-09-21T04:15:22.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/529: getdents d0/d12/d16/d1a/dac 0 2024-09-21T04:15:22.565 INFO:tasks.workunit.client.0.smithi110.stdout:0/955: rename d4 to d4/d47/d2c/d3a/d122 22 2024-09-21T04:15:22.568 INFO:tasks.workunit.client.0.smithi110.stdout:3/530: mknod d0/d22/cb6 0 2024-09-21T04:15:22.568 INFO:tasks.workunit.client.0.smithi110.stdout:3/531: write d0/f1f [7386474,61724] 0 2024-09-21T04:15:22.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/781: dwrite d4/d66/d7f/d2a/f36 [0,4194304] 0 2024-09-21T04:15:22.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/532: write d0/d12/f9d [304855,31163] 0 2024-09-21T04:15:22.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/533: creat d0/d12/d3c/dad/fb7 x:0 0 0 2024-09-21T04:15:22.569 INFO:tasks.workunit.client.0.smithi110.stdout:0/956: symlink d4/d1c/d62/l123 0 2024-09-21T04:15:22.570 INFO:tasks.workunit.client.0.smithi110.stdout:1/958: dread d2/d9/d26/d3d/d8f/f90 [0,4194304] 0 2024-09-21T04:15:22.575 INFO:tasks.workunit.client.1.smithi148.stdout:5/746: write d3/d24/f76 [4586493,42404] 0 2024-09-21T04:15:22.575 INFO:tasks.workunit.client.0.smithi110.stdout:3/534: creat d0/d12/d2a/d80/d9c/fb8 x:0 0 0 2024-09-21T04:15:22.576 INFO:tasks.workunit.client.1.smithi148.stdout:4/782: mkdir d4/d66/d7f/d9a/dce/df2 0 2024-09-21T04:15:22.577 INFO:tasks.workunit.client.0.smithi110.stdout:0/957: dread d4/d47/d2c/fce [0,4194304] 0 2024-09-21T04:15:22.578 INFO:tasks.workunit.client.1.smithi148.stdout:5/747: dread d3/d14/d1f/f80 [0,4194304] 0 2024-09-21T04:15:22.578 INFO:tasks.workunit.client.0.smithi110.stdout:1/959: creat d2/d9/d49/d5c/f137 x:0 0 0 2024-09-21T04:15:22.578 INFO:tasks.workunit.client.0.smithi110.stdout:1/960: write d2/d9/d23/f29 [3484904,73929] 0 2024-09-21T04:15:22.578 INFO:tasks.workunit.client.0.smithi110.stdout:1/961: fsync d2/d9/d23/de0/ff5 0 2024-09-21T04:15:22.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/704: symlink d5/d8/lf7 0 2024-09-21T04:15:22.579 INFO:tasks.workunit.client.0.smithi110.stdout:0/958: symlink d4/dd/d53/d66/l124 0 2024-09-21T04:15:22.580 INFO:tasks.workunit.client.0.smithi110.stdout:0/959: readlink d4/dd/d10/d29/l95 0 2024-09-21T04:15:22.580 INFO:tasks.workunit.client.0.smithi110.stdout:0/960: fdatasync d4/d1c/fe6 0 2024-09-21T04:15:22.580 INFO:tasks.workunit.client.1.smithi148.stdout:4/783: mkdir d4/d12/d16/d29/d61/d6c/df3 0 2024-09-21T04:15:22.583 INFO:tasks.workunit.client.0.smithi110.stdout:5/638: sync 2024-09-21T04:15:22.583 INFO:tasks.workunit.client.0.smithi110.stdout:7/633: sync 2024-09-21T04:15:22.584 INFO:tasks.workunit.client.0.smithi110.stdout:8/656: sync 2024-09-21T04:15:22.584 INFO:tasks.workunit.client.1.smithi148.stdout:5/748: dread d3/d14/d1f/d26/d2e/d56/da7/fab [0,4194304] 0 2024-09-21T04:15:22.585 INFO:tasks.workunit.client.1.smithi148.stdout:1/689: dwrite db/dd/d21/f5c [4194304,4194304] 0 2024-09-21T04:15:22.586 INFO:tasks.workunit.client.0.smithi110.stdout:4/766: dwrite d17/d1e/d21/de1/ff1 [0,4194304] 0 2024-09-21T04:15:22.586 INFO:tasks.workunit.client.0.smithi110.stdout:4/767: dread - d17/d1e/d7b/daf/fff zero size 2024-09-21T04:15:22.587 INFO:tasks.workunit.client.1.smithi148.stdout:4/784: mkdir d4/d87/d2d/df4 0 2024-09-21T04:15:22.587 INFO:tasks.workunit.client.0.smithi110.stdout:9/572: dwrite d7/d96/d99/d3c/d35/f76 [0,4194304] 0 2024-09-21T04:15:22.587 INFO:tasks.workunit.client.0.smithi110.stdout:9/573: dread - d7/d96/d99/d3c/d4f/d9f/d6f/d78/fb9 zero size 2024-09-21T04:15:22.587 INFO:tasks.workunit.client.0.smithi110.stdout:3/535: write d0/d12/d16/f6a [1655116,107316] 0 2024-09-21T04:15:22.589 INFO:tasks.workunit.client.0.smithi110.stdout:5/639: dread d1/d7/fa1 [0,4194304] 0 2024-09-21T04:15:22.589 INFO:tasks.workunit.client.0.smithi110.stdout:1/962: creat d2/d9/f138 x:0 0 0 2024-09-21T04:15:22.590 INFO:tasks.workunit.client.0.smithi110.stdout:0/961: creat d4/dd/d10/d29/d45/f125 x:0 0 0 2024-09-21T04:15:22.593 INFO:tasks.workunit.client.0.smithi110.stdout:7/634: link dd/f57 dd/d1c/dad/db2/d4a/d52/fcf 0 2024-09-21T04:15:22.595 INFO:tasks.workunit.client.1.smithi148.stdout:1/690: mkdir db/d24/d2e/d71/d94/dde 0 2024-09-21T04:15:22.598 INFO:tasks.workunit.client.0.smithi110.stdout:6/680: sync 2024-09-21T04:15:22.598 INFO:tasks.workunit.client.0.smithi110.stdout:6/681: stat d4/d12/d17/d2e/d38/c41 0 2024-09-21T04:15:22.598 INFO:tasks.workunit.client.1.smithi148.stdout:5/749: getdents d3/d14/d1f/d26/d2e/d3d/da0 0 2024-09-21T04:15:22.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/691: rename db/d24/d2a/l2d to db/dd/db9/dc2/ldf 0 2024-09-21T04:15:22.604 INFO:tasks.workunit.client.0.smithi110.stdout:4/768: write d17/d34/d3d/fa3 [3193546,99750] 0 2024-09-21T04:15:22.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/692: symlink db/d24/d2e/d71/le0 0 2024-09-21T04:15:22.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/574: rename d7/d96/d99/d3c/d35/dbe to d7/d96/d99/d3c/d1f/d43/db5/dbf 0 2024-09-21T04:15:22.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/575: fsync d7/d96/d99/d3c/d35/fa7 0 2024-09-21T04:15:22.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/576: chown d7/d96/d99/d3c/d1f 10 1 2024-09-21T04:15:22.612 INFO:tasks.workunit.client.0.smithi110.stdout:3/536: mkdir d0/d12/d16/d1a/d6f/db9 0 2024-09-21T04:15:22.613 INFO:tasks.workunit.client.1.smithi148.stdout:5/750: write d3/d37/f7f [1463904,37399] 0 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/640: rmdir d1/d49/d4d/d1e/d4a/dae 39 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/641: readlink d1/d22/db6/lc5 0 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/642: write d1/d7/d18/f8b [679744,130902] 0 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/643: chown d1/d49/d4d/d1e/c5b 11 1 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/644: dread - d1/fa7 zero size 2024-09-21T04:15:22.614 INFO:tasks.workunit.client.0.smithi110.stdout:5/645: write d1/d7/d18/d8f/fd3 [4081862,28892] 0 2024-09-21T04:15:22.617 INFO:tasks.workunit.client.0.smithi110.stdout:7/635: unlink dd/d1c/dad/db2/d4a/d52/fcf 0 2024-09-21T04:15:22.617 INFO:tasks.workunit.client.0.smithi110.stdout:8/657: dwrite d0/d30/d34/d3e/d32/fb9 [0,4194304] 0 2024-09-21T04:15:22.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/682: symlink d4/d12/d17/d33/d76/ldb 0 2024-09-21T04:15:22.619 INFO:tasks.workunit.client.1.smithi148.stdout:7/705: dwrite d5/d14/d25/d79/fed [0,4194304] 0 2024-09-21T04:15:22.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/751: write d3/d14/f44 [854384,83864] 0 2024-09-21T04:15:22.621 INFO:tasks.workunit.client.0.smithi110.stdout:4/769: symlink d17/d1e/d21/de1/l103 0 2024-09-21T04:15:22.622 INFO:tasks.workunit.client.0.smithi110.stdout:4/770: chown d17/d1e/d77/dcf 1831 1 2024-09-21T04:15:22.622 INFO:tasks.workunit.client.0.smithi110.stdout:4/771: stat d17/f1a 0 2024-09-21T04:15:22.624 INFO:tasks.workunit.client.0.smithi110.stdout:9/577: creat d7/d10/fc0 x:0 0 0 2024-09-21T04:15:22.625 INFO:tasks.workunit.client.1.smithi148.stdout:5/752: dread d3/d14/d1f/d26/d2e/d56/f73 [0,4194304] 0 2024-09-21T04:15:22.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/646: creat d1/dc/fd6 x:0 0 0 2024-09-21T04:15:22.627 INFO:tasks.workunit.client.0.smithi110.stdout:8/658: dread d0/d7/d13/d4b/f67 [0,4194304] 0 2024-09-21T04:15:22.627 INFO:tasks.workunit.client.0.smithi110.stdout:1/963: dwrite d2/db7/f105 [0,4194304] 0 2024-09-21T04:15:22.627 INFO:tasks.workunit.client.0.smithi110.stdout:8/659: truncate d0/d30/d54/fb3 80913 0 2024-09-21T04:15:22.633 INFO:tasks.workunit.client.1.smithi148.stdout:5/753: mkdir d3/d34/d78/dbf/d101 0 2024-09-21T04:15:22.633 INFO:tasks.workunit.client.1.smithi148.stdout:5/754: truncate d3/d24/dad/fe6 468217 0 2024-09-21T04:15:22.633 INFO:tasks.workunit.client.1.smithi148.stdout:5/755: fdatasync d3/d34/d3c/d5d/d75/fec 0 2024-09-21T04:15:22.633 INFO:tasks.workunit.client.0.smithi110.stdout:6/683: symlink d4/d12/d17/d2e/d40/dad/ldc 0 2024-09-21T04:15:22.634 INFO:tasks.workunit.client.1.smithi148.stdout:2/835: sync 2024-09-21T04:15:22.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/734: sync 2024-09-21T04:15:22.635 INFO:tasks.workunit.client.0.smithi110.stdout:4/772: mknod d17/d34/d3d/d20/d43/d51/d7a/c104 0 2024-09-21T04:15:22.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/636: dread dd/d14/d17/f85 [0,4194304] 0 2024-09-21T04:15:22.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/637: write dd/d48/fc8 [182207,37155] 0 2024-09-21T04:15:22.635 INFO:tasks.workunit.client.0.smithi110.stdout:5/647: symlink d1/d7/d18/ld7 0 2024-09-21T04:15:22.637 INFO:tasks.workunit.client.1.smithi148.stdout:9/735: truncate d5/f23 2174 0 2024-09-21T04:15:22.639 INFO:tasks.workunit.client.0.smithi110.stdout:8/660: rmdir d0/d30/d54/d6b/da8 39 2024-09-21T04:15:22.643 INFO:tasks.workunit.client.0.smithi110.stdout:9/578: dread d7/d96/d99/d3c/d11/f4e [0,4194304] 0 2024-09-21T04:15:22.643 INFO:tasks.workunit.client.1.smithi148.stdout:9/736: rename d5/d39/fc1 to d5/d60/d28/f105 0 2024-09-21T04:15:22.644 INFO:tasks.workunit.client.0.smithi110.stdout:4/773: symlink d17/d3e/l105 0 2024-09-21T04:15:22.646 INFO:tasks.workunit.client.0.smithi110.stdout:5/648: creat d1/d39/da2/fd8 x:0 0 0 2024-09-21T04:15:22.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/785: dwrite d4/d12/d16/d29/f3d [0,4194304] 0 2024-09-21T04:15:22.647 INFO:tasks.workunit.client.1.smithi148.stdout:2/836: dread d0/f14 [0,4194304] 0 2024-09-21T04:15:22.648 INFO:tasks.workunit.client.1.smithi148.stdout:9/737: getdents d5/d2b/d4e/db3 0 2024-09-21T04:15:22.649 INFO:tasks.workunit.client.1.smithi148.stdout:9/738: stat d5/d2b/d32/f82 0 2024-09-21T04:15:22.650 INFO:tasks.workunit.client.0.smithi110.stdout:2/511: dwrite d2/f44 [0,4194304] 0 2024-09-21T04:15:22.652 INFO:tasks.workunit.client.1.smithi148.stdout:1/693: dwrite db/dd/f8c [0,4194304] 0 2024-09-21T04:15:22.654 INFO:tasks.workunit.client.0.smithi110.stdout:8/661: mknod d0/ce4 0 2024-09-21T04:15:22.656 INFO:tasks.workunit.client.0.smithi110.stdout:3/537: dwrite d0/d22/f46 [0,4194304] 0 2024-09-21T04:15:22.656 INFO:tasks.workunit.client.0.smithi110.stdout:3/538: creat d0/d12/d2a/d80/d9c/fba x:0 0 0 2024-09-21T04:15:22.656 INFO:tasks.workunit.client.0.smithi110.stdout:3/539: truncate d0/d12/d2a/d80/d9c/fb8 600611 0 2024-09-21T04:15:22.657 INFO:tasks.workunit.client.0.smithi110.stdout:4/774: write d17/d34/d3d/d20/dca/fd1 [4062508,55629] 0 2024-09-21T04:15:22.660 INFO:tasks.workunit.client.1.smithi148.stdout:7/706: dwrite d5/d8/fe3 [0,4194304] 0 2024-09-21T04:15:22.666 INFO:tasks.workunit.client.0.smithi110.stdout:7/638: dwrite dd/d4d/f71 [0,4194304] 0 2024-09-21T04:15:22.666 INFO:tasks.workunit.client.0.smithi110.stdout:8/662: write d0/d7/d13/d46/f51 [1696378,93136] 0 2024-09-21T04:15:22.666 INFO:tasks.workunit.client.1.smithi148.stdout:2/837: symlink d0/d2b/da3/d8f/d9f/l109 0 2024-09-21T04:15:22.667 INFO:tasks.workunit.client.0.smithi110.stdout:4/775: dread d17/f3c [0,4194304] 0 2024-09-21T04:15:22.667 INFO:tasks.workunit.client.0.smithi110.stdout:3/540: mknod d0/d12/d3c/d5a/d70/cbb 0 2024-09-21T04:15:22.669 INFO:tasks.workunit.client.1.smithi148.stdout:9/739: stat d5/d2b/dbe/cec 0 2024-09-21T04:15:22.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/694: unlink db/d46/d76/fd3 0 2024-09-21T04:15:22.675 INFO:tasks.workunit.client.1.smithi148.stdout:5/756: dwrite d3/d34/d3c/d5d/f95 [0,4194304] 0 2024-09-21T04:15:22.682 INFO:tasks.workunit.client.0.smithi110.stdout:8/663: dread d0/d7/d13/d4b/f17 [4194304,4194304] 0 2024-09-21T04:15:22.682 INFO:tasks.workunit.client.0.smithi110.stdout:8/664: creat d0/d7/d23/d52/d8b/fe5 x:0 0 0 2024-09-21T04:15:22.682 INFO:tasks.workunit.client.0.smithi110.stdout:7/639: symlink dd/d1c/dad/db2/d4a/ld0 0 2024-09-21T04:15:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/838: symlink d0/d9/dc/d13/dec/l10a 0 2024-09-21T04:15:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/839: chown d0/d9/dc/ddf/le5 123908 1 2024-09-21T04:15:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/840: fdatasync d0/f1e 0 2024-09-21T04:15:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/841: chown d0/d9/dc/d13/daf/db2/ff9 1282 1 2024-09-21T04:15:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/842: truncate d0/d9/dc/d13/d25/f2a 2904502 0 2024-09-21T04:15:22.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/740: getdents d5/de3/de7 0 2024-09-21T04:15:22.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/741: readlink d5/d2b/lf2 0 2024-09-21T04:15:22.684 INFO:tasks.workunit.client.1.smithi148.stdout:9/742: stat d5/d1f/f24 0 2024-09-21T04:15:22.684 INFO:tasks.workunit.client.1.smithi148.stdout:1/695: mkdir db/d24/d2e/d71/d94/dde/de1 0 2024-09-21T04:15:22.685 INFO:tasks.workunit.client.0.smithi110.stdout:4/776: symlink d17/d1e/l106 0 2024-09-21T04:15:22.685 INFO:tasks.workunit.client.0.smithi110.stdout:4/777: write d17/d3e/f8b [565412,14765] 0 2024-09-21T04:15:22.686 INFO:tasks.workunit.client.1.smithi148.stdout:1/696: dread db/f75 [0,4194304] 0 2024-09-21T04:15:22.686 INFO:tasks.workunit.client.0.smithi110.stdout:3/541: mknod d0/d12/d3c/dad/cbc 0 2024-09-21T04:15:22.688 INFO:tasks.workunit.client.1.smithi148.stdout:9/743: rename d5/d2b/d32/d67 to d5/d1f/d81/dcf/d106 0 2024-09-21T04:15:22.689 INFO:tasks.workunit.client.1.smithi148.stdout:9/744: chown d5/d39/f6f 129 1 2024-09-21T04:15:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/778: mkdir d17/d34/d3d/d20/dca/df3/d107 0 2024-09-21T04:15:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/779: write d17/f32 [2888835,98971] 0 2024-09-21T04:15:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/780: read - d17/d34/d3d/d57/db9/fc8 zero size 2024-09-21T04:15:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/781: dread - d17/d1e/d7b/fc4 zero size 2024-09-21T04:15:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:3/542: symlink d0/d12/d2a/d80/da2/db0/lbd 0 2024-09-21T04:15:22.691 INFO:tasks.workunit.client.1.smithi148.stdout:1/697: rename db/d46/d4c/c96 to db/dd/d21/d89/dc9/ce2 0 2024-09-21T04:15:22.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/698: truncate db/f75 952631 0 2024-09-21T04:15:22.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/699: readlink db/d46/d4c/l85 0 2024-09-21T04:15:22.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/700: write db/d46/fb6 [245619,52502] 0 2024-09-21T04:15:22.692 INFO:tasks.workunit.client.0.smithi110.stdout:4/782: mknod d17/d1e/d7b/dc5/c108 0 2024-09-21T04:15:22.693 INFO:tasks.workunit.client.1.smithi148.stdout:1/701: creat db/d46/d76/fe3 x:0 0 0 2024-09-21T04:15:22.693 INFO:tasks.workunit.client.0.smithi110.stdout:3/543: rename d0/d22/d29/d76/c9f to d0/d12/d2a/d80/da2/da7/cbe 0 2024-09-21T04:15:22.694 INFO:tasks.workunit.client.0.smithi110.stdout:4/783: symlink d17/d34/d3d/d20/d43/d51/d52/d98/l109 0 2024-09-21T04:15:22.694 INFO:tasks.workunit.client.0.smithi110.stdout:4/784: chown f4 3 1 2024-09-21T04:15:22.695 INFO:tasks.workunit.client.1.smithi148.stdout:1/702: mkdir db/d24/d2e/d71/d94/de4 0 2024-09-21T04:15:22.695 INFO:tasks.workunit.client.0.smithi110.stdout:3/544: rmdir d0/d12/d3c/d5a/d70/dab 39 2024-09-21T04:15:22.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/703: mknod db/dd/ddd/ce5 0 2024-09-21T04:15:22.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/704: fdatasync db/d24/f70 0 2024-09-21T04:15:22.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/705: stat db/d24/d2e/d71/f8e 0 2024-09-21T04:15:22.697 INFO:tasks.workunit.client.0.smithi110.stdout:3/545: truncate d0/f88 999756 0 2024-09-21T04:15:22.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/706: mknod db/dd/d21/d9b/ce6 0 2024-09-21T04:15:22.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/707: creat db/d24/fe7 x:0 0 0 2024-09-21T04:15:22.698 INFO:tasks.workunit.client.0.smithi110.stdout:3/546: unlink d0/d12/f5d 0 2024-09-21T04:15:22.700 INFO:tasks.workunit.client.1.smithi148.stdout:1/708: truncate db/f64 1005717 0 2024-09-21T04:15:22.700 INFO:tasks.workunit.client.1.smithi148.stdout:1/709: write db/f92 [679284,10121] 0 2024-09-21T04:15:22.701 INFO:tasks.workunit.client.1.smithi148.stdout:1/710: link db/d24/d2a/da4/fb3 db/d24/d2a/dbd/fe8 0 2024-09-21T04:15:22.702 INFO:tasks.workunit.client.1.smithi148.stdout:1/711: fsync db/d46/d8f/fbb 0 2024-09-21T04:15:22.702 INFO:tasks.workunit.client.0.smithi110.stdout:8/665: dread d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:22.703 INFO:tasks.workunit.client.1.smithi148.stdout:1/712: symlink db/dd/d21/db0/le9 0 2024-09-21T04:15:22.703 INFO:tasks.workunit.client.1.smithi148.stdout:1/713: write db/dd/d21/f7a [851002,29185] 0 2024-09-21T04:15:22.704 INFO:tasks.workunit.client.1.smithi148.stdout:1/714: truncate db/d24/d2e/d59/da7/fb4 985696 0 2024-09-21T04:15:22.704 INFO:tasks.workunit.client.0.smithi110.stdout:8/666: dread d0/d7/d23/d82/f83 [0,4194304] 0 2024-09-21T04:15:22.705 INFO:tasks.workunit.client.0.smithi110.stdout:8/667: link d0/d30/d54/d6b/da8/cd9 d0/d7/d13/d4b/da7/ce6 0 2024-09-21T04:15:22.705 INFO:tasks.workunit.client.0.smithi110.stdout:6/684: dwrite d4/d12/d17/d33/d3e/f53 [0,4194304] 0 2024-09-21T04:15:22.706 INFO:tasks.workunit.client.0.smithi110.stdout:7/640: read dd/d14/f46 [591951,400] 0 2024-09-21T04:15:22.706 INFO:tasks.workunit.client.1.smithi148.stdout:7/707: dwrite d5/d14/d25/d40/d7b/f9e [0,4194304] 0 2024-09-21T04:15:22.707 INFO:tasks.workunit.client.1.smithi148.stdout:1/715: creat db/d24/d2e/d71/d94/dde/fea x:0 0 0 2024-09-21T04:15:22.708 INFO:tasks.workunit.client.0.smithi110.stdout:6/685: mknod d4/d12/d17/d81/dbf/cdd 0 2024-09-21T04:15:22.708 INFO:tasks.workunit.client.0.smithi110.stdout:3/547: write d0/f7 [6753349,56965] 0 2024-09-21T04:15:22.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/786: dwrite d4/d12/d75/d85/fbb [0,4194304] 0 2024-09-21T04:15:22.710 INFO:tasks.workunit.client.0.smithi110.stdout:2/512: dwrite d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:22.710 INFO:tasks.workunit.client.0.smithi110.stdout:4/785: dread d17/d1e/f4c [0,4194304] 0 2024-09-21T04:15:22.710 INFO:tasks.workunit.client.0.smithi110.stdout:9/579: dwrite d7/d26/f90 [0,4194304] 0 2024-09-21T04:15:22.712 INFO:tasks.workunit.client.0.smithi110.stdout:7/641: mkdir dd/d14/d17/dd1 0 2024-09-21T04:15:22.714 INFO:tasks.workunit.client.0.smithi110.stdout:4/786: mkdir d17/d1e/d77/dcf/d10a 0 2024-09-21T04:15:22.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/708: mkdir d5/d8/d6f/dcd/df8 0 2024-09-21T04:15:22.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/709: chown d5/d8 205385 1 2024-09-21T04:15:22.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/710: fsync d5/d8/d6f/d7d/f68 0 2024-09-21T04:15:22.717 INFO:tasks.workunit.client.0.smithi110.stdout:6/686: write d4/d12/d17/d2e/d40/f2f [5536540,128951] 0 2024-09-21T04:15:22.717 INFO:tasks.workunit.client.0.smithi110.stdout:6/687: stat d4/f6 0 2024-09-21T04:15:22.719 INFO:tasks.workunit.client.0.smithi110.stdout:7/642: mkdir dd/d14/d17/d8a/dd2 0 2024-09-21T04:15:22.720 INFO:tasks.workunit.client.1.smithi148.stdout:0/722: sync 2024-09-21T04:15:22.720 INFO:tasks.workunit.client.1.smithi148.stdout:3/669: sync 2024-09-21T04:15:22.720 INFO:tasks.workunit.client.1.smithi148.stdout:0/723: write d5/d6/d2a/d2b/d84/da5/f95 [41833,48530] 0 2024-09-21T04:15:22.720 INFO:tasks.workunit.client.1.smithi148.stdout:0/724: write d5/d6/d2a/d45/d70/fc6 [2244058,108996] 0 2024-09-21T04:15:22.720 INFO:tasks.workunit.client.1.smithi148.stdout:8/672: sync 2024-09-21T04:15:22.723 INFO:tasks.workunit.client.0.smithi110.stdout:4/787: symlink d17/d1e/d7b/daf/l10b 0 2024-09-21T04:15:22.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/580: dread d7/d96/d99/d3c/d14/f54 [0,4194304] 0 2024-09-21T04:15:22.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/581: write d7/d96/d99/d3c/d35/f75 [1040461,69671] 0 2024-09-21T04:15:22.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/582: chown d7/fa4 6266 1 2024-09-21T04:15:22.724 INFO:tasks.workunit.client.1.smithi148.stdout:4/787: mknod d4/d87/dc1/cf5 0 2024-09-21T04:15:22.724 INFO:tasks.workunit.client.1.smithi148.stdout:4/788: readlink d4/d12/d16/lae 0 2024-09-21T04:15:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:6/688: mknod d4/d78/daa/cde 0 2024-09-21T04:15:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:6/689: dread - d4/d12/d17/d2e/d38/f98 zero size 2024-09-21T04:15:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:6/690: stat d4/f8c 0 2024-09-21T04:15:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:6/691: chown d4/d12/d17/d2e/l58 40837022 1 2024-09-21T04:15:22.727 INFO:tasks.workunit.client.0.smithi110.stdout:7/643: truncate dd/d1c/f8c 2819496 0 2024-09-21T04:15:22.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/673: symlink dc/d43/d4c/d4f/dc5/ld1 0 2024-09-21T04:15:22.731 INFO:tasks.workunit.client.1.smithi148.stdout:7/711: dread d5/d8/f1e [0,4194304] 0 2024-09-21T04:15:22.731 INFO:tasks.workunit.client.1.smithi148.stdout:6/707: sync 2024-09-21T04:15:22.731 INFO:tasks.workunit.client.1.smithi148.stdout:6/708: read - d0/d1a/d23/f61 zero size 2024-09-21T04:15:22.732 INFO:tasks.workunit.client.0.smithi110.stdout:9/583: mkdir d7/d96/d99/d3c/d11/dc1 0 2024-09-21T04:15:22.732 INFO:tasks.workunit.client.0.smithi110.stdout:9/584: stat d7/d96/d99/d3c/d35/f73 0 2024-09-21T04:15:22.732 INFO:tasks.workunit.client.0.smithi110.stdout:9/585: fdatasync d7/d96/d99/d3c/d1f/d43/fb2 0 2024-09-21T04:15:22.735 INFO:tasks.workunit.client.1.smithi148.stdout:7/712: mknod d5/d8/d6f/d7d/d90/cf9 0 2024-09-21T04:15:22.735 INFO:tasks.workunit.client.1.smithi148.stdout:7/713: truncate d5/d14/f9f 218139 0 2024-09-21T04:15:22.736 INFO:tasks.workunit.client.0.smithi110.stdout:7/644: creat dd/d1c/dad/db2/d6d/fd3 x:0 0 0 2024-09-21T04:15:22.736 INFO:tasks.workunit.client.0.smithi110.stdout:7/645: fdatasync dd/d1c/dad/db2/d56/d66/da6/d98/fb5 0 2024-09-21T04:15:22.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/709: symlink d0/d34/da0/ld1 0 2024-09-21T04:15:22.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/710: truncate d0/d1a/d23/f61 392428 0 2024-09-21T04:15:22.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/711: creat d0/d1a/d23/d32/d93/fd2 x:0 0 0 2024-09-21T04:15:22.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/712: creat d0/d34/d38/dc6/fd3 x:0 0 0 2024-09-21T04:15:22.738 INFO:tasks.workunit.client.1.smithi148.stdout:1/716: dwrite db/d46/f8d [0,4194304] 0 2024-09-21T04:15:22.740 INFO:tasks.workunit.client.0.smithi110.stdout:3/548: dwrite d0/d12/d16/d1a/dac/fb5 [0,4194304] 0 2024-09-21T04:15:22.740 INFO:tasks.workunit.client.0.smithi110.stdout:3/549: fdatasync d0/f6 0 2024-09-21T04:15:22.740 INFO:tasks.workunit.client.0.smithi110.stdout:3/550: write d0/d12/d3c/f45 [587296,57358] 0 2024-09-21T04:15:22.741 INFO:tasks.workunit.client.0.smithi110.stdout:8/668: dwrite d0/d7/d13/fb1 [4194304,4194304] 0 2024-09-21T04:15:22.741 INFO:tasks.workunit.client.1.smithi148.stdout:8/674: dread dc/d13/d39/d6e/f6f [0,4194304] 0 2024-09-21T04:15:22.744 INFO:tasks.workunit.client.1.smithi148.stdout:6/713: mkdir d0/d34/da0/dd4 0 2024-09-21T04:15:22.745 INFO:tasks.workunit.client.0.smithi110.stdout:2/513: dwrite d2/d50/d56/f51 [0,4194304] 0 2024-09-21T04:15:22.745 INFO:tasks.workunit.client.0.smithi110.stdout:2/514: write d2/d50/d56/f51 [4577327,7842] 0 2024-09-21T04:15:22.746 INFO:tasks.workunit.client.0.smithi110.stdout:9/586: creat d7/d96/d99/d3c/d14/fc2 x:0 0 0 2024-09-21T04:15:22.749 INFO:tasks.workunit.client.0.smithi110.stdout:6/692: symlink d4/d12/d16/d66/d7a/d83/ldf 0 2024-09-21T04:15:22.749 INFO:tasks.workunit.client.0.smithi110.stdout:6/693: chown d4/d12/d17/d2e/d40/fd8 5 1 2024-09-21T04:15:22.749 INFO:tasks.workunit.client.0.smithi110.stdout:6/694: fdatasync d4/d12/d17/d33/d76/fc1 0 2024-09-21T04:15:22.751 INFO:tasks.workunit.client.0.smithi110.stdout:7/646: stat dd/c10 0 2024-09-21T04:15:22.753 INFO:tasks.workunit.client.1.smithi148.stdout:9/745: getdents d5/d2b/d32 0 2024-09-21T04:15:22.762 INFO:tasks.workunit.client.0.smithi110.stdout:8/669: symlink d0/d7/d6f/d86/le7 0 2024-09-21T04:15:22.763 INFO:tasks.workunit.client.0.smithi110.stdout:4/788: rename d17/d34/d3d to d17/d1e/d77/d10c 0 2024-09-21T04:15:22.765 INFO:tasks.workunit.client.0.smithi110.stdout:2/515: symlink d2/d50/d56/d97/l98 0 2024-09-21T04:15:22.765 INFO:tasks.workunit.client.0.smithi110.stdout:2/516: stat d2/d1a/d1f 0 2024-09-21T04:15:22.767 INFO:tasks.workunit.client.1.smithi148.stdout:6/714: creat d0/d34/d97/d4f/fd5 x:0 0 0 2024-09-21T04:15:22.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/714: dwrite d5/d14/d25/fc2 [0,4194304] 0 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/789: dwrite d4/d12/d16/d29/d61/d6c/f88 [0,4194304] 0 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/790: chown d4/l43 3545784 1 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/791: write d4/d12/d16/d29/d61/f8e [1595538,34938] 0 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/792: dread - d4/d12/d16/d29/dbd/f68 zero size 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.1.smithi148.stdout:0/725: dwrite d5/d6/d2a/d2b/fac [4194304,4194304] 0 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.0.smithi110.stdout:6/695: rmdir d4/d12/d17/d2e 39 2024-09-21T04:15:22.769 INFO:tasks.workunit.client.0.smithi110.stdout:3/551: dwrite d0/d12/d16/f62 [0,4194304] 0 2024-09-21T04:15:22.770 INFO:tasks.workunit.client.1.smithi148.stdout:1/717: dwrite db/d46/f66 [0,4194304] 0 2024-09-21T04:15:22.770 INFO:tasks.workunit.client.1.smithi148.stdout:1/718: fsync db/dd/f12 0 2024-09-21T04:15:22.774 INFO:tasks.workunit.client.1.smithi148.stdout:9/746: rename d5/d2b/d4e/cd0 to d5/d2b/d4e/db3/ded/c107 0 2024-09-21T04:15:22.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/587: rename d7/d96/d99/d3c/d1f/d43/fb2 to d7/d96/fc3 0 2024-09-21T04:15:22.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/588: write d7/d96/d99/d3c/f20 [2312152,125370] 0 2024-09-21T04:15:22.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/719: dread db/d24/f8b [0,4194304] 0 2024-09-21T04:15:22.777 INFO:tasks.workunit.client.1.smithi148.stdout:1/720: chown db/dd/d21/d89/dc9 6932 1 2024-09-21T04:15:22.778 INFO:tasks.workunit.client.1.smithi148.stdout:1/721: dread db/dd/fc3 [0,4194304] 0 2024-09-21T04:15:22.779 INFO:tasks.workunit.client.0.smithi110.stdout:4/789: mknod d17/d3e/d6b/dc7/c10d 0 2024-09-21T04:15:22.779 INFO:tasks.workunit.client.0.smithi110.stdout:4/790: chown d17/d3e/dd3 21518 1 2024-09-21T04:15:22.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/715: creat d0/d34/d38/fd6 x:0 0 0 2024-09-21T04:15:22.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/552: link d0/d12/fb3 d0/d12/d3c/d5a/d70/fbf 0 2024-09-21T04:15:22.787 INFO:tasks.workunit.client.1.smithi148.stdout:6/716: read d0/f6b [229614,89943] 0 2024-09-21T04:15:22.787 INFO:tasks.workunit.client.0.smithi110.stdout:7/647: getdents dd/d1c/dad/db2/d56/d66/da6/d98 0 2024-09-21T04:15:22.790 INFO:tasks.workunit.client.0.smithi110.stdout:8/670: dwrite d0/d7/f78 [0,4194304] 0 2024-09-21T04:15:22.790 INFO:tasks.workunit.client.1.smithi148.stdout:7/715: mknod d5/d8/d6f/d7d/d90/cfa 0 2024-09-21T04:15:22.792 INFO:tasks.workunit.client.0.smithi110.stdout:9/589: rename d7/d96/d99/d3c/d4f/d9f/d6f/d78/c79 to d7/d96/d99/cc4 0 2024-09-21T04:15:22.794 INFO:tasks.workunit.client.1.smithi148.stdout:4/793: creat d4/d12/d75/d9c/ff6 x:0 0 0 2024-09-21T04:15:22.800 INFO:tasks.workunit.client.1.smithi148.stdout:9/747: mkdir d5/d60/d28/d73/d108 0 2024-09-21T04:15:22.801 INFO:tasks.workunit.client.0.smithi110.stdout:4/791: truncate d17/d1e/fbf 392444 0 2024-09-21T04:15:22.803 INFO:tasks.workunit.client.1.smithi148.stdout:6/717: creat d0/d34/fd7 x:0 0 0 2024-09-21T04:15:22.805 INFO:tasks.workunit.client.1.smithi148.stdout:4/794: mknod d4/d66/cf7 0 2024-09-21T04:15:22.805 INFO:tasks.workunit.client.1.smithi148.stdout:4/795: dread - d4/d66/fbe zero size 2024-09-21T04:15:22.806 INFO:tasks.workunit.client.0.smithi110.stdout:2/517: rmdir d2/d1a/d1f 39 2024-09-21T04:15:22.809 INFO:tasks.workunit.client.0.smithi110.stdout:8/671: mkdir d0/d30/d34/d3e/d5d/dd7/de8 0 2024-09-21T04:15:22.811 INFO:tasks.workunit.client.0.smithi110.stdout:9/590: creat d7/d96/d99/d3c/d35/d71/fc5 x:0 0 0 2024-09-21T04:15:22.811 INFO:tasks.workunit.client.0.smithi110.stdout:9/591: truncate d7/d96/d99/d3c/d35/f5e 5066973 0 2024-09-21T04:15:22.814 INFO:tasks.workunit.client.1.smithi148.stdout:6/718: dread d0/d1a/d23/d27/d80/fb2 [0,4194304] 0 2024-09-21T04:15:22.814 INFO:tasks.workunit.client.1.smithi148.stdout:0/726: dwrite d5/d82/f87 [0,4194304] 0 2024-09-21T04:15:22.814 INFO:tasks.workunit.client.0.smithi110.stdout:4/792: mknod d17/d1e/d77/d10c/d20/c10e 0 2024-09-21T04:15:22.816 INFO:tasks.workunit.client.0.smithi110.stdout:8/672: creat d0/d30/d34/d3e/d5d/fe9 x:0 0 0 2024-09-21T04:15:22.818 INFO:tasks.workunit.client.0.smithi110.stdout:4/793: creat d17/d1e/d77/d10c/d20/dca/f10f x:0 0 0 2024-09-21T04:15:22.818 INFO:tasks.workunit.client.0.smithi110.stdout:4/794: stat d17/d1e/d77/d10c/fa3 0 2024-09-21T04:15:22.818 INFO:tasks.workunit.client.0.smithi110.stdout:4/795: chown d17/d1e/d77/d10c/d20/d43/d51/d7a/d8d/lbb 25398 1 2024-09-21T04:15:22.818 INFO:tasks.workunit.client.0.smithi110.stdout:4/796: chown d17/d1e/c5d 145446 1 2024-09-21T04:15:22.821 INFO:tasks.workunit.client.0.smithi110.stdout:4/797: link d17/d1e/d77/d10c/d20/f80 d17/d3e/dbd/f110 0 2024-09-21T04:15:22.823 INFO:tasks.workunit.client.0.smithi110.stdout:4/798: mknod d17/d34/c111 0 2024-09-21T04:15:22.823 INFO:tasks.workunit.client.0.smithi110.stdout:4/799: chown d17/d1e/d77/d10c/d20/d43/d51 49 1 2024-09-21T04:15:22.823 INFO:tasks.workunit.client.1.smithi148.stdout:1/722: dwrite db/f58 [0,4194304] 0 2024-09-21T04:15:22.823 INFO:tasks.workunit.client.1.smithi148.stdout:1/723: chown db/d46/d4c/cb7 1102 1 2024-09-21T04:15:22.824 INFO:tasks.workunit.client.0.smithi110.stdout:4/800: creat d17/d1e/d77/d10c/f112 x:0 0 0 2024-09-21T04:15:22.824 INFO:tasks.workunit.client.0.smithi110.stdout:8/673: dread d0/d7/fd [0,4194304] 0 2024-09-21T04:15:22.825 INFO:tasks.workunit.client.1.smithi148.stdout:1/724: symlink db/dd/d21/d89/leb 0 2024-09-21T04:15:22.825 INFO:tasks.workunit.client.1.smithi148.stdout:1/725: chown db/d24/d2e/d59/da7/fd8 502 1 2024-09-21T04:15:22.827 INFO:tasks.workunit.client.0.smithi110.stdout:4/801: unlink d17/f1a 0 2024-09-21T04:15:22.827 INFO:tasks.workunit.client.0.smithi110.stdout:4/802: readlink d17/d1e/d77/d10c/d20/d43/d51/d7a/d86/lf9 0 2024-09-21T04:15:22.827 INFO:tasks.workunit.client.0.smithi110.stdout:4/803: fdatasync d17/d1e/d77/d10c/d20/d43/d51/d7a/d86/f9b 0 2024-09-21T04:15:22.828 INFO:tasks.workunit.client.0.smithi110.stdout:8/674: symlink d0/d7/d13/d1c/d41/lea 0 2024-09-21T04:15:22.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/648: dwrite dd/d1c/dad/db2/d4a/d52/f9c [0,4194304] 0 2024-09-21T04:15:22.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/649: chown dd/d1c/l23 595 1 2024-09-21T04:15:22.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/650: write dd/d48/f4b [1236466,14404] 0 2024-09-21T04:15:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:1/726: write db/d24/f8b [95975,96319] 0 2024-09-21T04:15:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:1/727: creat db/d24/d2a/da4/fec x:0 0 0 2024-09-21T04:15:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:1/728: write db/d24/fd0 [804743,36234] 0 2024-09-21T04:15:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:6/719: read d0/d34/d38/d43/f98 [1270888,55915] 0 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.0.smithi110.stdout:7/651: creat dd/d1c/dad/db2/d4a/fd4 x:0 0 0 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.0.smithi110.stdout:7/652: fdatasync dd/f7e 0 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.0.smithi110.stdout:7/653: fdatasync dd/d4d/fa1 0 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.0.smithi110.stdout:8/675: dread d0/d7/d13/d4b/f7e [0,4194304] 0 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.0.smithi110.stdout:8/676: chown d0/d7/d23/d82/d92/lc7 1545 1 2024-09-21T04:15:22.835 INFO:tasks.workunit.client.1.smithi148.stdout:1/729: symlink db/d46/d4c/led 0 2024-09-21T04:15:22.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/730: truncate db/d24/d2e/daf/fb1 1727589 0 2024-09-21T04:15:22.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/731: chown db/d46/c9d 658465 1 2024-09-21T04:15:22.836 INFO:tasks.workunit.client.1.smithi148.stdout:1/732: stat db/d46/d4c/f5b 0 2024-09-21T04:15:22.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/677: rename d0/d7/d13/d4b/da7/ce6 to d0/d30/d34/d3e/d32/ceb 0 2024-09-21T04:15:22.838 INFO:tasks.workunit.client.0.smithi110.stdout:8/678: mkdir d0/d30/d34/d3e/d60/dec 0 2024-09-21T04:15:22.840 INFO:tasks.workunit.client.0.smithi110.stdout:8/679: dread d0/d7/d23/d82/f83 [0,4194304] 0 2024-09-21T04:15:22.842 INFO:tasks.workunit.client.0.smithi110.stdout:8/680: mknod d0/d30/d34/d3e/d5d/dd7/ced 0 2024-09-21T04:15:22.842 INFO:tasks.workunit.client.0.smithi110.stdout:7/654: dread dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:22.844 INFO:tasks.workunit.client.0.smithi110.stdout:8/681: rename d0/d7/c7a to d0/d30/d34/d3e/d5d/cee 0 2024-09-21T04:15:22.844 INFO:tasks.workunit.client.0.smithi110.stdout:7/655: dread dd/d14/f41 [0,4194304] 0 2024-09-21T04:15:22.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/720: write d0/d1a/d23/d53/fc1 [2144702,111623] 0 2024-09-21T04:15:22.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/682: mknod d0/d30/d34/d3e/d5d/dd7/dda/cef 0 2024-09-21T04:15:22.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/683: stat d0/d7/f1b 0 2024-09-21T04:15:22.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/684: readlink d0/d30/d34/d3e/d60/lb6 0 2024-09-21T04:15:22.845 INFO:tasks.workunit.client.0.smithi110.stdout:8/685: creat d0/d30/d9d/ff0 x:0 0 0 2024-09-21T04:15:22.846 INFO:tasks.workunit.client.1.smithi148.stdout:6/721: creat d0/d34/d38/dc6/dc9/fd8 x:0 0 0 2024-09-21T04:15:22.849 INFO:tasks.workunit.client.0.smithi110.stdout:8/686: rename d0/d30/d34/d3e/d5d/l72 to d0/d7/d23/d82/d92/lf1 0 2024-09-21T04:15:22.851 INFO:tasks.workunit.client.0.smithi110.stdout:7/656: dread dd/d1c/dad/db2/d4a/d52/faf [0,4194304] 0 2024-09-21T04:15:22.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/722: rename d0/d34/d97/f1f to d0/d1a/d23/d27/d80/da4/fd9 0 2024-09-21T04:15:22.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/553: dwrite d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:22.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/554: write d0/d12/f82 [1101952,37699] 0 2024-09-21T04:15:22.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/555: stat d0/d12/d16/d1a/d95 0 2024-09-21T04:15:22.853 INFO:tasks.workunit.client.0.smithi110.stdout:8/687: mkdir d0/d7/d23/d6c/df2 0 2024-09-21T04:15:22.853 INFO:tasks.workunit.client.1.smithi148.stdout:7/716: dwrite d5/d14/d25/d40/db9/ff6 [0,4194304] 0 2024-09-21T04:15:22.854 INFO:tasks.workunit.client.1.smithi148.stdout:9/748: dwrite d5/d2b/d4e/d95/fad [0,4194304] 0 2024-09-21T04:15:22.854 INFO:tasks.workunit.client.1.smithi148.stdout:9/749: fsync d5/d2b/dbe/f47 0 2024-09-21T04:15:22.854 INFO:tasks.workunit.client.1.smithi148.stdout:4/796: dwrite d4/d66/d7f/d2a/fc7 [0,4194304] 0 2024-09-21T04:15:22.855 INFO:tasks.workunit.client.0.smithi110.stdout:7/657: rename dd/d14/l53 to dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/ld5 0 2024-09-21T04:15:22.855 INFO:tasks.workunit.client.0.smithi110.stdout:2/518: dwrite d2/d1a/d1f/d5f/f95 [0,4194304] 0 2024-09-21T04:15:22.855 INFO:tasks.workunit.client.1.smithi148.stdout:0/727: dwrite d5/d6/d2a/f79 [0,4194304] 0 2024-09-21T04:15:22.855 INFO:tasks.workunit.client.1.smithi148.stdout:0/728: fsync d5/d6/d2a/d2b/d84/daf/f90 0 2024-09-21T04:15:22.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/723: mkdir d0/d1a/d23/d27/d62/dc4/dda 0 2024-09-21T04:15:22.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/724: chown d0/d1a/d23/d27/d62/d9a/d92/fa2 216169 1 2024-09-21T04:15:22.856 INFO:tasks.workunit.client.0.smithi110.stdout:3/556: symlink d0/lc0 0 2024-09-21T04:15:22.856 INFO:tasks.workunit.client.0.smithi110.stdout:4/804: dwrite d17/d1e/d21/f5b [4194304,4194304] 0 2024-09-21T04:15:22.857 INFO:tasks.workunit.client.0.smithi110.stdout:8/688: creat d0/d7/d89/ff3 x:0 0 0 2024-09-21T04:15:22.858 INFO:tasks.workunit.client.0.smithi110.stdout:2/519: dread d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:22.859 INFO:tasks.workunit.client.0.smithi110.stdout:3/557: dread d0/d12/d16/f6a [0,4194304] 0 2024-09-21T04:15:22.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/658: symlink dd/d1c/dad/db2/d56/d66/da6/d98/ld6 0 2024-09-21T04:15:22.868 INFO:tasks.workunit.client.0.smithi110.stdout:8/689: write d0/d7/f36 [3733567,118914] 0 2024-09-21T04:15:22.868 INFO:tasks.workunit.client.0.smithi110.stdout:8/690: creat d0/d7/d23/d52/d8b/ff4 x:0 0 0 2024-09-21T04:15:22.871 INFO:tasks.workunit.client.1.smithi148.stdout:7/717: creat d5/d8/d6f/d7d/df5/ffb x:0 0 0 2024-09-21T04:15:22.871 INFO:tasks.workunit.client.1.smithi148.stdout:7/718: creat d5/d14/d25/d40/dab/ffc x:0 0 0 2024-09-21T04:15:22.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/719: read d5/d8/d6f/d7d/d90/f5d [89640,70236] 0 2024-09-21T04:15:22.873 INFO:tasks.workunit.client.1.smithi148.stdout:7/720: dread - d5/d14/d25/d40/f41 zero size 2024-09-21T04:15:22.875 INFO:tasks.workunit.client.1.smithi148.stdout:9/750: truncate d5/d1e/d37/fb0 138984 0 2024-09-21T04:15:22.875 INFO:tasks.workunit.client.1.smithi148.stdout:9/751: fsync d5/d1e/f90 0 2024-09-21T04:15:22.881 INFO:tasks.workunit.client.1.smithi148.stdout:4/797: truncate d4/d87/f32 69343 0 2024-09-21T04:15:22.886 INFO:tasks.workunit.client.1.smithi148.stdout:0/729: unlink d5/d6/d7/d18/c30 0 2024-09-21T04:15:22.886 INFO:tasks.workunit.client.0.smithi110.stdout:4/805: mkdir d17/d1e/d7b/dc5/d113 0 2024-09-21T04:15:22.886 INFO:tasks.workunit.client.0.smithi110.stdout:4/806: truncate d17/d1e/d77/d10c/f91 757035 0 2024-09-21T04:15:22.888 INFO:tasks.workunit.client.0.smithi110.stdout:2/520: mknod d2/c99 0 2024-09-21T04:15:22.894 INFO:tasks.workunit.client.0.smithi110.stdout:7/659: symlink dd/d1c/dad/db2/d4a/da5/ld7 0 2024-09-21T04:15:22.895 INFO:tasks.workunit.client.1.smithi148.stdout:9/752: mknod d5/d1e/d37/d46/c109 0 2024-09-21T04:15:22.896 INFO:tasks.workunit.client.1.smithi148.stdout:9/753: write d5/f8a [647557,32855] 0 2024-09-21T04:15:22.898 INFO:tasks.workunit.client.1.smithi148.stdout:6/725: rmdir d0/d1a/d23/dc/d68/d5b 39 2024-09-21T04:15:22.898 INFO:tasks.workunit.client.1.smithi148.stdout:6/726: truncate d0/d1a/d23/d32/d93/fca 363466 0 2024-09-21T04:15:22.898 INFO:tasks.workunit.client.1.smithi148.stdout:6/727: fsync d0/d1a/d23/dc/f8e 0 2024-09-21T04:15:22.900 INFO:tasks.workunit.client.1.smithi148.stdout:7/721: getdents d5/d14/d25/d40/dab/dad 0 2024-09-21T04:15:22.903 INFO:tasks.workunit.client.0.smithi110.stdout:8/691: rename d0/d7/d13/d1c/f1d to d0/d30/d34/d3e/d32/ff5 0 2024-09-21T04:15:22.903 INFO:tasks.workunit.client.0.smithi110.stdout:8/692: creat d0/d30/d34/d3e/d5d/dd7/ff6 x:0 0 0 2024-09-21T04:15:22.903 INFO:tasks.workunit.client.0.smithi110.stdout:8/693: dread - d0/d7/d13/fbc zero size 2024-09-21T04:15:22.903 INFO:tasks.workunit.client.0.smithi110.stdout:8/694: chown d0/d7/d13/f37 3 1 2024-09-21T04:15:22.904 INFO:tasks.workunit.client.1.smithi148.stdout:9/754: mkdir d5/d60/d28/d2f/d10a 0 2024-09-21T04:15:22.905 INFO:tasks.workunit.client.1.smithi148.stdout:4/798: dwrite d4/d12/d16/d29/dbd/f9e [0,4194304] 0 2024-09-21T04:15:22.906 INFO:tasks.workunit.client.1.smithi148.stdout:0/730: dwrite d5/d6/d2a/d2b/d84/daf/fc1 [0,4194304] 0 2024-09-21T04:15:22.906 INFO:tasks.workunit.client.1.smithi148.stdout:0/731: write d5/d6/d7/d11/fca [83650,11063] 0 2024-09-21T04:15:22.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/807: mkdir d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d114 0 2024-09-21T04:15:22.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/808: creat d17/d3e/f115 x:0 0 0 2024-09-21T04:15:22.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/809: dread d17/d1e/d77/d10c/f4a [0,4194304] 0 2024-09-21T04:15:22.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/728: link d0/l2f d0/d1a/ldb 0 2024-09-21T04:15:22.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/729: chown d0/d1a/d23/d27/d80/db0 0 1 2024-09-21T04:15:22.909 INFO:tasks.workunit.client.0.smithi110.stdout:3/558: rmdir d0/d12/d3c/dad 39 2024-09-21T04:15:22.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/559: chown d0/c78 3177668 1 2024-09-21T04:15:22.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/560: fdatasync d0/d12/f9d 0 2024-09-21T04:15:22.911 INFO:tasks.workunit.client.0.smithi110.stdout:7/660: dwrite dd/d14/d17/f3a [4194304,4194304] 0 2024-09-21T04:15:22.911 INFO:tasks.workunit.client.0.smithi110.stdout:7/661: write dd/d1c/dad/db2/d56/d66/da6/f8e [784616,58599] 0 2024-09-21T04:15:22.911 INFO:tasks.workunit.client.0.smithi110.stdout:7/662: write dd/d1c/dad/db2/d56/fac [439640,23153] 0 2024-09-21T04:15:22.911 INFO:tasks.workunit.client.0.smithi110.stdout:7/663: write dd/d14/f46 [847752,71201] 0 2024-09-21T04:15:22.912 INFO:tasks.workunit.client.1.smithi148.stdout:7/722: getdents d5/d8/d6f/d7d/d90 0 2024-09-21T04:15:22.913 INFO:tasks.workunit.client.1.smithi148.stdout:9/755: mknod d5/d1e/c10b 0 2024-09-21T04:15:22.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/521: rename d2/f16 to d2/d81/f9a 0 2024-09-21T04:15:22.916 INFO:tasks.workunit.client.1.smithi148.stdout:4/799: creat d4/d66/d7f/d9a/ff8 x:0 0 0 2024-09-21T04:15:22.928 INFO:tasks.workunit.client.0.smithi110.stdout:8/695: rename d0/d30/d34/d3e/d5d/fde to d0/d30/d54/d6b/da8/ff7 0 2024-09-21T04:15:22.931 INFO:tasks.workunit.client.0.smithi110.stdout:3/561: write d0/d22/d29/f5f [1852186,1274] 0 2024-09-21T04:15:22.931 INFO:tasks.workunit.client.0.smithi110.stdout:4/810: dwrite f14 [0,4194304] 0 2024-09-21T04:15:22.933 INFO:tasks.workunit.client.0.smithi110.stdout:7/664: rmdir dd/d1c/dad/db2/d56/d66 39 2024-09-21T04:15:22.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/732: symlink d5/d6/d7/le9 0 2024-09-21T04:15:22.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/733: chown d5/d6/d2a/d2b/fd3 5349 1 2024-09-21T04:15:22.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/734: truncate d5/d6/d2a/d3f/d8b/fc2 3633087 0 2024-09-21T04:15:22.936 INFO:tasks.workunit.client.1.smithi148.stdout:0/735: chown d5/d6/d2a/d3f/d6d/da3/dcc 14053 1 2024-09-21T04:15:22.937 INFO:tasks.workunit.client.1.smithi148.stdout:4/800: dwrite d4/d12/d16/d29/dbd/f9e [0,4194304] 0 2024-09-21T04:15:22.937 INFO:tasks.workunit.client.0.smithi110.stdout:2/522: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:22.938 INFO:tasks.workunit.client.1.smithi148.stdout:6/730: mknod d0/d34/da0/dd4/cdc 0 2024-09-21T04:15:22.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/723: symlink d5/d14/d25/d40/dab/lfd 0 2024-09-21T04:15:22.941 INFO:tasks.workunit.client.0.smithi110.stdout:4/811: mknod d17/d3e/dbd/c116 0 2024-09-21T04:15:22.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/756: unlink d5/d2b/dbe/f47 0 2024-09-21T04:15:22.947 INFO:tasks.workunit.client.0.smithi110.stdout:2/523: rename d2/d50/d56/d97 to d2/d1a/d1f/d29/d19/d62/d86/d9b 0 2024-09-21T04:15:22.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/736: dread d5/d6/d7/d11/f29 [0,4194304] 0 2024-09-21T04:15:22.949 INFO:tasks.workunit.client.0.smithi110.stdout:4/812: dread d17/d1e/d77/d10c/fa3 [4194304,4194304] 0 2024-09-21T04:15:22.950 INFO:tasks.workunit.client.1.smithi148.stdout:6/731: dread d0/d34/d97/f3f [0,4194304] 0 2024-09-21T04:15:22.950 INFO:tasks.workunit.client.1.smithi148.stdout:6/732: dread - d0/d1a/d23/dc/f8e zero size 2024-09-21T04:15:22.950 INFO:tasks.workunit.client.0.smithi110.stdout:2/524: mkdir d2/d50/d9c 0 2024-09-21T04:15:22.952 INFO:tasks.workunit.client.0.smithi110.stdout:4/813: truncate d17/d1e/d77/d10c/d20/fa8 107797 0 2024-09-21T04:15:22.952 INFO:tasks.workunit.client.0.smithi110.stdout:4/814: write d17/d1e/d77/d10c/f112 [78108,4701] 0 2024-09-21T04:15:22.952 INFO:tasks.workunit.client.1.smithi148.stdout:4/801: rmdir d4/d66/d7f/d2a/da9 39 2024-09-21T04:15:22.952 INFO:tasks.workunit.client.1.smithi148.stdout:4/802: chown d4/d12/d16/df0/cb8 996229 1 2024-09-21T04:15:22.955 INFO:tasks.workunit.client.1.smithi148.stdout:7/724: rename d5/d14/c45 to d5/d8/d6f/d7d/d8a/cfe 0 2024-09-21T04:15:22.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/815: symlink d17/d1e/d77/d10c/d20/dca/l117 0 2024-09-21T04:15:22.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/816: truncate d17/d1e/d77/d10c/d20/d43/d51/d7a/d86/fc9 843125 0 2024-09-21T04:15:22.958 INFO:tasks.workunit.client.0.smithi110.stdout:4/817: chown d17/f3c 3161 1 2024-09-21T04:15:22.958 INFO:tasks.workunit.client.0.smithi110.stdout:4/818: stat d17/d1e/d77/d10c/d20/c71 0 2024-09-21T04:15:22.958 INFO:tasks.workunit.client.0.smithi110.stdout:4/819: dread - d17/d1e/d21/f4d zero size 2024-09-21T04:15:22.963 INFO:tasks.workunit.client.0.smithi110.stdout:8/696: dwrite d0/d7/f2a [0,4194304] 0 2024-09-21T04:15:22.963 INFO:tasks.workunit.client.1.smithi148.stdout:4/803: link d4/d12/d16/d29/d61/d6c/d9d/fb0 d4/d66/d7f/d9a/ff9 0 2024-09-21T04:15:22.963 INFO:tasks.workunit.client.1.smithi148.stdout:4/804: stat d4/d3a/f53 0 2024-09-21T04:15:22.963 INFO:tasks.workunit.client.1.smithi148.stdout:4/805: read d4/f10 [1994293,30468] 0 2024-09-21T04:15:22.965 INFO:tasks.workunit.client.1.smithi148.stdout:9/757: rename f0 to d5/d1f/d81/f10c 0 2024-09-21T04:15:22.966 INFO:tasks.workunit.client.1.smithi148.stdout:9/758: creat d5/d1e/f10d x:0 0 0 2024-09-21T04:15:22.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/562: dwrite d0/d12/f82 [0,4194304] 0 2024-09-21T04:15:22.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/563: fdatasync d0/d12/f23 0 2024-09-21T04:15:22.967 INFO:tasks.workunit.client.1.smithi148.stdout:7/725: mkdir d5/d14/d25/d40/db9/dff 0 2024-09-21T04:15:22.968 INFO:tasks.workunit.client.0.smithi110.stdout:2/525: dwrite d2/d50/d7b/f91 [0,4194304] 0 2024-09-21T04:15:22.969 INFO:tasks.workunit.client.0.smithi110.stdout:2/526: fsync d2/d1a/d1f/d5f/f95 0 2024-09-21T04:15:22.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/806: getdents d4/d66/d7f/d2a/da9 0 2024-09-21T04:15:22.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/807: dread d4/d12/d16/fcb [0,4194304] 0 2024-09-21T04:15:22.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/808: dread - d4/d66/fbe zero size 2024-09-21T04:15:22.977 INFO:tasks.workunit.client.1.smithi148.stdout:4/809: chown d4/d87/c31 6689518 1 2024-09-21T04:15:22.978 INFO:tasks.workunit.client.0.smithi110.stdout:4/820: dread d17/d1e/d77/d10c/fa3 [0,4194304] 0 2024-09-21T04:15:22.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/737: rename d5/d6/d2a/fd7 to d5/d6/d7/dd0/fea 0 2024-09-21T04:15:22.980 INFO:tasks.workunit.client.1.smithi148.stdout:6/733: truncate d0/d1a/d23/d32/d93/fca 353179 0 2024-09-21T04:15:22.980 INFO:tasks.workunit.client.1.smithi148.stdout:7/726: dread d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:22.985 INFO:tasks.workunit.client.1.smithi148.stdout:9/759: truncate d5/d2b/dbe/f52 2159459 0 2024-09-21T04:15:22.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/727: dread d5/d14/d25/d79/d86/fa3 [0,4194304] 0 2024-09-21T04:15:22.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/728: fsync d5/d14/d25/fdd 0 2024-09-21T04:15:22.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/729: write d5/d14/d25/d79/fe2 [266885,84305] 0 2024-09-21T04:15:22.988 INFO:tasks.workunit.client.1.smithi148.stdout:7/730: read d5/f3d [1571076,93110] 0 2024-09-21T04:15:22.990 INFO:tasks.workunit.client.0.smithi110.stdout:3/564: dwrite d0/d22/f72 [0,4194304] 0 2024-09-21T04:15:22.993 INFO:tasks.workunit.client.1.smithi148.stdout:5/757: sync 2024-09-21T04:15:22.994 INFO:tasks.workunit.client.1.smithi148.stdout:9/760: link d5/d2b/c31 d5/d1e/d37/d50/c10e 0 2024-09-21T04:15:22.994 INFO:tasks.workunit.client.1.smithi148.stdout:9/761: chown d5/d2b/dce/df6 876214682 1 2024-09-21T04:15:22.994 INFO:tasks.workunit.client.1.smithi148.stdout:9/762: read d5/d1e/f2d [3737745,11670] 0 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/733: sync 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/843: sync 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/670: sync 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:8/675: sync 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/844: dread - d0/d9/d16/fd1 zero size 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/671: fdatasync d3/d5/d6/d2b/d96/f9d 0 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/672: stat d3/d5/d6/d12/d43/d54/fc6 0 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:8/676: creat dc/d3e/fd2 x:0 0 0 2024-09-21T04:15:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/673: creat d3/d5/d5c/fdb x:0 0 0 2024-09-21T04:15:22.996 INFO:tasks.workunit.client.1.smithi148.stdout:2/845: read d0/d2f/f84 [39943,23608] 0 2024-09-21T04:15:22.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/734: dwrite d0/d1a/d23/f61 [0,4194304] 0 2024-09-21T04:15:22.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/735: write d0/d34/d38/dc6/dc9/fd8 [104214,90682] 0 2024-09-21T04:15:22.996 INFO:tasks.workunit.client.1.smithi148.stdout:4/810: getdents d4/d12/d78 0 2024-09-21T04:15:22.998 INFO:tasks.workunit.client.0.smithi110.stdout:4/821: creat d17/d1e/d77/d10c/d20/dca/f118 x:0 0 0 2024-09-21T04:15:23.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/697: rename d0/d7/d23/d52/d8b to d0/d7/d13/d46/df8 0 2024-09-21T04:15:23.005 INFO:tasks.workunit.client.1.smithi148.stdout:0/738: rename d5/d6/d2a/d2b/d84 to d5/d6/d2a/d2b/d99/deb 0 2024-09-21T04:15:23.007 INFO:tasks.workunit.client.1.smithi148.stdout:5/758: mkdir d3/d14/d1f/d26/d59/d97/d102 0 2024-09-21T04:15:23.009 INFO:tasks.workunit.client.0.smithi110.stdout:3/565: creat d0/d12/d16/d1a/fc1 x:0 0 0 2024-09-21T04:15:23.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/763: rename d5/d2b/d4e/db3/ce9 to d5/d2b/d4e/db3/c10f 0 2024-09-21T04:15:23.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/764: write d5/d1f/d81/dcf/d106/d80/d104/f99 [1651590,92200] 0 2024-09-21T04:15:23.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/765: readlink d5/l1c 0 2024-09-21T04:15:23.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/766: chown d5/d60/d28/fbb 2 1 2024-09-21T04:15:23.012 INFO:tasks.workunit.client.1.smithi148.stdout:9/767: chown d5/d1e/d37/d50 49417 1 2024-09-21T04:15:23.012 INFO:tasks.workunit.client.1.smithi148.stdout:1/734: dwrite db/d46/f66 [0,4194304] 0 2024-09-21T04:15:23.012 INFO:tasks.workunit.client.0.smithi110.stdout:4/822: symlink d17/d3e/l119 0 2024-09-21T04:15:23.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/698: symlink d0/d30/d34/lf9 0 2024-09-21T04:15:23.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/699: dread - d0/d7/d23/d6c/f90 zero size 2024-09-21T04:15:23.019 INFO:tasks.workunit.client.0.smithi110.stdout:2/527: rename d2/d1a/d1f/d29/d19/d62 to d2/d1a/d1f/d5f/d9d 0 2024-09-21T04:15:23.020 INFO:tasks.workunit.client.1.smithi148.stdout:8/677: creat dc/d13/d3c/d81/fd3 x:0 0 0 2024-09-21T04:15:23.022 INFO:tasks.workunit.client.0.smithi110.stdout:4/823: symlink d17/d1e/d77/d10c/d20/d43/d51/d52/l11a 0 2024-09-21T04:15:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:3/674: mknod d3/d1f/d4d/d82/d23/d28/cdc 0 2024-09-21T04:15:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/678: read dc/d13/d28/d62/fb6 [518718,55260] 0 2024-09-21T04:15:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:3/675: stat d3/d5/d6/d2b/d96/d99 0 2024-09-21T04:15:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/679: write dc/d43/f47 [506344,112857] 0 2024-09-21T04:15:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/680: truncate dc/d13/d3c/d81/d85/da1/fca 747839 0 2024-09-21T04:15:23.025 INFO:tasks.workunit.client.0.smithi110.stdout:3/566: write d0/d12/d16/d1a/f4e [113768,25093] 0 2024-09-21T04:15:23.025 INFO:tasks.workunit.client.0.smithi110.stdout:3/567: write d0/d12/d3c/d5a/d70/f84 [490066,96606] 0 2024-09-21T04:15:23.026 INFO:tasks.workunit.client.0.smithi110.stdout:8/700: rename d0/d7/d13/d1c/d41/fba to d0/d7/d23/d82/ffa 0 2024-09-21T04:15:23.026 INFO:tasks.workunit.client.1.smithi148.stdout:0/739: dwrite d5/d82/f87 [0,4194304] 0 2024-09-21T04:15:23.027 INFO:tasks.workunit.client.1.smithi148.stdout:0/740: fdatasync d5/dcb/fe4 0 2024-09-21T04:15:23.027 INFO:tasks.workunit.client.1.smithi148.stdout:0/741: write d5/d6/d2a/d2b/d99/deb/daf/dc8/fe3 [474335,62728] 0 2024-09-21T04:15:23.027 INFO:tasks.workunit.client.1.smithi148.stdout:0/742: readlink d5/d6/d2a/d3f/d6d/da3/dcc/ld2 0 2024-09-21T04:15:23.027 INFO:tasks.workunit.client.1.smithi148.stdout:2/846: mkdir d0/d9/d59/d10b 0 2024-09-21T04:15:23.027 INFO:tasks.workunit.client.1.smithi148.stdout:2/847: write d0/d9/d16/d55/d71/dc2/dcb/fe6 [228737,15680] 0 2024-09-21T04:15:23.028 INFO:tasks.workunit.client.1.smithi148.stdout:2/848: dread - d0/d9/d59/fc1 zero size 2024-09-21T04:15:23.028 INFO:tasks.workunit.client.1.smithi148.stdout:2/849: creat d0/d9/d16/d9e/f10c x:0 0 0 2024-09-21T04:15:23.028 INFO:tasks.workunit.client.1.smithi148.stdout:1/735: dwrite db/d24/d2a/da4/fbc [0,4194304] 0 2024-09-21T04:15:23.028 INFO:tasks.workunit.client.0.smithi110.stdout:2/528: mknod d2/d50/d56/c9e 0 2024-09-21T04:15:23.031 INFO:tasks.workunit.client.0.smithi110.stdout:3/568: symlink d0/d12/d2a/d80/d9c/lc2 0 2024-09-21T04:15:23.032 INFO:tasks.workunit.client.1.smithi148.stdout:6/736: mkdir d0/d1a/d23/d27/ddd 0 2024-09-21T04:15:23.032 INFO:tasks.workunit.client.1.smithi148.stdout:6/737: fsync d0/d1a/f83 0 2024-09-21T04:15:23.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/811: truncate d4/d66/d7f/d2a/fc7 2432533 0 2024-09-21T04:15:23.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/812: dread - d4/d66/d7f/d9a/dc2/fe1 zero size 2024-09-21T04:15:23.035 INFO:tasks.workunit.client.0.smithi110.stdout:4/824: rename d17/d34/c111 to d17/d1e/d7b/c11b 0 2024-09-21T04:15:23.038 INFO:tasks.workunit.client.1.smithi148.stdout:7/731: rename d5/d8/d6f/dcd/fd7 to d5/d14/d25/d40/dab/dad/f100 0 2024-09-21T04:15:23.038 INFO:tasks.workunit.client.1.smithi148.stdout:7/732: fsync d5/d8/d6f/d7d/fdc 0 2024-09-21T04:15:23.039 INFO:tasks.workunit.client.0.smithi110.stdout:8/701: mknod d0/d30/d9d/dad/cfb 0 2024-09-21T04:15:23.039 INFO:tasks.workunit.client.0.smithi110.stdout:8/702: fdatasync d0/d7/d23/d82/ffa 0 2024-09-21T04:15:23.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/759: mknod d3/d14/d1f/db1/db8/c103 0 2024-09-21T04:15:23.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/760: chown d3/l6b 112466457 1 2024-09-21T04:15:23.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/825: creat d17/d3e/d6b/dc7/f11c x:0 0 0 2024-09-21T04:15:23.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/826: dread - d17/d1e/d77/d10c/d20/d43/fa2 zero size 2024-09-21T04:15:23.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/827: truncate d17/d3e/f9a 873931 0 2024-09-21T04:15:23.042 INFO:tasks.workunit.client.1.smithi148.stdout:9/768: mkdir d5/d60/d28/d73/d110 0 2024-09-21T04:15:23.044 INFO:tasks.workunit.client.1.smithi148.stdout:6/738: dread d0/d8/f29 [0,4194304] 0 2024-09-21T04:15:23.044 INFO:tasks.workunit.client.1.smithi148.stdout:6/739: dread - d0/d1a/d23/d27/d62/d9a/d92/fa2 zero size 2024-09-21T04:15:23.046 INFO:tasks.workunit.client.0.smithi110.stdout:8/703: link d0/d7/d13/d4b/ddc/c9c d0/d7/cfc 0 2024-09-21T04:15:23.046 INFO:tasks.workunit.client.0.smithi110.stdout:8/704: chown d0/d7/d13/fb1 26761 1 2024-09-21T04:15:23.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/676: truncate d3/d5/f35 2028530 0 2024-09-21T04:15:23.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/569: write d0/d22/f72 [985550,107414] 0 2024-09-21T04:15:23.049 INFO:tasks.workunit.client.0.smithi110.stdout:4/828: symlink d17/d1e/d77/d10c/l11d 0 2024-09-21T04:15:23.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/813: dread d4/fd8 [0,4194304] 0 2024-09-21T04:15:23.052 INFO:tasks.workunit.client.1.smithi148.stdout:0/743: write d5/d6/d2a/d2b/f55 [803459,113879] 0 2024-09-21T04:15:23.053 INFO:tasks.workunit.client.0.smithi110.stdout:3/570: rmdir d0/d12/d16/d1a/d6f/db9 0 2024-09-21T04:15:23.055 INFO:tasks.workunit.client.0.smithi110.stdout:8/705: rename d0/d7/d23/c9a to d0/cfd 0 2024-09-21T04:15:23.057 INFO:tasks.workunit.client.0.smithi110.stdout:3/571: symlink d0/d12/d16/d1a/d6f/lc3 0 2024-09-21T04:15:23.059 INFO:tasks.workunit.client.0.smithi110.stdout:8/706: creat d0/d7/d13/dc5/ffe x:0 0 0 2024-09-21T04:15:23.061 INFO:tasks.workunit.client.1.smithi148.stdout:1/736: symlink db/dd/db9/lee 0 2024-09-21T04:15:23.061 INFO:tasks.workunit.client.1.smithi148.stdout:7/733: dread d5/d8/fe3 [0,4194304] 0 2024-09-21T04:15:23.061 INFO:tasks.workunit.client.1.smithi148.stdout:7/734: stat d5/d14/d25/d40/f41 0 2024-09-21T04:15:23.061 INFO:tasks.workunit.client.1.smithi148.stdout:1/737: creat db/d46/d4c/d82/fef x:0 0 0 2024-09-21T04:15:23.061 INFO:tasks.workunit.client.1.smithi148.stdout:1/738: chown db/d24/d2e/l3d 0 1 2024-09-21T04:15:23.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/829: dread d17/d1e/f4c [0,4194304] 0 2024-09-21T04:15:23.062 INFO:tasks.workunit.client.0.smithi110.stdout:4/830: readlink d17/d1e/d77/d10c/d57/db9/le0 0 2024-09-21T04:15:23.062 INFO:tasks.workunit.client.0.smithi110.stdout:3/572: mkdir d0/d12/d2a/d80/dc4 0 2024-09-21T04:15:23.064 INFO:tasks.workunit.client.0.smithi110.stdout:4/831: mkdir d17/d34/df5/d11e 0 2024-09-21T04:15:23.065 INFO:tasks.workunit.client.1.smithi148.stdout:6/740: creat d0/d1a/d23/d27/fde x:0 0 0 2024-09-21T04:15:23.065 INFO:tasks.workunit.client.1.smithi148.stdout:6/741: write d0/d34/d97/f30 [700929,6872] 0 2024-09-21T04:15:23.066 INFO:tasks.workunit.client.1.smithi148.stdout:3/677: rmdir d3/d5/d6/d12/d15/da7 39 2024-09-21T04:15:23.067 INFO:tasks.workunit.client.0.smithi110.stdout:8/707: rename d0/d7/d6f/d86/le7 to d0/lff 0 2024-09-21T04:15:23.068 INFO:tasks.workunit.client.1.smithi148.stdout:4/814: mknod d4/d87/cfa 0 2024-09-21T04:15:23.070 INFO:tasks.workunit.client.0.smithi110.stdout:3/573: link d0/d12/d2a/f3e d0/d22/d29/fc5 0 2024-09-21T04:15:23.070 INFO:tasks.workunit.client.1.smithi148.stdout:0/744: link d5/d6/d2a/d2b/d99/deb/db4/fb5 d5/d6/d2a/d2b/d99/deb/daf/dc8/fec 0 2024-09-21T04:15:23.071 INFO:tasks.workunit.client.1.smithi148.stdout:2/850: dwrite d0/d9/d16/d55/d71/dc2/dcb/fd6 [0,4194304] 0 2024-09-21T04:15:23.071 INFO:tasks.workunit.client.0.smithi110.stdout:8/708: mkdir d0/d30/d34/d3e/d100 0 2024-09-21T04:15:23.072 INFO:tasks.workunit.client.0.smithi110.stdout:2/529: dwrite d2/d1a/d1f/d5f/f58 [0,4194304] 0 2024-09-21T04:15:23.072 INFO:tasks.workunit.client.0.smithi110.stdout:2/530: fdatasync d2/d1a/d1f/d2a/f61 0 2024-09-21T04:15:23.073 INFO:tasks.workunit.client.1.smithi148.stdout:9/769: dwrite d5/d2b/d4e/d95/ff0 [0,4194304] 0 2024-09-21T04:15:23.075 INFO:tasks.workunit.client.1.smithi148.stdout:1/739: getdents db/dd/d21/db0 0 2024-09-21T04:15:23.075 INFO:tasks.workunit.client.0.smithi110.stdout:3/574: mkdir d0/d12/d16/d1a/dc6 0 2024-09-21T04:15:23.076 INFO:tasks.workunit.client.1.smithi148.stdout:2/851: write d0/d9/d38/fcc [679951,55968] 0 2024-09-21T04:15:23.076 INFO:tasks.workunit.client.1.smithi148.stdout:2/852: write d0/d9/d16/d9e/f107 [891066,26298] 0 2024-09-21T04:15:23.077 INFO:tasks.workunit.client.1.smithi148.stdout:8/681: rename dc/d13/d76/lab to dc/d13/d39/ld4 0 2024-09-21T04:15:23.081 INFO:tasks.workunit.client.0.smithi110.stdout:2/531: mkdir d2/d1a/d1f/d5f/d9d/d86/d9f 0 2024-09-21T04:15:23.082 INFO:tasks.workunit.client.1.smithi148.stdout:6/742: creat d0/d1a/d23/fdf x:0 0 0 2024-09-21T04:15:23.085 INFO:tasks.workunit.client.1.smithi148.stdout:3/678: link d3/d1f/d4d/d82/d23/d28/f42 d3/fdd 0 2024-09-21T04:15:23.086 INFO:tasks.workunit.client.1.smithi148.stdout:3/679: read d3/d1f/f3e [855333,13170] 0 2024-09-21T04:15:23.087 INFO:tasks.workunit.client.1.smithi148.stdout:4/815: mkdir d4/d12/d16/d29/d61/d6c/d9d/dfb 0 2024-09-21T04:15:23.088 INFO:tasks.workunit.client.0.smithi110.stdout:3/575: dread d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:23.088 INFO:tasks.workunit.client.0.smithi110.stdout:3/576: fdatasync d0/d12/d2a/f61 0 2024-09-21T04:15:23.089 INFO:tasks.workunit.client.0.smithi110.stdout:2/532: write d2/d50/d7b/f91 [3006342,68343] 0 2024-09-21T04:15:23.093 INFO:tasks.workunit.client.0.smithi110.stdout:3/577: symlink d0/d12/d16/d1a/dac/lc7 0 2024-09-21T04:15:23.094 INFO:tasks.workunit.client.0.smithi110.stdout:2/533: creat d2/d1a/d1f/d65/fa0 x:0 0 0 2024-09-21T04:15:23.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/832: dwrite d17/d1e/d77/d10c/d57/db9/fc8 [0,4194304] 0 2024-09-21T04:15:23.094 INFO:tasks.workunit.client.0.smithi110.stdout:4/833: chown d17/d1e/d77/d10c/d20/c3a 119235 1 2024-09-21T04:15:23.097 INFO:tasks.workunit.client.0.smithi110.stdout:8/709: dwrite d0/d7/d13/d46/df8/ff4 [0,4194304] 0 2024-09-21T04:15:23.097 INFO:tasks.workunit.client.0.smithi110.stdout:3/578: mkdir d0/d12/d3c/d5a/dc8 0 2024-09-21T04:15:23.097 INFO:tasks.workunit.client.0.smithi110.stdout:3/579: truncate d0/d12/d3c/f4f 5061690 0 2024-09-21T04:15:23.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/534: mkdir d2/d1a/d1f/d2a/da1 0 2024-09-21T04:15:23.104 INFO:tasks.workunit.client.0.smithi110.stdout:4/834: link d17/d1e/d77/d10c/d20/dca/f10f d17/d1e/d77/d10c/d57/db9/f11f 0 2024-09-21T04:15:23.104 INFO:tasks.workunit.client.0.smithi110.stdout:4/835: chown d17/d1e/d77/d10c/d20/d43/d51/d7a/fc3 13 1 2024-09-21T04:15:23.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/770: creat d5/d60/d28/d2f/d10a/f111 x:0 0 0 2024-09-21T04:15:23.109 INFO:tasks.workunit.client.0.smithi110.stdout:8/710: creat d0/d7/d6f/f101 x:0 0 0 2024-09-21T04:15:23.109 INFO:tasks.workunit.client.1.smithi148.stdout:9/771: dread d5/fcc [0,4194304] 0 2024-09-21T04:15:23.109 INFO:tasks.workunit.client.1.smithi148.stdout:9/772: fdatasync d5/d2b/d4e/f97 0 2024-09-21T04:15:23.110 INFO:tasks.workunit.client.1.smithi148.stdout:1/740: mknod db/d24/d2e/d59/cf0 0 2024-09-21T04:15:23.110 INFO:tasks.workunit.client.1.smithi148.stdout:1/741: write db/dd/d21/fc6 [918675,13108] 0 2024-09-21T04:15:23.111 INFO:tasks.workunit.client.0.smithi110.stdout:3/580: creat d0/d12/d16/d1a/d95/fc9 x:0 0 0 2024-09-21T04:15:23.111 INFO:tasks.workunit.client.1.smithi148.stdout:2/853: mknod d0/d9/d16/d55/d71/dc2/dcb/c10d 0 2024-09-21T04:15:23.118 INFO:tasks.workunit.client.0.smithi110.stdout:8/711: symlink d0/d7/d23/dc1/l102 0 2024-09-21T04:15:23.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/836: write d17/d1e/d7b/f93 [252246,68907] 0 2024-09-21T04:15:23.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/837: creat d17/d3e/d6b/dc7/f120 x:0 0 0 2024-09-21T04:15:23.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/838: truncate d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/ffc 323824 0 2024-09-21T04:15:23.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/839: fsync d17/d1e/d77/d10c/f4a 0 2024-09-21T04:15:23.119 INFO:tasks.workunit.client.1.smithi148.stdout:4/816: dwrite d4/d66/d7f/f4a [0,4194304] 0 2024-09-21T04:15:23.121 INFO:tasks.workunit.client.0.smithi110.stdout:3/581: rename d0/d12/d3c/dad/cbc to d0/d12/d3c/d5a/d70/cca 0 2024-09-21T04:15:23.121 INFO:tasks.workunit.client.0.smithi110.stdout:3/582: creat d0/fcb x:0 0 0 2024-09-21T04:15:23.123 INFO:tasks.workunit.client.1.smithi148.stdout:1/742: dread f6 [4194304,4194304] 0 2024-09-21T04:15:23.125 INFO:tasks.workunit.client.1.smithi148.stdout:7/735: rename d5/d8/d6f/d7d/c36 to d5/daa/c101 0 2024-09-21T04:15:23.126 INFO:tasks.workunit.client.0.smithi110.stdout:3/583: read d0/d22/d29/fc5 [1731634,88132] 0 2024-09-21T04:15:23.126 INFO:tasks.workunit.client.1.smithi148.stdout:1/743: dread db/dd/d21/f93 [0,4194304] 0 2024-09-21T04:15:23.127 INFO:tasks.workunit.client.1.smithi148.stdout:1/744: write db/dd/d21/d89/dc9/fcd [236505,37118] 0 2024-09-21T04:15:23.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/682: getdents dc/d13/d76 0 2024-09-21T04:15:23.127 INFO:tasks.workunit.client.1.smithi148.stdout:8/683: chown dc/f41 5 1 2024-09-21T04:15:23.129 INFO:tasks.workunit.client.1.smithi148.stdout:1/745: dread db/d24/d2e/f9c [0,4194304] 0 2024-09-21T04:15:23.129 INFO:tasks.workunit.client.0.smithi110.stdout:2/535: rename d2/d1a/d1f/d29/c3e to d2/d1a/d60/ca2 0 2024-09-21T04:15:23.130 INFO:tasks.workunit.client.1.smithi148.stdout:6/743: mkdir d0/d1a/d23/d27/dc2/de0 0 2024-09-21T04:15:23.130 INFO:tasks.workunit.client.1.smithi148.stdout:6/744: readlink d0/d1a/d23/d27/lcf 0 2024-09-21T04:15:23.133 INFO:tasks.workunit.client.0.smithi110.stdout:2/536: dread d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:23.133 INFO:tasks.workunit.client.1.smithi148.stdout:9/773: mknod d5/d2b/dce/c112 0 2024-09-21T04:15:23.135 INFO:tasks.workunit.client.0.smithi110.stdout:3/584: mknod d0/d12/d3c/d5a/d70/ccc 0 2024-09-21T04:15:23.135 INFO:tasks.workunit.client.1.smithi148.stdout:1/746: dread db/d24/fd0 [0,4194304] 0 2024-09-21T04:15:23.135 INFO:tasks.workunit.client.1.smithi148.stdout:1/747: chown db/d46 12476853 1 2024-09-21T04:15:23.137 INFO:tasks.workunit.client.1.smithi148.stdout:4/817: symlink d4/d66/d7f/d2a/da9/lfc 0 2024-09-21T04:15:23.142 INFO:tasks.workunit.client.1.smithi148.stdout:7/736: rename d5/d14/d25/d40/d8b/c8f to d5/d8/d6f/d7d/d90/d9b/c102 0 2024-09-21T04:15:23.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/745: dread d0/d1a/d23/d27/d80/da4/fd9 [0,4194304] 0 2024-09-21T04:15:23.142 INFO:tasks.workunit.client.1.smithi148.stdout:4/818: symlink d4/d66/d7f/d2a/da9/lfd 0 2024-09-21T04:15:23.147 INFO:tasks.workunit.client.1.smithi148.stdout:8/684: rmdir dc/d13/d3c 39 2024-09-21T04:15:23.148 INFO:tasks.workunit.client.1.smithi148.stdout:8/685: fsync dc/f68 0 2024-09-21T04:15:23.148 INFO:tasks.workunit.client.1.smithi148.stdout:8/686: chown dc/d43 0 1 2024-09-21T04:15:23.148 INFO:tasks.workunit.client.1.smithi148.stdout:8/687: read - dc/d3e/fa0 zero size 2024-09-21T04:15:23.148 INFO:tasks.workunit.client.0.smithi110.stdout:0/962: sync 2024-09-21T04:15:23.148 INFO:tasks.workunit.client.0.smithi110.stdout:0/963: write d4/dd/d53/feb [331986,77501] 0 2024-09-21T04:15:23.149 INFO:tasks.workunit.client.1.smithi148.stdout:7/737: symlink d5/d8/d6f/d7d/d90/dee/l103 0 2024-09-21T04:15:23.149 INFO:tasks.workunit.client.1.smithi148.stdout:7/738: fsync d5/d14/f78 0 2024-09-21T04:15:23.149 INFO:tasks.workunit.client.1.smithi148.stdout:6/746: getdents d0/d59 0 2024-09-21T04:15:23.149 INFO:tasks.workunit.client.1.smithi148.stdout:6/747: creat d0/d1a/d23/d27/fe1 x:0 0 0 2024-09-21T04:15:23.151 INFO:tasks.workunit.client.1.smithi148.stdout:4/819: creat d4/d66/d7f/d9a/dce/ffe x:0 0 0 2024-09-21T04:15:23.151 INFO:tasks.workunit.client.0.smithi110.stdout:2/537: dread d2/d1a/d1f/d5f/f24 [4194304,4194304] 0 2024-09-21T04:15:23.151 INFO:tasks.workunit.client.0.smithi110.stdout:2/538: chown d2/d1a/d1f/d5f/f95 81822 1 2024-09-21T04:15:23.152 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:22 smithi148 ceph-mon[34957]: pgmap v8: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 158 MiB/s rd, 188 MiB/s wr, 302 op/s 2024-09-21T04:15:23.152 INFO:tasks.workunit.client.1.smithi148.stdout:8/688: rename dc/d13/d3c/d81/db0/lbc to dc/d13/d76/d5d/d78/ld5 0 2024-09-21T04:15:23.153 INFO:tasks.workunit.client.0.smithi110.stdout:8/712: dwrite d0/d7/d13/d4b/da7/fac [0,4194304] 0 2024-09-21T04:15:23.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/713: dread d0/d7/d13/d4b/ddc/f95 [0,4194304] 0 2024-09-21T04:15:23.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/714: fdatasync d0/d7/d13/fbc 0 2024-09-21T04:15:23.156 INFO:tasks.workunit.client.1.smithi148.stdout:6/748: mknod d0/d1a/d23/d27/d62/dc4/dda/ce2 0 2024-09-21T04:15:23.158 INFO:tasks.workunit.client.1.smithi148.stdout:4/820: mkdir d4/d66/d7f/d9a/dc2/dff 0 2024-09-21T04:15:23.159 INFO:tasks.workunit.client.1.smithi148.stdout:7/739: dread d5/d8/d6f/dcd/fe6 [0,4194304] 0 2024-09-21T04:15:23.160 INFO:tasks.workunit.client.1.smithi148.stdout:6/749: mknod d0/d8/d85/ce3 0 2024-09-21T04:15:23.160 INFO:tasks.workunit.client.1.smithi148.stdout:6/750: write d0/d1a/d23/d32/d93/fca [616159,17537] 0 2024-09-21T04:15:23.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/740: creat d5/d8/d6f/d7d/d90/dee/f104 x:0 0 0 2024-09-21T04:15:23.164 INFO:tasks.workunit.client.0.smithi110.stdout:8/715: mknod d0/d30/d34/d3e/d5d/dd7/c103 0 2024-09-21T04:15:23.165 INFO:tasks.workunit.client.1.smithi148.stdout:6/751: rename d0/d34/d97/d4f/fd5 to d0/d8/dba/fe4 0 2024-09-21T04:15:23.166 INFO:tasks.workunit.client.1.smithi148.stdout:7/741: rmdir d5/d14/d25/d79/d86 39 2024-09-21T04:15:23.166 INFO:tasks.workunit.client.1.smithi148.stdout:6/752: symlink d0/d34/le5 0 2024-09-21T04:15:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:22 smithi110 ceph-mon[27723]: pgmap v8: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 158 MiB/s rd, 188 MiB/s wr, 302 op/s 2024-09-21T04:15:23.167 INFO:tasks.workunit.client.1.smithi148.stdout:7/742: mknod d5/d8/d6f/d7d/c105 0 2024-09-21T04:15:23.167 INFO:tasks.workunit.client.1.smithi148.stdout:7/743: write d5/d8/f1e [475946,95818] 0 2024-09-21T04:15:23.167 INFO:tasks.workunit.client.1.smithi148.stdout:7/744: fdatasync d5/d8/d6f/fc9 0 2024-09-21T04:15:23.168 INFO:tasks.workunit.client.1.smithi148.stdout:6/753: rename d0/d1a/c76 to d0/d34/ce6 0 2024-09-21T04:15:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:4/821: write d4/d66/d7f/d2a/f36 [1992988,22304] 0 2024-09-21T04:15:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/854: dwrite d0/d9/dc/d13/d50/dc0/fde [0,4194304] 0 2024-09-21T04:15:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/855: dread - d0/d2b/da3/d82/fef zero size 2024-09-21T04:15:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/856: write d0/d9/dc/d13/f69 [103554,105984] 0 2024-09-21T04:15:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/857: fsync d0/ff 0 2024-09-21T04:15:23.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/858: write d0/d9/da2/ff2 [930573,104874] 0 2024-09-21T04:15:23.171 INFO:tasks.workunit.client.0.smithi110.stdout:8/716: dread d0/d7/d13/f5a [0,4194304] 0 2024-09-21T04:15:23.171 INFO:tasks.workunit.client.0.smithi110.stdout:8/717: readlink d0/d7/d13/d46/l80 0 2024-09-21T04:15:23.171 INFO:tasks.workunit.client.0.smithi110.stdout:4/840: dwrite d17/fe9 [0,4194304] 0 2024-09-21T04:15:23.171 INFO:tasks.workunit.client.1.smithi148.stdout:7/745: rename d5/d14/d25/d40/dab/dad/db8/fbc to d5/d8/d6f/dcd/f106 0 2024-09-21T04:15:23.172 INFO:tasks.workunit.client.1.smithi148.stdout:6/754: creat d0/d1a/d23/d27/d80/da4/fe7 x:0 0 0 2024-09-21T04:15:23.175 INFO:tasks.workunit.client.1.smithi148.stdout:4/822: mknod d4/d12/d78/c100 0 2024-09-21T04:15:23.177 INFO:tasks.workunit.client.1.smithi148.stdout:1/748: dwrite db/dd/fae [0,4194304] 0 2024-09-21T04:15:23.179 INFO:tasks.workunit.client.1.smithi148.stdout:4/823: mkdir d4/d12/d83/dcc/d101 0 2024-09-21T04:15:23.179 INFO:tasks.workunit.client.1.smithi148.stdout:4/824: stat d4/d66/d7f/d9a/dc2/ldd 0 2024-09-21T04:15:23.180 INFO:tasks.workunit.client.1.smithi148.stdout:1/749: mknod db/dd/d21/d9b/cf1 0 2024-09-21T04:15:23.182 INFO:tasks.workunit.client.1.smithi148.stdout:8/689: dwrite dc/d43/d4c/d4f/dc5/fcf [0,4194304] 0 2024-09-21T04:15:23.182 INFO:tasks.workunit.client.1.smithi148.stdout:8/690: write dc/d13/d3c/d81/d85/f87 [4210403,1728] 0 2024-09-21T04:15:23.183 INFO:tasks.workunit.client.1.smithi148.stdout:6/755: link d0/d8/f10 d0/d34/d38/d43/fe8 0 2024-09-21T04:15:23.183 INFO:tasks.workunit.client.1.smithi148.stdout:2/859: dread d0/d9/d16/d55/d71/dc2/dcb/fd8 [0,4194304] 0 2024-09-21T04:15:23.183 INFO:tasks.workunit.client.1.smithi148.stdout:9/774: dwrite d5/d1f/d81/f10c [0,4194304] 0 2024-09-21T04:15:23.184 INFO:tasks.workunit.client.0.smithi110.stdout:2/539: dwrite d2/d1a/d1f/d5f/f95 [0,4194304] 0 2024-09-21T04:15:23.184 INFO:tasks.workunit.client.0.smithi110.stdout:2/540: chown d2/d1a/d8b 504505176 1 2024-09-21T04:15:23.184 INFO:tasks.workunit.client.0.smithi110.stdout:2/541: fdatasync d2/d1a/d1f/d2a/f61 0 2024-09-21T04:15:23.184 INFO:tasks.workunit.client.0.smithi110.stdout:8/718: dread d0/d30/d9d/fd5 [0,4194304] 0 2024-09-21T04:15:23.185 INFO:tasks.workunit.client.1.smithi148.stdout:4/825: creat d4/d12/d78/f102 x:0 0 0 2024-09-21T04:15:23.186 INFO:tasks.workunit.client.0.smithi110.stdout:4/841: symlink d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d114/l121 0 2024-09-21T04:15:23.189 INFO:tasks.workunit.client.0.smithi110.stdout:8/719: creat d0/d30/d9d/dad/f104 x:0 0 0 2024-09-21T04:15:23.192 INFO:tasks.workunit.client.0.smithi110.stdout:8/720: mkdir d0/d30/d105 0 2024-09-21T04:15:23.192 INFO:tasks.workunit.client.0.smithi110.stdout:8/721: chown d0/d7/d23/d82/caa 142348 1 2024-09-21T04:15:23.193 INFO:tasks.workunit.client.0.smithi110.stdout:8/722: truncate d0/d7/d13/d46/df8/faf 1088717 0 2024-09-21T04:15:23.193 INFO:tasks.workunit.client.1.smithi148.stdout:1/750: creat db/d24/d2e/d59/da7/dc1/ff2 x:0 0 0 2024-09-21T04:15:23.194 INFO:tasks.workunit.client.0.smithi110.stdout:8/723: dread d0/d7/d13/d4b/ddc/f95 [0,4194304] 0 2024-09-21T04:15:23.194 INFO:tasks.workunit.client.0.smithi110.stdout:8/724: creat d0/d7/d13/d46/f106 x:0 0 0 2024-09-21T04:15:23.194 INFO:tasks.workunit.client.0.smithi110.stdout:8/725: chown d0/d30/d34/d3e/d60 376 1 2024-09-21T04:15:23.194 INFO:tasks.workunit.client.1.smithi148.stdout:7/746: dread d5/d8/f1e [4194304,4194304] 0 2024-09-21T04:15:23.200 INFO:tasks.workunit.client.0.smithi110.stdout:8/726: link d0/d7/d13/d1c/f50 d0/d7/f107 0 2024-09-21T04:15:23.201 INFO:tasks.workunit.client.1.smithi148.stdout:8/691: creat dc/d13/d76/d42/fd6 x:0 0 0 2024-09-21T04:15:23.201 INFO:tasks.workunit.client.1.smithi148.stdout:8/692: write dc/d13/d76/d42/f90 [425422,116013] 0 2024-09-21T04:15:23.201 INFO:tasks.workunit.client.1.smithi148.stdout:8/693: stat dc/d13 0 2024-09-21T04:15:23.201 INFO:tasks.workunit.client.1.smithi148.stdout:8/694: chown dc/d43/f44 2094913858 1 2024-09-21T04:15:23.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/695: creat dc/d13/d39/d6e/fd7 x:0 0 0 2024-09-21T04:15:23.203 INFO:tasks.workunit.client.1.smithi148.stdout:6/756: rename d0/d1a/d23/d32/fad to d0/d8/d18/fe9 0 2024-09-21T04:15:23.203 INFO:tasks.workunit.client.0.smithi110.stdout:8/727: rmdir d0/d7/d23/d82 39 2024-09-21T04:15:23.206 INFO:tasks.workunit.client.0.smithi110.stdout:8/728: mkdir d0/d7/d13/d4b/d108 0 2024-09-21T04:15:23.206 INFO:tasks.workunit.client.0.smithi110.stdout:8/729: dread - d0/d30/d34/d3e/d5d/dd7/ff6 zero size 2024-09-21T04:15:23.206 INFO:tasks.workunit.client.1.smithi148.stdout:6/757: dread d0/d34/d38/d43/f98 [0,4194304] 0 2024-09-21T04:15:23.206 INFO:tasks.workunit.client.1.smithi148.stdout:6/758: fsync d0/d1a/d23/d27/d80/da4/fd9 0 2024-09-21T04:15:23.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/860: creat d0/d9/dc/d13/d25/f10e x:0 0 0 2024-09-21T04:15:23.207 INFO:tasks.workunit.client.0.smithi110.stdout:1/964: sync 2024-09-21T04:15:23.207 INFO:tasks.workunit.client.0.smithi110.stdout:6/696: sync 2024-09-21T04:15:23.208 INFO:tasks.workunit.client.0.smithi110.stdout:5/649: sync 2024-09-21T04:15:23.208 INFO:tasks.workunit.client.0.smithi110.stdout:7/665: sync 2024-09-21T04:15:23.208 INFO:tasks.workunit.client.0.smithi110.stdout:9/592: sync 2024-09-21T04:15:23.208 INFO:tasks.workunit.client.0.smithi110.stdout:9/593: dread - d7/d96/d99/d3c/d35/fa7 zero size 2024-09-21T04:15:23.209 INFO:tasks.workunit.client.1.smithi148.stdout:9/775: mknod d5/d60/c113 0 2024-09-21T04:15:23.210 INFO:tasks.workunit.client.0.smithi110.stdout:8/730: mknod d0/d30/d9d/dad/c109 0 2024-09-21T04:15:23.212 INFO:tasks.workunit.client.1.smithi148.stdout:1/751: creat db/d24/d2e/d71/d94/ff3 x:0 0 0 2024-09-21T04:15:23.213 INFO:tasks.workunit.client.0.smithi110.stdout:6/697: creat d4/d12/d16/d66/d7a/d83/fe0 x:0 0 0 2024-09-21T04:15:23.214 INFO:tasks.workunit.client.0.smithi110.stdout:5/650: dread d1/d7/f13 [4194304,4194304] 0 2024-09-21T04:15:23.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/651: write d1/d39/fa4 [29436,95405] 0 2024-09-21T04:15:23.216 INFO:tasks.workunit.client.0.smithi110.stdout:9/594: link d7/d96/d99/f2a d7/d96/d99/d3c/d11/fc6 0 2024-09-21T04:15:23.216 INFO:tasks.workunit.client.0.smithi110.stdout:9/595: creat d7/d96/d99/d3c/d14/fc7 x:0 0 0 2024-09-21T04:15:23.216 INFO:tasks.workunit.client.1.smithi148.stdout:7/747: mknod d5/d14/d25/d40/dab/dad/c107 0 2024-09-21T04:15:23.217 INFO:tasks.workunit.client.1.smithi148.stdout:7/748: chown d5/d8/d6f/d7d/d90/d28/c2c 15004 1 2024-09-21T04:15:23.217 INFO:tasks.workunit.client.1.smithi148.stdout:7/749: dread - d5/d8/d6f/d7d/d8a/fa1 zero size 2024-09-21T04:15:23.219 INFO:tasks.workunit.client.0.smithi110.stdout:6/698: symlink d4/d12/d17/d9a/le1 0 2024-09-21T04:15:23.220 INFO:tasks.workunit.client.1.smithi148.stdout:1/752: mkdir db/d24/df4 0 2024-09-21T04:15:23.222 INFO:tasks.workunit.client.0.smithi110.stdout:5/652: symlink d1/d49/d4d/d1e/d4a/dae/ld9 0 2024-09-21T04:15:23.222 INFO:tasks.workunit.client.1.smithi148.stdout:8/696: dread f7 [0,4194304] 0 2024-09-21T04:15:23.223 INFO:tasks.workunit.client.1.smithi148.stdout:8/697: dread dc/d13/f34 [0,4194304] 0 2024-09-21T04:15:23.224 INFO:tasks.workunit.client.0.smithi110.stdout:9/596: getdents d7 0 2024-09-21T04:15:23.224 INFO:tasks.workunit.client.1.smithi148.stdout:1/753: mkdir db/d24/d2a/dbd/df5 0 2024-09-21T04:15:23.224 INFO:tasks.workunit.client.1.smithi148.stdout:1/754: readlink db/d24/d2e/d59/l5d 0 2024-09-21T04:15:23.226 INFO:tasks.workunit.client.0.smithi110.stdout:5/653: mkdir d1/d49/d4d/d1e/d4a/dd4/dda 0 2024-09-21T04:15:23.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/597: creat d7/d10/d88/d8f/da0/fc8 x:0 0 0 2024-09-21T04:15:23.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/598: truncate d7/d96/d99/d3c/d4f/d9f/d6f/d78/fb9 786501 0 2024-09-21T04:15:23.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/599: fdatasync d7/d96/d99/d3c/d1f/d43/f6e 0 2024-09-21T04:15:23.227 INFO:tasks.workunit.client.1.smithi148.stdout:2/861: dread d0/d9/d16/d55/d6f/f3a [4194304,4194304] 0 2024-09-21T04:15:23.230 INFO:tasks.workunit.client.1.smithi148.stdout:2/862: symlink d0/d2b/da3/da1/dcd/l10f 0 2024-09-21T04:15:23.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/863: dread d0/d9/d16/d9e/f107 [0,4194304] 0 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/864: rename d0/d9/d16/d55/d6f/f67 to d0/d2b/da3/d82/f110 0 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/865: dread - d0/d2b/da3/d8f/d9f/ff0 zero size 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/866: write d0/d9/d16/d55/d6f/f33 [688552,113828] 0 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/867: readlink d0/d2b/l51 0 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/868: creat d0/d2b/da3/d82/f111 x:0 0 0 2024-09-21T04:15:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/869: creat d0/d9/dc/f112 x:0 0 0 2024-09-21T04:15:23.239 INFO:tasks.workunit.client.0.smithi110.stdout:6/699: dread d4/d78/daa/f1f [0,4194304] 0 2024-09-21T04:15:23.240 INFO:tasks.workunit.client.0.smithi110.stdout:9/600: dread d7/d96/d99/d3c/d4f/f64 [0,4194304] 0 2024-09-21T04:15:23.240 INFO:tasks.workunit.client.1.smithi148.stdout:2/870: mknod d0/d2b/da3/d8f/d9f/dc9/c113 0 2024-09-21T04:15:23.240 INFO:tasks.workunit.client.1.smithi148.stdout:6/759: dread d0/d1a/d23/d32/d93/fca [0,4194304] 0 2024-09-21T04:15:23.242 INFO:tasks.workunit.client.1.smithi148.stdout:4/826: dwrite d4/fe [0,4194304] 0 2024-09-21T04:15:23.242 INFO:tasks.workunit.client.1.smithi148.stdout:6/760: truncate d0/d34/d97/d4f/fa7 2806506 0 2024-09-21T04:15:23.243 INFO:tasks.workunit.client.0.smithi110.stdout:6/700: creat d4/d12/d17/d9a/dba/fe2 x:0 0 0 2024-09-21T04:15:23.245 INFO:tasks.workunit.client.0.smithi110.stdout:9/601: mknod d7/d10/cc9 0 2024-09-21T04:15:23.246 INFO:tasks.workunit.client.0.smithi110.stdout:3/585: dwrite d0/f88 [0,4194304] 0 2024-09-21T04:15:23.248 INFO:tasks.workunit.client.0.smithi110.stdout:6/701: rename d4/l88 to d4/d12/d17/d81/dc5/le3 0 2024-09-21T04:15:23.248 INFO:tasks.workunit.client.0.smithi110.stdout:6/702: write d4/d12/d17/d33/d76/fa6 [198125,79604] 0 2024-09-21T04:15:23.248 INFO:tasks.workunit.client.1.smithi148.stdout:9/776: dwrite d5/f8a [0,4194304] 0 2024-09-21T04:15:23.249 INFO:tasks.workunit.client.0.smithi110.stdout:4/842: dwrite d17/d1e/d77/d10c/d20/d43/ff4 [0,4194304] 0 2024-09-21T04:15:23.249 INFO:tasks.workunit.client.0.smithi110.stdout:4/843: readlink d17/d1e/d77/d10c/d20/d43/l63 0 2024-09-21T04:15:23.250 INFO:tasks.workunit.client.0.smithi110.stdout:9/602: mkdir d7/d96/d99/d3c/d1f/d43/db5/dca 0 2024-09-21T04:15:23.255 INFO:tasks.workunit.client.1.smithi148.stdout:4/827: mkdir d4/d12/d83/d103 0 2024-09-21T04:15:23.256 INFO:tasks.workunit.client.0.smithi110.stdout:3/586: write d0/d12/d16/d1a/d5c/f60 [514211,114448] 0 2024-09-21T04:15:23.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/761: creat d0/d1a/db8/fea x:0 0 0 2024-09-21T04:15:23.258 INFO:tasks.workunit.client.1.smithi148.stdout:6/762: stat d0/d34/d38/d57 0 2024-09-21T04:15:23.259 INFO:tasks.workunit.client.0.smithi110.stdout:6/703: mkdir d4/d12/d17/d9a/de4 0 2024-09-21T04:15:23.259 INFO:tasks.workunit.client.0.smithi110.stdout:6/704: chown d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99 6 1 2024-09-21T04:15:23.263 INFO:tasks.workunit.client.0.smithi110.stdout:4/844: read d17/d1e/d77/d10c/d57/db9/fc8 [993888,129929] 0 2024-09-21T04:15:23.263 INFO:tasks.workunit.client.1.smithi148.stdout:4/828: unlink d4/d66/d7f/lc5 0 2024-09-21T04:15:23.265 INFO:tasks.workunit.client.0.smithi110.stdout:3/587: dread d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:23.265 INFO:tasks.workunit.client.0.smithi110.stdout:9/603: rename d7/cb4 to d7/d96/d99/ccb 0 2024-09-21T04:15:23.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/829: rename d4/d12/d75/d9c/ff6 to d4/d87/d91/f104 0 2024-09-21T04:15:23.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/830: readlink d4/d12/d16/d29/d61/d6c/l84 0 2024-09-21T04:15:23.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/831: truncate d4/d12/d16/d29/f46 4691734 0 2024-09-21T04:15:23.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/832: creat d4/d66/d7f/d9a/dce/f105 x:0 0 0 2024-09-21T04:15:23.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/833: fdatasync d4/d87/fba 0 2024-09-21T04:15:23.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/845: symlink d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d114/l122 0 2024-09-21T04:15:23.268 INFO:tasks.workunit.client.0.smithi110.stdout:3/588: symlink d0/d12/d16/d1a/d95/lcd 0 2024-09-21T04:15:23.269 INFO:tasks.workunit.client.0.smithi110.stdout:9/604: link d7/d96/d99/cc4 d7/d10/d88/d8f/ccc 0 2024-09-21T04:15:23.271 INFO:tasks.workunit.client.1.smithi148.stdout:4/834: symlink d4/d66/d7f/d9a/dc2/l106 0 2024-09-21T04:15:23.273 INFO:tasks.workunit.client.0.smithi110.stdout:4/846: unlink d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d114/l122 0 2024-09-21T04:15:23.273 INFO:tasks.workunit.client.0.smithi110.stdout:4/847: write d17/d1e/d77/d10c/d57/fee [590866,57994] 0 2024-09-21T04:15:23.274 INFO:tasks.workunit.client.0.smithi110.stdout:3/589: mkdir d0/d12/d3c/dce 0 2024-09-21T04:15:23.275 INFO:tasks.workunit.client.0.smithi110.stdout:3/590: creat d0/d12/d3c/d5a/d70/fcf x:0 0 0 2024-09-21T04:15:23.276 INFO:tasks.workunit.client.0.smithi110.stdout:3/591: getdents d0/d12/d3c/dad 0 2024-09-21T04:15:23.277 INFO:tasks.workunit.client.0.smithi110.stdout:3/592: symlink d0/d12/d2a/d80/d9c/ld0 0 2024-09-21T04:15:23.277 INFO:tasks.workunit.client.1.smithi148.stdout:4/835: dread d4/d12/d16/f4f [0,4194304] 0 2024-09-21T04:15:23.277 INFO:tasks.workunit.client.1.smithi148.stdout:4/836: truncate d4/d12/d16/d29/f46 5263123 0 2024-09-21T04:15:23.277 INFO:tasks.workunit.client.0.smithi110.stdout:3/593: mknod d0/d12/d3c/dce/cd1 0 2024-09-21T04:15:23.279 INFO:tasks.workunit.client.1.smithi148.stdout:4/837: truncate d4/d12/d16/d29/f5a 7602318 0 2024-09-21T04:15:23.279 INFO:tasks.workunit.client.0.smithi110.stdout:3/594: mkdir d0/d22/dd2 0 2024-09-21T04:15:23.280 INFO:tasks.workunit.client.0.smithi110.stdout:3/595: stat d0/d12/d3c/d5a/d70/l8a 0 2024-09-21T04:15:23.280 INFO:tasks.workunit.client.0.smithi110.stdout:3/596: write d0/d12/d16/f6a [866236,104127] 0 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/838: mkdir d4/d87/d2d/dd0/d107 0 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/839: dread - d4/d87/d2d/fc3 zero size 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/840: fdatasync d4/d3a/f53 0 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/841: readlink d4/d66/d7f/d2a/da9/lfc 0 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/842: write d4/d12/d75/db4/fdf [81530,13826] 0 2024-09-21T04:15:23.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/843: creat d4/d12/d75/d85/f108 x:0 0 0 2024-09-21T04:15:23.283 INFO:tasks.workunit.client.1.smithi148.stdout:4/844: rename d4/d66/d94/ld7 to d4/d12/d16/d29/d61/d6c/d9d/l109 0 2024-09-21T04:15:23.293 INFO:tasks.workunit.client.0.smithi110.stdout:6/705: dread d4/f8c [0,4194304] 0 2024-09-21T04:15:23.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/745: sync 2024-09-21T04:15:23.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/761: sync 2024-09-21T04:15:23.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/746: truncate d5/d6/d2a/d2b/d99/deb/da5/fbf 1128482 0 2024-09-21T04:15:23.299 INFO:tasks.workunit.client.1.smithi148.stdout:9/777: dwrite d5/d1f/fe6 [0,4194304] 0 2024-09-21T04:15:23.300 INFO:tasks.workunit.client.1.smithi148.stdout:9/778: read d5/d1f/f4b [12927,122638] 0 2024-09-21T04:15:23.300 INFO:tasks.workunit.client.1.smithi148.stdout:0/747: truncate d5/d6/d2a/d3f/d6d/faa 2398233 0 2024-09-21T04:15:23.300 INFO:tasks.workunit.client.1.smithi148.stdout:0/748: truncate d5/d6/d7/d18/fc3 567689 0 2024-09-21T04:15:23.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/762: getdents d3/d34/d78 0 2024-09-21T04:15:23.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/763: dread - d3/d34/ff7 zero size 2024-09-21T04:15:23.303 INFO:tasks.workunit.client.1.smithi148.stdout:9/779: link d5/d2b/dbe/cfd d5/d1e/d37/c114 0 2024-09-21T04:15:23.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/764: symlink d3/d34/d78/dbf/d6f/l104 0 2024-09-21T04:15:23.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/845: write d4/d12/d16/d29/d61/f4b [3262425,49148] 0 2024-09-21T04:15:23.305 INFO:tasks.workunit.client.1.smithi148.stdout:9/780: getdents d5 0 2024-09-21T04:15:23.305 INFO:tasks.workunit.client.1.smithi148.stdout:9/781: creat d5/d60/d4c/f115 x:0 0 0 2024-09-21T04:15:23.306 INFO:tasks.workunit.client.1.smithi148.stdout:9/782: stat d5/d60/d28/d2f/d10a 0 2024-09-21T04:15:23.306 INFO:tasks.workunit.client.0.smithi110.stdout:6/706: dwrite f3 [0,4194304] 0 2024-09-21T04:15:23.308 INFO:tasks.workunit.client.1.smithi148.stdout:9/783: symlink d5/d1f/d81/dcf/d106/d80/d104/l116 0 2024-09-21T04:15:23.310 INFO:tasks.workunit.client.0.smithi110.stdout:6/707: link d4/d3c/c4c d4/d12/d17/d9a/de4/ce5 0 2024-09-21T04:15:23.310 INFO:tasks.workunit.client.0.smithi110.stdout:6/708: write d4/d12/d16/d66/d7a/d83/fcf [838119,92260] 0 2024-09-21T04:15:23.310 INFO:tasks.workunit.client.0.smithi110.stdout:6/709: readlink d4/d12/l2c 0 2024-09-21T04:15:23.310 INFO:tasks.workunit.client.1.smithi148.stdout:8/698: dwrite dc/d43/f47 [0,4194304] 0 2024-09-21T04:15:23.310 INFO:tasks.workunit.client.1.smithi148.stdout:4/846: rmdir d4/d3a 39 2024-09-21T04:15:23.311 INFO:tasks.workunit.client.1.smithi148.stdout:4/847: truncate d4/d12/d78/f102 893329 0 2024-09-21T04:15:23.311 INFO:tasks.workunit.client.0.smithi110.stdout:8/731: dwrite d0/d30/d34/d3e/d32/ff5 [0,4194304] 0 2024-09-21T04:15:23.311 INFO:tasks.workunit.client.0.smithi110.stdout:1/965: dwrite d2/d9/d49/f77 [0,4194304] 0 2024-09-21T04:15:23.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/710: mkdir d4/d12/d17/d9a/dba/de6 0 2024-09-21T04:15:23.312 INFO:tasks.workunit.client.1.smithi148.stdout:1/755: dwrite db/dd/f13 [0,4194304] 0 2024-09-21T04:15:23.312 INFO:tasks.workunit.client.1.smithi148.stdout:9/784: creat d5/f117 x:0 0 0 2024-09-21T04:15:23.312 INFO:tasks.workunit.client.1.smithi148.stdout:9/785: chown d5/d60/d4c 73710 1 2024-09-21T04:15:23.313 INFO:tasks.workunit.client.0.smithi110.stdout:7/666: dwrite dd/d14/d17/f85 [0,4194304] 0 2024-09-21T04:15:23.313 INFO:tasks.workunit.client.0.smithi110.stdout:9/605: dwrite d7/d96/d99/d3c/d1f/f23 [0,4194304] 0 2024-09-21T04:15:23.313 INFO:tasks.workunit.client.1.smithi148.stdout:2/871: dwrite d0/ff [0,4194304] 0 2024-09-21T04:15:23.313 INFO:tasks.workunit.client.1.smithi148.stdout:2/872: creat d0/d2b/da3/d8f/d9f/f114 x:0 0 0 2024-09-21T04:15:23.313 INFO:tasks.workunit.client.1.smithi148.stdout:5/765: dread f0 [0,4194304] 0 2024-09-21T04:15:23.314 INFO:tasks.workunit.client.1.smithi148.stdout:5/766: write d3/d34/d3c/d5d/d75/fa9 [1010466,67535] 0 2024-09-21T04:15:23.314 INFO:tasks.workunit.client.1.smithi148.stdout:5/767: readlink d3/d34/d78/l91 0 2024-09-21T04:15:23.314 INFO:tasks.workunit.client.0.smithi110.stdout:5/654: dwrite d1/d49/fab [0,4194304] 0 2024-09-21T04:15:23.315 INFO:tasks.workunit.client.1.smithi148.stdout:6/763: dwrite d0/d1a/d23/d27/d62/d9a/d92/fa2 [0,4194304] 0 2024-09-21T04:15:23.315 INFO:tasks.workunit.client.1.smithi148.stdout:6/764: write d0/d1a/d23/d32/d93/f9c [1609535,19436] 0 2024-09-21T04:15:23.316 INFO:tasks.workunit.client.0.smithi110.stdout:1/966: mkdir d2/d9/d49/d4f/d87/d126/d139 0 2024-09-21T04:15:23.316 INFO:tasks.workunit.client.0.smithi110.stdout:1/967: read d2/d9/d49/d5c/d91/d101/d103/f11f [4059482,115375] 0 2024-09-21T04:15:23.316 INFO:tasks.workunit.client.0.smithi110.stdout:1/968: fsync d2/d9/d26/d3d/f11a 0 2024-09-21T04:15:23.317 INFO:tasks.workunit.client.1.smithi148.stdout:8/699: mkdir dc/d13/d76/dd8 0 2024-09-21T04:15:23.318 INFO:tasks.workunit.client.0.smithi110.stdout:6/711: rename d4/d12/d17/d33/d76/ldb to d4/d12/d16/d66/d7a/d54/le7 0 2024-09-21T04:15:23.319 INFO:tasks.workunit.client.1.smithi148.stdout:7/750: getdents d5/d14/d25/d40/dab/dad 0 2024-09-21T04:15:23.320 INFO:tasks.workunit.client.0.smithi110.stdout:6/712: write d4/f6 [872089,96268] 0 2024-09-21T04:15:23.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/756: mknod db/d24/d2a/da4/cf6 0 2024-09-21T04:15:23.322 INFO:tasks.workunit.client.1.smithi148.stdout:0/749: dwrite d5/d6/f16 [0,4194304] 0 2024-09-21T04:15:23.322 INFO:tasks.workunit.client.0.smithi110.stdout:7/667: symlink dd/d1c/d89/ld8 0 2024-09-21T04:15:23.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/848: dread d4/d12/d75/d85/fbb [0,4194304] 0 2024-09-21T04:15:23.324 INFO:tasks.workunit.client.1.smithi148.stdout:9/786: truncate d5/d1f/d81/d9e/da2/fc5 301701 0 2024-09-21T04:15:23.324 INFO:tasks.workunit.client.1.smithi148.stdout:9/787: truncate d5/d1f/f4b 223429 0 2024-09-21T04:15:23.324 INFO:tasks.workunit.client.1.smithi148.stdout:9/788: fdatasync d5/fcc 0 2024-09-21T04:15:23.324 INFO:tasks.workunit.client.1.smithi148.stdout:9/789: readlink d5/d2b/d4e/le0 0 2024-09-21T04:15:23.325 INFO:tasks.workunit.client.1.smithi148.stdout:9/790: write d5/d2b/d4e/db3/fcb [265569,72434] 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:6/765: dread d0/d8/d18/f4d [0,4194304] 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:6/766: fsync d0/d1a/d23/d27/d62/f7f 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:6/767: truncate d0/d34/d38/d43/f5e 8651171 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:2/873: mknod d0/d2b/da3/d8f/d9f/df5/c115 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:2/874: fsync d0/f1d 0 2024-09-21T04:15:23.329 INFO:tasks.workunit.client.1.smithi148.stdout:2/875: creat d0/d9/d16/d9e/f116 x:0 0 0 2024-09-21T04:15:23.330 INFO:tasks.workunit.client.0.smithi110.stdout:8/732: dread d0/d30/d34/d3e/d32/ff5 [4194304,4194304] 0 2024-09-21T04:15:23.330 INFO:tasks.workunit.client.0.smithi110.stdout:6/713: dread d4/f6 [0,4194304] 0 2024-09-21T04:15:23.332 INFO:tasks.workunit.client.0.smithi110.stdout:5/655: mknod d1/d49/d4d/d1e/d4a/dd4/cdb 0 2024-09-21T04:15:23.333 INFO:tasks.workunit.client.1.smithi148.stdout:5/768: dwrite d3/d14/d1a/f4d [0,4194304] 0 2024-09-21T04:15:23.333 INFO:tasks.workunit.client.1.smithi148.stdout:5/769: fdatasync d3/d14/d1f/f7e 0 2024-09-21T04:15:23.335 INFO:tasks.workunit.client.0.smithi110.stdout:9/606: rename d7/d10/d88/d8f/fb8 to d7/d10/d88/d8f/db7/fcd 0 2024-09-21T04:15:23.337 INFO:tasks.workunit.client.1.smithi148.stdout:8/700: rename dc/d13/d39/d6e/dac/cb3 to dc/d43/cd9 0 2024-09-21T04:15:23.338 INFO:tasks.workunit.client.0.smithi110.stdout:7/668: stat dd/d1c/l51 0 2024-09-21T04:15:23.338 INFO:tasks.workunit.client.0.smithi110.stdout:7/669: chown dd/d14/d17/f3a 4400817 1 2024-09-21T04:15:23.338 INFO:tasks.workunit.client.0.smithi110.stdout:7/670: write dd/d14/d17/d8a/f8b [586503,62522] 0 2024-09-21T04:15:23.343 INFO:tasks.workunit.client.0.smithi110.stdout:8/733: link d0/d30/d34/d3e/d5d/dd7/ced d0/d30/d54/c10a 0 2024-09-21T04:15:23.345 INFO:tasks.workunit.client.1.smithi148.stdout:1/757: dwrite db/dd/d21/d89/fd1 [0,4194304] 0 2024-09-21T04:15:23.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/750: mknod d5/d82/ced 0 2024-09-21T04:15:23.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/751: fdatasync d5/d6/d2a/d2b/d99/deb/da5/f86 0 2024-09-21T04:15:23.347 INFO:tasks.workunit.client.0.smithi110.stdout:5/656: write d1/d39/f21 [2519575,94447] 0 2024-09-21T04:15:23.349 INFO:tasks.workunit.client.1.smithi148.stdout:4/849: creat d4/d66/d7f/d2a/da9/f10a x:0 0 0 2024-09-21T04:15:23.350 INFO:tasks.workunit.client.0.smithi110.stdout:6/714: rmdir d4/d12/d17/d81/dc5 39 2024-09-21T04:15:23.355 INFO:tasks.workunit.client.1.smithi148.stdout:6/768: rmdir d0/d1a/d23/dc/d68/d5b 39 2024-09-21T04:15:23.358 INFO:tasks.workunit.client.0.smithi110.stdout:1/969: rename d2/db7/d86/f95 to d2/d9/d49/d4f/d53/f13a 0 2024-09-21T04:15:23.360 INFO:tasks.workunit.client.1.smithi148.stdout:2/876: truncate d0/d2b/da3/da1/f2e 742080 0 2024-09-21T04:15:23.360 INFO:tasks.workunit.client.0.smithi110.stdout:9/607: symlink d7/d96/d99/d3c/dae/lce 0 2024-09-21T04:15:23.362 INFO:tasks.workunit.client.1.smithi148.stdout:5/770: mkdir d3/d14/d1f/d26/d2e/ddb/deb/d105 0 2024-09-21T04:15:23.362 INFO:tasks.workunit.client.1.smithi148.stdout:5/771: read d3/d14/d1f/d26/d2e/d56/da7/fd9 [1068723,690] 0 2024-09-21T04:15:23.362 INFO:tasks.workunit.client.0.smithi110.stdout:7/671: symlink dd/d14/d17/dd1/ld9 0 2024-09-21T04:15:23.367 INFO:tasks.workunit.client.0.smithi110.stdout:1/970: dread d2/db7/f105 [0,4194304] 0 2024-09-21T04:15:23.367 INFO:tasks.workunit.client.0.smithi110.stdout:6/715: creat d4/d12/d16/d66/fe8 x:0 0 0 2024-09-21T04:15:23.368 INFO:tasks.workunit.client.0.smithi110.stdout:6/716: dread d4/d78/daa/f84 [0,4194304] 0 2024-09-21T04:15:23.370 INFO:tasks.workunit.client.0.smithi110.stdout:7/672: rename dd/d1c/dad/db2/d56/d66/da6/d7a/l96 to dd/d14/lda 0 2024-09-21T04:15:23.372 INFO:tasks.workunit.client.0.smithi110.stdout:1/971: mkdir d2/d9/d49/d4f/d13b 0 2024-09-21T04:15:23.373 INFO:tasks.workunit.client.0.smithi110.stdout:6/717: creat d4/d78/fe9 x:0 0 0 2024-09-21T04:15:23.374 INFO:tasks.workunit.client.0.smithi110.stdout:6/718: chown d4/f8c 0 1 2024-09-21T04:15:23.375 INFO:tasks.workunit.client.0.smithi110.stdout:4/848: write d17/d1e/d77/d10c/d20/d43/d51/d7a/d86/fc9 [1050658,102854] 0 2024-09-21T04:15:23.375 INFO:tasks.workunit.client.0.smithi110.stdout:4/849: fdatasync d17/d1e/d77/d10c/d20/d43/f8c 0 2024-09-21T04:15:23.375 INFO:tasks.workunit.client.0.smithi110.stdout:7/673: truncate dd/d14/f41 395555 0 2024-09-21T04:15:23.376 INFO:tasks.workunit.client.0.smithi110.stdout:9/608: dread d7/d96/d99/d3c/d11/f81 [0,4194304] 0 2024-09-21T04:15:23.377 INFO:tasks.workunit.client.0.smithi110.stdout:1/972: rename d2/d9/d122/f12e to d2/d9/d23/d3b/dca/d107/d124/f13c 0 2024-09-21T04:15:23.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/751: rmdir d5/d8 39 2024-09-21T04:15:23.379 INFO:tasks.workunit.client.0.smithi110.stdout:5/657: dwrite d1/f15 [0,4194304] 0 2024-09-21T04:15:23.380 INFO:tasks.workunit.client.0.smithi110.stdout:8/734: dwrite d0/d7/d13/f37 [0,4194304] 0 2024-09-21T04:15:23.381 INFO:tasks.workunit.client.1.smithi148.stdout:6/769: dwrite d0/d1a/d23/d27/d80/da4/fe7 [0,4194304] 0 2024-09-21T04:15:23.382 INFO:tasks.workunit.client.0.smithi110.stdout:6/719: write d4/d12/d16/d66/d7a/d4b/f51 [4139306,115247] 0 2024-09-21T04:15:23.386 INFO:tasks.workunit.client.0.smithi110.stdout:1/973: rmdir d2/d9/d49/d4f/d87/d126 39 2024-09-21T04:15:23.389 INFO:tasks.workunit.client.0.smithi110.stdout:5/658: mknod d1/d49/d4d/d1e/d4a/dd4/dda/cdc 0 2024-09-21T04:15:23.392 INFO:tasks.workunit.client.1.smithi148.stdout:1/758: creat db/d24/d2a/ff7 x:0 0 0 2024-09-21T04:15:23.394 INFO:tasks.workunit.client.1.smithi148.stdout:0/752: rename d5/d6/d2a/d3f/c74 to d5/d6/d2a/d2b/d99/db7/cee 0 2024-09-21T04:15:23.402 INFO:tasks.workunit.client.0.smithi110.stdout:1/974: truncate d2/d9/d49/d5c/d91/f96 134593 0 2024-09-21T04:15:23.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/659: creat d1/d22/db6/fdd x:0 0 0 2024-09-21T04:15:23.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/660: chown d1/d49/d4d/d1e/f81 12632 1 2024-09-21T04:15:23.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/661: dread - d1/d22/f97 zero size 2024-09-21T04:15:23.404 INFO:tasks.workunit.client.0.smithi110.stdout:5/662: creat d1/d39/da2/fde x:0 0 0 2024-09-21T04:15:23.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/770: dwrite d0/d1a/d23/d27/d62/d9a/d92/fa2 [0,4194304] 0 2024-09-21T04:15:23.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/771: chown d0/d8/f29 3183331 1 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:4/850: rename d17/d1e/d7b/dc5 to d17/d1e/d64/d123 0 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:6/720: dwrite d4/d78/daa/f84 [0,4194304] 0 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:6/721: chown d4/d12/d17/d33/d3e/f53 838118005 1 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:6/722: stat d4/d12/d16/d66/d7a/d83/fcf 0 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:6/723: fdatasync d4/d12/d16/d66/d7a/d4b/f97 0 2024-09-21T04:15:23.407 INFO:tasks.workunit.client.0.smithi110.stdout:6/724: dread - d4/d12/d17/d81/fcc zero size 2024-09-21T04:15:23.408 INFO:tasks.workunit.client.0.smithi110.stdout:6/725: creat d4/d78/fea x:0 0 0 2024-09-21T04:15:23.408 INFO:tasks.workunit.client.1.smithi148.stdout:3/680: sync 2024-09-21T04:15:23.411 INFO:tasks.workunit.client.0.smithi110.stdout:5/663: creat d1/d49/d4d/d1e/d5a/d64/d7b/fdf x:0 0 0 2024-09-21T04:15:23.412 INFO:tasks.workunit.client.0.smithi110.stdout:6/726: dread d4/d12/d16/d66/d7a/d4b/f51 [0,4194304] 0 2024-09-21T04:15:23.413 INFO:tasks.workunit.client.0.smithi110.stdout:8/735: rename d0/d7/d13/f5a to d0/d30/d34/d3e/d5d/dd7/de8/f10b 0 2024-09-21T04:15:23.414 INFO:tasks.workunit.client.1.smithi148.stdout:2/877: mknod d0/d9/dc/d13/daf/c117 0 2024-09-21T04:15:23.416 INFO:tasks.workunit.client.1.smithi148.stdout:7/752: unlink d5/d8/d6f/d7d/d90/d28/f29 0 2024-09-21T04:15:23.421 INFO:tasks.workunit.client.1.smithi148.stdout:9/791: write d5/d2b/dbe/f52 [423137,27104] 0 2024-09-21T04:15:23.421 INFO:tasks.workunit.client.1.smithi148.stdout:9/792: creat d5/d1e/f118 x:0 0 0 2024-09-21T04:15:23.422 INFO:tasks.workunit.client.0.smithi110.stdout:8/736: dread d0/d7/f20 [0,4194304] 0 2024-09-21T04:15:23.423 INFO:tasks.workunit.client.1.smithi148.stdout:1/759: symlink db/d24/d2e/d71/lf8 0 2024-09-21T04:15:23.423 INFO:tasks.workunit.client.1.smithi148.stdout:1/760: truncate db/d24/d2e/d71/f8e 1076782 0 2024-09-21T04:15:23.423 INFO:tasks.workunit.client.1.smithi148.stdout:1/761: chown db/d24/d2e/d71/d94/dde/fea 1862651024 1 2024-09-21T04:15:23.423 INFO:tasks.workunit.client.1.smithi148.stdout:1/762: write db/dd/d21/d89/dc9/fcd [348271,38965] 0 2024-09-21T04:15:23.424 INFO:tasks.workunit.client.0.smithi110.stdout:6/727: unlink d4/d12/d17/d33/f96 0 2024-09-21T04:15:23.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/753: symlink d5/d6/d7/dd0/lef 0 2024-09-21T04:15:23.426 INFO:tasks.workunit.client.1.smithi148.stdout:0/754: truncate d5/d6/d2a/d3f/d6d/da3/fae 574499 0 2024-09-21T04:15:23.426 INFO:tasks.workunit.client.1.smithi148.stdout:0/755: write d5/d6/d7/d22/f8a [1022069,24335] 0 2024-09-21T04:15:23.426 INFO:tasks.workunit.client.0.smithi110.stdout:8/737: dread d0/d30/d34/d3e/d32/ff5 [4194304,4194304] 0 2024-09-21T04:15:23.429 INFO:tasks.workunit.client.1.smithi148.stdout:6/772: rmdir d0/d1a/d23/d32/d93/d96 39 2024-09-21T04:15:23.429 INFO:tasks.workunit.client.1.smithi148.stdout:0/756: write d5/d6/d2a/d2b/f5b [931402,82078] 0 2024-09-21T04:15:23.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/757: write d5/d6/d7/d11/fcf [636074,106462] 0 2024-09-21T04:15:23.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/758: write d5/d6/d2a/d2b/f55 [1169749,105203] 0 2024-09-21T04:15:23.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/763: write db/dd/f13 [2044512,6298] 0 2024-09-21T04:15:23.431 INFO:tasks.workunit.client.1.smithi148.stdout:9/793: dread d5/d2b/f4f [0,4194304] 0 2024-09-21T04:15:23.431 INFO:tasks.workunit.client.0.smithi110.stdout:1/975: rename d2/db7/ded to d2/d9/d49/d7a/dee/de1/d13d 0 2024-09-21T04:15:23.431 INFO:tasks.workunit.client.0.smithi110.stdout:1/976: write d2/d9/d49/d4f/d53/f109 [4032322,49465] 0 2024-09-21T04:15:23.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/681: mknod d3/d1f/d4d/d82/d23/da1/cde 0 2024-09-21T04:15:23.433 INFO:tasks.workunit.client.1.smithi148.stdout:3/682: fdatasync d3/d5/d6/d12/d43/d54/f6a 0 2024-09-21T04:15:23.433 INFO:tasks.workunit.client.1.smithi148.stdout:2/878: dwrite d0/d9/dc/f112 [0,4194304] 0 2024-09-21T04:15:23.434 INFO:tasks.workunit.client.0.smithi110.stdout:5/664: mknod d1/d7/ce0 0 2024-09-21T04:15:23.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/753: mkdir d5/d14/d25/d40/db9/d108 0 2024-09-21T04:15:23.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/754: creat d5/d14/d25/d40/dab/f109 x:0 0 0 2024-09-21T04:15:23.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/755: truncate d5/d8/d6f/f9d 1306442 0 2024-09-21T04:15:23.437 INFO:tasks.workunit.client.0.smithi110.stdout:8/738: mkdir d0/d30/d54/d10c 0 2024-09-21T04:15:23.438 INFO:tasks.workunit.client.0.smithi110.stdout:7/674: rmdir dd/d14 39 2024-09-21T04:15:23.440 INFO:tasks.workunit.client.1.smithi148.stdout:6/773: creat d0/d34/d38/d43/feb x:0 0 0 2024-09-21T04:15:23.440 INFO:tasks.workunit.client.1.smithi148.stdout:6/774: fsync d0/d1a/d23/d53/f9f 0 2024-09-21T04:15:23.441 INFO:tasks.workunit.client.0.smithi110.stdout:4/851: rename d17/d1e/d77/d10c/f4a to d17/d1e/d21/de1/f124 0 2024-09-21T04:15:23.442 INFO:tasks.workunit.client.1.smithi148.stdout:0/759: mkdir d5/d82/df0 0 2024-09-21T04:15:23.444 INFO:tasks.workunit.client.0.smithi110.stdout:1/977: unlink d2/d9/d49/d5c/d91/fde 0 2024-09-21T04:15:23.444 INFO:tasks.workunit.client.0.smithi110.stdout:1/978: write d2/d9/d49/d5c/d91/d101/d103/f11f [3529744,127299] 0 2024-09-21T04:15:23.445 INFO:tasks.workunit.client.0.smithi110.stdout:1/979: dread d2/fd0 [0,4194304] 0 2024-09-21T04:15:23.445 INFO:tasks.workunit.client.1.smithi148.stdout:1/764: symlink db/d46/d76/lf9 0 2024-09-21T04:15:23.446 INFO:tasks.workunit.client.0.smithi110.stdout:5/665: mkdir d1/d22/db6/de1 0 2024-09-21T04:15:23.449 INFO:tasks.workunit.client.1.smithi148.stdout:6/775: dread d0/d34/d97/f39 [0,4194304] 0 2024-09-21T04:15:23.449 INFO:tasks.workunit.client.1.smithi148.stdout:9/794: creat d5/d60/f119 x:0 0 0 2024-09-21T04:15:23.449 INFO:tasks.workunit.client.1.smithi148.stdout:9/795: creat d5/d1e/d37/d50/f11a x:0 0 0 2024-09-21T04:15:23.450 INFO:tasks.workunit.client.0.smithi110.stdout:6/728: unlink d4/d12/d16/d66/fe8 0 2024-09-21T04:15:23.452 INFO:tasks.workunit.client.1.smithi148.stdout:3/683: rename d3/l45 to d3/d84/ldf 0 2024-09-21T04:15:23.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/675: creat dd/d48/d9f/fdb x:0 0 0 2024-09-21T04:15:23.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/676: chown dd/d1c/dad/db2/d56/d66/da6/d7a/f6a 2842 1 2024-09-21T04:15:23.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/677: fdatasync dd/d1c/dad/db2/f3b 0 2024-09-21T04:15:23.457 INFO:tasks.workunit.client.0.smithi110.stdout:4/852: mkdir d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d125 0 2024-09-21T04:15:23.457 INFO:tasks.workunit.client.0.smithi110.stdout:4/853: chown d17/led 2315 1 2024-09-21T04:15:23.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/879: unlink d0/d9/d38/fcc 0 2024-09-21T04:15:23.459 INFO:tasks.workunit.client.0.smithi110.stdout:1/980: symlink d2/d9/d23/d4e/d8e/l13e 0 2024-09-21T04:15:23.459 INFO:tasks.workunit.client.0.smithi110.stdout:1/981: chown d2/d9/d26/f2d 257556 1 2024-09-21T04:15:23.462 INFO:tasks.workunit.client.0.smithi110.stdout:5/666: symlink d1/d7/d58/d9e/le2 0 2024-09-21T04:15:23.463 INFO:tasks.workunit.client.1.smithi148.stdout:0/760: getdents d5/dcb 0 2024-09-21T04:15:23.465 INFO:tasks.workunit.client.0.smithi110.stdout:6/729: rename d4/f8c to d4/d78/feb 0 2024-09-21T04:15:23.465 INFO:tasks.workunit.client.0.smithi110.stdout:6/730: chown d4/d12/d17/d33/d3e 61075329 1 2024-09-21T04:15:23.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/684: dwrite d3/d1f/d4d/d82/d23/daf/fd3 [0,4194304] 0 2024-09-21T04:15:23.467 INFO:tasks.workunit.client.0.smithi110.stdout:6/731: dread d4/d12/d17/d2e/fb9 [0,4194304] 0 2024-09-21T04:15:23.467 INFO:tasks.workunit.client.0.smithi110.stdout:7/678: mkdir dd/d1c/dad/db2/d6d/ddc 0 2024-09-21T04:15:23.467 INFO:tasks.workunit.client.0.smithi110.stdout:7/679: stat dd/d1c/dad/db2/d4a/d52/cb4 0 2024-09-21T04:15:23.468 INFO:tasks.workunit.client.1.smithi148.stdout:6/776: mkdir d0/d1a/d23/dec 0 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.0.smithi110.stdout:4/854: rename d17/d3e/d6b/f7d to d17/d1e/d77/d10c/d20/d43/d51/d52/d98/f126 0 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.0.smithi110.stdout:4/855: readlink d17/d1e/d77/la4 0 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.0.smithi110.stdout:4/856: write d17/d1e/d77/d10c/d20/d43/d51/d7a/fc3 [465033,112225] 0 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.1.smithi148.stdout:3/685: write d3/d5/d6/d12/d43/d54/f6a [1000551,74026] 0 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.1.smithi148.stdout:3/686: chown d3/d1f/d4d/d82/d23/da1/cc7 1 1 2024-09-21T04:15:23.471 INFO:tasks.workunit.client.1.smithi148.stdout:3/687: chown d3/d1f/d4d/d82/d23/l34 91 1 2024-09-21T04:15:23.473 INFO:tasks.workunit.client.0.smithi110.stdout:6/732: symlink d4/d86/lec 0 2024-09-21T04:15:23.476 INFO:tasks.workunit.client.0.smithi110.stdout:7/680: creat dd/d14/d17/dd1/fdd x:0 0 0 2024-09-21T04:15:23.478 INFO:tasks.workunit.client.1.smithi148.stdout:6/777: dread d0/d8/d18/f25 [0,4194304] 0 2024-09-21T04:15:23.479 INFO:tasks.workunit.client.0.smithi110.stdout:5/667: rename d1/d39/f3d to d1/d22/fe3 0 2024-09-21T04:15:23.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/681: mknod dd/d1c/dad/db2/d56/d66/da6/cde 0 2024-09-21T04:15:23.486 INFO:tasks.workunit.client.0.smithi110.stdout:5/668: mkdir d1/d49/de4 0 2024-09-21T04:15:23.486 INFO:tasks.workunit.client.0.smithi110.stdout:5/669: stat d1/d49/fab 0 2024-09-21T04:15:23.486 INFO:tasks.workunit.client.1.smithi148.stdout:0/761: mknod d5/d6/d7/cf1 0 2024-09-21T04:15:23.487 INFO:tasks.workunit.client.0.smithi110.stdout:7/682: link dd/d4d/f71 dd/d1c/dad/db2/d56/d66/da6/d98/fdf 0 2024-09-21T04:15:23.488 INFO:tasks.workunit.client.0.smithi110.stdout:1/982: dwrite d2/d9/d49/fbe [0,4194304] 0 2024-09-21T04:15:23.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/683: dread dd/d1c/dad/db2/d56/d66/da6/f5f [0,4194304] 0 2024-09-21T04:15:23.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/684: truncate dd/d48/f4b 2183025 0 2024-09-21T04:15:23.490 INFO:tasks.workunit.client.0.smithi110.stdout:4/857: dwrite d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/fd8 [0,4194304] 0 2024-09-21T04:15:23.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/685: dread dd/d48/f4b [0,4194304] 0 2024-09-21T04:15:23.494 INFO:tasks.workunit.client.1.smithi148.stdout:6/778: creat d0/d1a/d23/dc/d81/fed x:0 0 0 2024-09-21T04:15:23.495 INFO:tasks.workunit.client.0.smithi110.stdout:1/983: symlink d2/d9/d23/d120/l13f 0 2024-09-21T04:15:23.495 INFO:tasks.workunit.client.0.smithi110.stdout:1/984: write d2/d9/d23/d3b/dca/f110 [1835333,7712] 0 2024-09-21T04:15:23.497 INFO:tasks.workunit.client.1.smithi148.stdout:7/756: rename d5/d14/l83 to d5/d8/l10a 0 2024-09-21T04:15:23.500 INFO:tasks.workunit.client.0.smithi110.stdout:4/858: write d17/d1e/d64/f9d [4174177,103324] 0 2024-09-21T04:15:23.500 INFO:tasks.workunit.client.0.smithi110.stdout:4/859: dread - d17/d1e/d77/d10c/d20/d43/fe2 zero size 2024-09-21T04:15:23.500 INFO:tasks.workunit.client.0.smithi110.stdout:4/860: dread - d17/d1e/d7b/fc4 zero size 2024-09-21T04:15:23.502 INFO:tasks.workunit.client.0.smithi110.stdout:5/670: dwrite d1/d49/d4d/f9c [0,4194304] 0 2024-09-21T04:15:23.502 INFO:tasks.workunit.client.0.smithi110.stdout:5/671: creat d1/d49/d4d/fe5 x:0 0 0 2024-09-21T04:15:23.504 INFO:tasks.workunit.client.1.smithi148.stdout:2/880: rmdir d0/d9/d16/d55 39 2024-09-21T04:15:23.504 INFO:tasks.workunit.client.1.smithi148.stdout:2/881: dread - d0/d2b/da3/d8f/d9f/ff0 zero size 2024-09-21T04:15:23.507 INFO:tasks.workunit.client.0.smithi110.stdout:1/985: creat d2/f140 x:0 0 0 2024-09-21T04:15:23.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/779: write d0/d34/d38/d43/f7b [3512767,64619] 0 2024-09-21T04:15:23.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/780: write d0/d1a/d23/fdf [1001679,1400] 0 2024-09-21T04:15:23.507 INFO:tasks.workunit.client.1.smithi148.stdout:0/762: chown d5/d6/d2a/d2b/c53 2124 1 2024-09-21T04:15:23.509 INFO:tasks.workunit.client.0.smithi110.stdout:7/686: write dd/f57 [222542,91601] 0 2024-09-21T04:15:23.509 INFO:tasks.workunit.client.1.smithi148.stdout:9/796: rename d5/d1e/d37/f3b to d5/d60/d69/de5/f11b 0 2024-09-21T04:15:23.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/861: creat d17/d3e/dcd/f127 x:0 0 0 2024-09-21T04:15:23.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/862: truncate d17/d1e/d21/fd4 656821 0 2024-09-21T04:15:23.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/863: fdatasync d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/fd8 0 2024-09-21T04:15:23.513 INFO:tasks.workunit.client.1.smithi148.stdout:6/781: unlink d0/d34/d97/cb3 0 2024-09-21T04:15:23.513 INFO:tasks.workunit.client.1.smithi148.stdout:6/782: creat d0/d1a/d23/d27/fee x:0 0 0 2024-09-21T04:15:23.513 INFO:tasks.workunit.client.1.smithi148.stdout:6/783: fsync d0/d1a/db8/fea 0 2024-09-21T04:15:23.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/784: chown d0/d34/d97/c1e 3212 1 2024-09-21T04:15:23.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/785: dread - d0/d34/d38/fd6 zero size 2024-09-21T04:15:23.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/786: dread - d0/d1a/d23/d27/fe1 zero size 2024-09-21T04:15:23.514 INFO:tasks.workunit.client.1.smithi148.stdout:6/787: chown d0/d1a 6598955 1 2024-09-21T04:15:23.516 INFO:tasks.workunit.client.1.smithi148.stdout:3/688: rename d3/d5/d6/d12/d15/da7/db6 to d3/d5/d6/d2b/da2/de0 0 2024-09-21T04:15:23.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/797: creat d5/d39/dba/f11c x:0 0 0 2024-09-21T04:15:23.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/798: chown d5/d60/d28/fbb 141068 1 2024-09-21T04:15:23.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/799: creat d5/d39/dba/f11d x:0 0 0 2024-09-21T04:15:23.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/800: fsync d5/d1f/f24 0 2024-09-21T04:15:23.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/801: fdatasync d5/d1e/d37/fc6 0 2024-09-21T04:15:23.520 INFO:tasks.workunit.client.0.smithi110.stdout:5/672: creat d1/d39/fe6 x:0 0 0 2024-09-21T04:15:23.522 INFO:tasks.workunit.client.0.smithi110.stdout:1/986: truncate d2/d9/d23/d4e/d57/f10d 1810040 0 2024-09-21T04:15:23.522 INFO:tasks.workunit.client.0.smithi110.stdout:1/987: chown d2/d9/d49/d7a/dee/de1/l102 686 1 2024-09-21T04:15:23.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/788: dread d0/d1a/d23/ff [0,4194304] 0 2024-09-21T04:15:23.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/789: stat d0/d1a/d23/d27/d62/d9a/fa3 0 2024-09-21T04:15:23.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/763: dwrite d5/d6/d7/f4f [4194304,4194304] 0 2024-09-21T04:15:23.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/882: rename d0/d2b/da3/d8f/fa8 to d0/d9/dc/ddf/f118 0 2024-09-21T04:15:23.531 INFO:tasks.workunit.client.0.smithi110.stdout:4/864: rmdir d17/d1e/d64/d123/d113 0 2024-09-21T04:15:23.534 INFO:tasks.workunit.client.1.smithi148.stdout:3/689: dread f1 [0,4194304] 0 2024-09-21T04:15:23.538 INFO:tasks.workunit.client.1.smithi148.stdout:9/802: dwrite d5/d60/d28/f6a [0,4194304] 0 2024-09-21T04:15:23.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/790: symlink d0/d1a/d23/d32/dc8/lef 0 2024-09-21T04:15:23.541 INFO:tasks.workunit.client.1.smithi148.stdout:6/791: write d0/d8/d18/f4d [4513837,98730] 0 2024-09-21T04:15:23.541 INFO:tasks.workunit.client.1.smithi148.stdout:0/764: unlink d5/d6/cb3 0 2024-09-21T04:15:23.542 INFO:tasks.workunit.client.1.smithi148.stdout:0/765: truncate d5/d6/d7/d11/d48/f7d 4780297 0 2024-09-21T04:15:23.547 INFO:tasks.workunit.client.1.smithi148.stdout:9/803: unlink d5/d1f/d81/dcf/d106/lf4 0 2024-09-21T04:15:23.547 INFO:tasks.workunit.client.1.smithi148.stdout:9/804: write d5/d1f/d81/dcf/d106/d80/d104/f99 [89931,118959] 0 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/792: rename d0/d1a/d23/dc/d68/d5b to d0/d1a/d23/d27/d80/df0 0 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/793: write d0/d1a/db8/dbe/fc3 [4483454,94788] 0 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/794: write d0/d34/d38/d43/f7b [2170628,16261] 0 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:6/795: chown d0/d1a/d23/dc/d68/f86 220 1 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:0/766: rmdir d5/d6/d2a/d2b/d99/deb/daf/dc8 39 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:0/767: read d5/d6/d2a/d2b/fd1 [2765407,72450] 0 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:0/768: chown d5/d6/d7/d11/d48/f80 645314035 1 2024-09-21T04:15:23.551 INFO:tasks.workunit.client.1.smithi148.stdout:0/769: chown d5/d6/d7/dd0 248 1 2024-09-21T04:15:23.552 INFO:tasks.workunit.client.1.smithi148.stdout:0/770: read d5/d6/d7/d11/f17 [5330672,4095] 0 2024-09-21T04:15:23.553 INFO:tasks.workunit.client.1.smithi148.stdout:6/796: mkdir d0/d1a/d23/d27/d80/da4/df1 0 2024-09-21T04:15:23.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/771: creat d5/d82/df0/ff2 x:0 0 0 2024-09-21T04:15:23.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/772: fsync d5/d6/d2a/d2b/d99/deb/db4/fb5 0 2024-09-21T04:15:23.556 INFO:tasks.workunit.client.1.smithi148.stdout:0/773: rename d5/d6/d7/d22/f6c to d5/d6/d2a/d45/d70/ff3 0 2024-09-21T04:15:23.558 INFO:tasks.workunit.client.1.smithi148.stdout:0/774: mkdir d5/d6/d2a/d3f/d6d/da3/df4 0 2024-09-21T04:15:23.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/797: dread d0/d1a/d23/dc/d68/fa6 [0,4194304] 0 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/798: mknod d0/d34/d38/d57/cf2 0 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/799: write d0/d1a/d23/dc/d68/f66 [916384,31166] 0 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/800: chown d0/d1a/d23/d27/d80/da4 14965 1 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/801: dread - d0/d8/d85/f9d zero size 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/802: dread - d0/d34/d38/d43/feb zero size 2024-09-21T04:15:23.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/803: chown d0/d1a 3 1 2024-09-21T04:15:23.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/804: creat d0/d34/da0/ff3 x:0 0 0 2024-09-21T04:15:23.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/805: chown d0/d1a/d23/l2b 4102686 1 2024-09-21T04:15:23.563 INFO:tasks.workunit.client.1.smithi148.stdout:6/806: link d0/d1a/d23/d27/d80/da4/fd9 d0/d8/dba/ff4 0 2024-09-21T04:15:23.563 INFO:tasks.workunit.client.1.smithi148.stdout:6/807: readlink d0/d1a/d23/dc/l56 0 2024-09-21T04:15:23.564 INFO:tasks.workunit.client.1.smithi148.stdout:6/808: mkdir d0/d34/da0/df5 0 2024-09-21T04:15:23.564 INFO:tasks.workunit.client.1.smithi148.stdout:6/809: fsync d0/d1a/d23/d32/d93/fd2 0 2024-09-21T04:15:23.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/810: symlink d0/d1a/db8/lf6 0 2024-09-21T04:15:23.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/811: mknod d0/d34/da0/df5/cf7 0 2024-09-21T04:15:23.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/812: mknod d0/d1a/d23/d32/d93/cf8 0 2024-09-21T04:15:23.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/813: write d0/d1a/d23/f14 [569477,103022] 0 2024-09-21T04:15:23.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/883: dwrite d0/d9/d16/d55/ffc [0,4194304] 0 2024-09-21T04:15:23.569 INFO:tasks.workunit.client.1.smithi148.stdout:6/814: unlink d0/d1a/d23/d32/l41 0 2024-09-21T04:15:23.571 INFO:tasks.workunit.client.1.smithi148.stdout:0/775: read d5/d6/d7/d18/f32 [3923300,12106] 0 2024-09-21T04:15:23.572 INFO:tasks.workunit.client.1.smithi148.stdout:2/884: mknod d0/d9/d38/c119 0 2024-09-21T04:15:23.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/690: dwrite d3/d5/d6/d12/d43/d54/f71 [0,4194304] 0 2024-09-21T04:15:23.573 INFO:tasks.workunit.client.0.smithi110.stdout:5/673: dwrite d1/d22/f27 [4194304,4194304] 0 2024-09-21T04:15:23.573 INFO:tasks.workunit.client.0.smithi110.stdout:1/988: dwrite d2/d9/d23/d4e/f7f [0,4194304] 0 2024-09-21T04:15:23.574 INFO:tasks.workunit.client.1.smithi148.stdout:6/815: mkdir d0/df9 0 2024-09-21T04:15:23.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/865: dwrite d17/d1e/d77/d10c/d20/d43/d51/d7a/d86/f9b [0,4194304] 0 2024-09-21T04:15:23.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/866: creat d17/d3e/dcd/f128 x:0 0 0 2024-09-21T04:15:23.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/867: write d17/d1e/d77/d10c/d20/d43/d51/d52/fb1 [5115729,6447] 0 2024-09-21T04:15:23.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/776: mkdir d5/d6/d2a/d3f/d8b/dab/df5 0 2024-09-21T04:15:23.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/777: truncate d5/d82/fc4 420064 0 2024-09-21T04:15:23.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/778: write d5/d6/d2a/d2b/fd1 [1779660,35366] 0 2024-09-21T04:15:23.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/779: readlink d5/d6/d2a/d3f/d6d/da3/lad 0 2024-09-21T04:15:23.576 INFO:tasks.workunit.client.0.smithi110.stdout:1/989: dread d2/d9/d26/d3d/d8f/fa3 [0,4194304] 0 2024-09-21T04:15:23.576 INFO:tasks.workunit.client.0.smithi110.stdout:1/990: readlink d2/d9/d49/d4f/d87/ld6 0 2024-09-21T04:15:23.576 INFO:tasks.workunit.client.0.smithi110.stdout:1/991: fsync d2/d9/d49/d4f/f67 0 2024-09-21T04:15:23.583 INFO:tasks.workunit.client.1.smithi148.stdout:3/691: symlink d3/d5/d6/le1 0 2024-09-21T04:15:23.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/805: dwrite d5/d60/d28/d73/fc3 [0,4194304] 0 2024-09-21T04:15:23.584 INFO:tasks.workunit.client.0.smithi110.stdout:1/992: dread d2/f48 [0,4194304] 0 2024-09-21T04:15:23.584 INFO:tasks.workunit.client.0.smithi110.stdout:1/993: truncate d2/d9/d23/f29 3843520 0 2024-09-21T04:15:23.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/994: creat d2/d9/d49/d5c/f141 x:0 0 0 2024-09-21T04:15:23.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/995: write d2/d9/d49/d4f/d87/f12d [728674,9239] 0 2024-09-21T04:15:23.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/996: readlink d2/d9/d26/lfa 0 2024-09-21T04:15:23.585 INFO:tasks.workunit.client.1.smithi148.stdout:6/816: symlink d0/d1a/d23/d27/lfa 0 2024-09-21T04:15:23.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/780: truncate d5/d6/d2a/d2b/d99/deb/da5/f49 2339636 0 2024-09-21T04:15:23.587 INFO:tasks.workunit.client.1.smithi148.stdout:0/781: fsync d5/d6/d7/d11/d48/f7d 0 2024-09-21T04:15:23.587 INFO:tasks.workunit.client.1.smithi148.stdout:0/782: truncate d5/d6/d7/d11/d48/fc0 603239 0 2024-09-21T04:15:23.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/674: dwrite d1/d49/d4d/f9c [0,4194304] 0 2024-09-21T04:15:23.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/675: fsync d1/d7/d18/f87 0 2024-09-21T04:15:23.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/676: getdents d1/d22/db6/de1 0 2024-09-21T04:15:23.589 INFO:tasks.workunit.client.0.smithi110.stdout:5/677: write d1/d49/d6f/f78 [222433,56363] 0 2024-09-21T04:15:23.589 INFO:tasks.workunit.client.1.smithi148.stdout:3/692: creat d3/d5/d6/d12/d43/fe2 x:0 0 0 2024-09-21T04:15:23.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/701: sync 2024-09-21T04:15:23.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/817: dread d0/d1a/d23/d27/d62/d9a/d92/fa2 [0,4194304] 0 2024-09-21T04:15:23.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/818: readlink d0/l35 0 2024-09-21T04:15:23.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/885: dwrite d0/d2b/f53 [0,4194304] 0 2024-09-21T04:15:23.603 INFO:tasks.workunit.client.0.smithi110.stdout:5/678: creat d1/d49/d4d/d1e/d4a/dd4/dda/fe7 x:0 0 0 2024-09-21T04:15:23.604 INFO:tasks.workunit.client.0.smithi110.stdout:5/679: chown d1/d49/d4d/l1f 1838 1 2024-09-21T04:15:23.604 INFO:tasks.workunit.client.0.smithi110.stdout:5/680: stat d1/d22/d51/lc0 0 2024-09-21T04:15:23.607 INFO:tasks.workunit.client.1.smithi148.stdout:9/806: mknod d5/d1e/d37/d50/c11e 0 2024-09-21T04:15:23.607 INFO:tasks.workunit.client.1.smithi148.stdout:9/807: chown d5/d1f/d81/d9e/da2/fc5 94 1 2024-09-21T04:15:23.608 INFO:tasks.workunit.client.0.smithi110.stdout:5/681: unlink d1/d7/d58/d9e/lc7 0 2024-09-21T04:15:23.608 INFO:tasks.workunit.client.0.smithi110.stdout:5/682: chown d1/d7/d18/d3f/caa 0 1 2024-09-21T04:15:23.608 INFO:tasks.workunit.client.0.smithi110.stdout:5/683: creat d1/d49/d4d/d1e/d4a/dd4/dda/fe8 x:0 0 0 2024-09-21T04:15:23.608 INFO:tasks.workunit.client.0.smithi110.stdout:5/684: read d1/d22/d51/f70 [707450,108098] 0 2024-09-21T04:15:23.611 INFO:tasks.workunit.client.1.smithi148.stdout:0/783: mknod d5/d6/d2a/d2b/d99/deb/daf/dc8/cf6 0 2024-09-21T04:15:23.611 INFO:tasks.workunit.client.1.smithi148.stdout:0/784: fdatasync d5/d6/d7/f83 0 2024-09-21T04:15:23.613 INFO:tasks.workunit.client.1.smithi148.stdout:3/693: symlink d3/d1f/le3 0 2024-09-21T04:15:23.613 INFO:tasks.workunit.client.1.smithi148.stdout:3/694: dread - d3/d5/d6/d12/d43/d54/d9b/fda zero size 2024-09-21T04:15:23.615 INFO:tasks.workunit.client.1.smithi148.stdout:8/702: unlink dc/d13/d28/f2f 0 2024-09-21T04:15:23.615 INFO:tasks.workunit.client.1.smithi148.stdout:8/703: dread - dc/d13/d39/fc6 zero size 2024-09-21T04:15:23.616 INFO:tasks.workunit.client.1.smithi148.stdout:6/819: dwrite d0/d1a/d23/d27/d62/d9a/f7a [0,4194304] 0 2024-09-21T04:15:23.616 INFO:tasks.workunit.client.1.smithi148.stdout:6/820: fsync d0/d1a/d23/d32/d93/d96/fae 0 2024-09-21T04:15:23.620 INFO:tasks.workunit.client.1.smithi148.stdout:9/808: link d5/d1e/d37/fb0 d5/d2b/d32/f11f 0 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/785: truncate d5/d6/d7/d22/f46 8268561 0 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/786: truncate d5/d6/f15 4807320 0 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/787: chown d5/d82/f87 2 1 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.0.smithi110.stdout:5/685: write d1/d7/d18/f28 [2925365,25756] 0 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.0.smithi110.stdout:5/686: readlink d1/d39/l42 0 2024-09-21T04:15:23.621 INFO:tasks.workunit.client.0.smithi110.stdout:5/687: dread - d1/d39/da2/fde zero size 2024-09-21T04:15:23.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/695: mkdir d3/d5/d6/d2b/da2/da3/de4 0 2024-09-21T04:15:23.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/696: chown d3/d5/d5c 66 1 2024-09-21T04:15:23.622 INFO:tasks.workunit.client.1.smithi148.stdout:3/697: fsync d3/d1f/d4d/d82/d23/f20 0 2024-09-21T04:15:23.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/698: creat d3/d5/fe5 x:0 0 0 2024-09-21T04:15:23.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/699: fsync d3/d5/d6/d2b/d96/f9d 0 2024-09-21T04:15:23.623 INFO:tasks.workunit.client.1.smithi148.stdout:8/704: creat dc/d13/d6a/fda x:0 0 0 2024-09-21T04:15:23.625 INFO:tasks.workunit.client.1.smithi148.stdout:2/886: dwrite d0/d2b/da3/f6b [4194304,4194304] 0 2024-09-21T04:15:23.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/688: mkdir d1/dc/de9 0 2024-09-21T04:15:23.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/689: getdents d1/d49/de4 0 2024-09-21T04:15:23.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/690: dread - d1/d7/d58/d72/fc9 zero size 2024-09-21T04:15:23.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/821: mkdir d0/d1a/d23/d27/d62/dfb 0 2024-09-21T04:15:23.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/822: chown d0/d1a/d23/cb1 4 1 2024-09-21T04:15:23.628 INFO:tasks.workunit.client.1.smithi148.stdout:0/788: mkdir d5/d6/d2a/d2b/d99/deb/daf/dc8/de8/df7 0 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:0/964: sync 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:2/542: sync 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/597: sync 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/598: stat d0/d12/d16/d1a/d6f/f73 0 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/599: chown d0/d12/fb3 3392 1 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/600: creat d0/fd3 x:0 0 0 2024-09-21T04:15:23.631 INFO:tasks.workunit.client.1.smithi148.stdout:0/789: read d5/d6/d2a/d45/d70/fc6 [1404020,33030] 0 2024-09-21T04:15:23.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/705: symlink dc/d13/d3c/d81/db0/ldb 0 2024-09-21T04:15:23.632 INFO:tasks.workunit.client.1.smithi148.stdout:9/809: dread d5/fd [0,4194304] 0 2024-09-21T04:15:23.633 INFO:tasks.workunit.client.1.smithi148.stdout:2/887: creat d0/d9/d16/d55/d71/dc2/dcb/f11a x:0 0 0 2024-09-21T04:15:23.634 INFO:tasks.workunit.client.1.smithi148.stdout:6/823: mknod d0/df9/cfc 0 2024-09-21T04:15:23.639 INFO:tasks.workunit.client.1.smithi148.stdout:9/810: symlink d5/d1e/d37/d46/l120 0 2024-09-21T04:15:23.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/824: creat d0/d8/dba/dbc/ffd x:0 0 0 2024-09-21T04:15:23.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/825: chown d0/d1a/d23/d27/d62 5 1 2024-09-21T04:15:23.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/826: chown d0/d34/d38/d57 843 1 2024-09-21T04:15:23.642 INFO:tasks.workunit.client.1.smithi148.stdout:9/811: mknod d5/d1f/d81/dcf/d106/d80/d104/c121 0 2024-09-21T04:15:23.642 INFO:tasks.workunit.client.1.smithi148.stdout:4/850: sync 2024-09-21T04:15:23.646 INFO:tasks.workunit.client.1.smithi148.stdout:4/851: mknod d4/d87/c10b 0 2024-09-21T04:15:23.647 INFO:tasks.workunit.client.1.smithi148.stdout:2/888: write d0/d9/dc/f112 [2571578,84120] 0 2024-09-21T04:15:23.647 INFO:tasks.workunit.client.0.smithi110.stdout:2/543: dread d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:23.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/852: mkdir d4/d12/d78/de4/d10c 0 2024-09-21T04:15:23.651 INFO:tasks.workunit.client.1.smithi148.stdout:2/889: creat d0/d2f/db1/f11b x:0 0 0 2024-09-21T04:15:23.653 INFO:tasks.workunit.client.0.smithi110.stdout:2/544: mkdir d2/d1a/da3 0 2024-09-21T04:15:23.654 INFO:tasks.workunit.client.0.smithi110.stdout:2/545: creat d2/d1a/fa4 x:0 0 0 2024-09-21T04:15:23.655 INFO:tasks.workunit.client.0.smithi110.stdout:2/546: chown d2/c99 1686098 1 2024-09-21T04:15:23.657 INFO:tasks.workunit.client.1.smithi148.stdout:4/853: write d4/d12/d16/d29/f3d [1800694,48775] 0 2024-09-21T04:15:23.657 INFO:tasks.workunit.client.1.smithi148.stdout:4/854: chown d4/d12/d75/d9c 34625229 1 2024-09-21T04:15:23.657 INFO:tasks.workunit.client.1.smithi148.stdout:4/855: write d4/d87/d2d/dd0/fd6 [517766,45253] 0 2024-09-21T04:15:23.660 INFO:tasks.workunit.client.1.smithi148.stdout:5/772: sync 2024-09-21T04:15:23.661 INFO:tasks.workunit.client.1.smithi148.stdout:1/765: sync 2024-09-21T04:15:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/856: creat d4/d12/d16/d29/d61/d6c/df3/f10d x:0 0 0 2024-09-21T04:15:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/700: dwrite d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 [0,4194304] 0 2024-09-21T04:15:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:1/766: symlink db/d24/d2e/d71/d94/dde/de1/lfa 0 2024-09-21T04:15:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:1/767: write db/d24/d2e/d71/d94/d9a/f72 [976163,59963] 0 2024-09-21T04:15:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/857: creat d4/d12/d16/d29/d61/d6c/df1/f10e x:0 0 0 2024-09-21T04:15:23.663 INFO:tasks.workunit.client.0.smithi110.stdout:3/601: dwrite d0/d22/d29/d76/fb1 [0,4194304] 0 2024-09-21T04:15:23.663 INFO:tasks.workunit.client.0.smithi110.stdout:3/602: readlink d0/d22/l59 0 2024-09-21T04:15:23.663 INFO:tasks.workunit.client.0.smithi110.stdout:0/965: dwrite d4/dd/d10/f1a [0,4194304] 0 2024-09-21T04:15:23.663 INFO:tasks.workunit.client.0.smithi110.stdout:5/691: dwrite d1/d7/d18/f74 [0,4194304] 0 2024-09-21T04:15:23.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/890: read d0/d2b/fbb [3446634,126865] 0 2024-09-21T04:15:23.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/891: write d0/d2b/da3/d95/fa9 [444157,27782] 0 2024-09-21T04:15:23.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/892: truncate d0/d2b/da3/d8f/f9d 1212268 0 2024-09-21T04:15:23.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/893: fsync d0/d2b/f4b 0 2024-09-21T04:15:23.669 INFO:tasks.workunit.client.1.smithi148.stdout:3/701: symlink d3/d5/db4/le6 0 2024-09-21T04:15:23.671 INFO:tasks.workunit.client.0.smithi110.stdout:3/603: mknod d0/d12/d3c/dad/cd4 0 2024-09-21T04:15:23.671 INFO:tasks.workunit.client.1.smithi148.stdout:0/790: rename d5/d6/d2a/d2b/d99/deb to d5/d6/d2a/dcd/df8 0 2024-09-21T04:15:23.671 INFO:tasks.workunit.client.1.smithi148.stdout:0/791: fdatasync d5/d6/d7/d11/d48/f6b 0 2024-09-21T04:15:23.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/768: link db/dd/d21/d89/dc9/fcd db/d24/d2e/d71/d94/dde/ffb 0 2024-09-21T04:15:23.674 INFO:tasks.workunit.client.1.smithi148.stdout:4/858: creat d4/d12/d16/df0/dec/f10f x:0 0 0 2024-09-21T04:15:23.674 INFO:tasks.workunit.client.0.smithi110.stdout:3/604: rename d0/d12/f1b to d0/d12/fd5 0 2024-09-21T04:15:23.676 INFO:tasks.workunit.client.1.smithi148.stdout:2/894: write d0/d9/d16/f60 [1349974,60489] 0 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/706: rename dc/d13/d28/c84 to dc/d43/d4c/d4f/dc5/cdc 0 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/707: dread - dc/d13/d76/d5d/fa6 zero size 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/708: write f7 [2161689,88104] 0 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/709: truncate dc/d13/d76/d5d/d78/f9f 769305 0 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:8/710: dread - dc/d13/d39/d6e/fd7 zero size 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/792: mkdir d5/d82/df9 0 2024-09-21T04:15:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/793: chown d5/d6/d2a/d3f/d6d/l72 10799 1 2024-09-21T04:15:23.686 INFO:tasks.workunit.client.1.smithi148.stdout:1/769: rmdir db/dd/db9/ddc 0 2024-09-21T04:15:23.687 INFO:tasks.workunit.client.1.smithi148.stdout:4/859: creat d4/d12/d16/d29/d61/f110 x:0 0 0 2024-09-21T04:15:23.687 INFO:tasks.workunit.client.1.smithi148.stdout:4/860: write d4/d87/f32 [768749,116497] 0 2024-09-21T04:15:23.689 INFO:tasks.workunit.client.0.smithi110.stdout:2/547: rmdir d2/d1a 39 2024-09-21T04:15:23.690 INFO:tasks.workunit.client.0.smithi110.stdout:3/605: dread d0/f56 [0,4194304] 0 2024-09-21T04:15:23.691 INFO:tasks.workunit.client.0.smithi110.stdout:3/606: stat d0/d12/d2a/f30 0 2024-09-21T04:15:23.691 INFO:tasks.workunit.client.0.smithi110.stdout:3/607: readlink d0/d12/d16/d1a/d6f/la0 0 2024-09-21T04:15:23.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/827: rename d0/d34/da0/df5/cf7 to d0/d1a/d23/d27/d62/dc4/cfe 0 2024-09-21T04:15:23.693 INFO:tasks.workunit.client.1.smithi148.stdout:6/828: dread - d0/d1a/d23/dc/d81/fed zero size 2024-09-21T04:15:23.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/794: write d5/d6/d2a/d3f/d8b/dab/fc9 [3617257,102665] 0 2024-09-21T04:15:23.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/795: chown d5/d6/d7/d11/c8e 7617 1 2024-09-21T04:15:23.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/608: rename d0/d12/f82 to d0/d22/fd6 0 2024-09-21T04:15:23.697 INFO:tasks.workunit.client.1.smithi148.stdout:4/861: mknod d4/d12/d16/df0/dec/c111 0 2024-09-21T04:15:23.697 INFO:tasks.workunit.client.1.smithi148.stdout:4/862: chown d4/d12/d16/d29/d61/d6c/d9d/fca 5176 1 2024-09-21T04:15:23.700 INFO:tasks.workunit.client.0.smithi110.stdout:2/548: write d2/d50/f7f [2138735,54591] 0 2024-09-21T04:15:23.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/895: rmdir d0/d2b/da3/d8f 39 2024-09-21T04:15:23.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/896: write d0/f4 [4485606,130107] 0 2024-09-21T04:15:23.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/897: chown d0/d9/dc/l5d 1898533186 1 2024-09-21T04:15:23.702 INFO:tasks.workunit.client.0.smithi110.stdout:2/549: mkdir d2/d1a/d8b/da5 0 2024-09-21T04:15:23.703 INFO:tasks.workunit.client.1.smithi148.stdout:6/829: creat d0/d1a/d23/d27/d80/db0/fff x:0 0 0 2024-09-21T04:15:23.703 INFO:tasks.workunit.client.0.smithi110.stdout:0/966: dwrite d4/d47/f23 [0,4194304] 0 2024-09-21T04:15:23.703 INFO:tasks.workunit.client.0.smithi110.stdout:5/692: dwrite d1/d39/f45 [0,4194304] 0 2024-09-21T04:15:23.704 INFO:tasks.workunit.client.0.smithi110.stdout:5/693: getdents d1/dc/de9 0 2024-09-21T04:15:23.704 INFO:tasks.workunit.client.1.smithi148.stdout:3/702: dwrite d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 [0,4194304] 0 2024-09-21T04:15:23.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/550: mknod d2/d1a/d1f/d65/ca6 0 2024-09-21T04:15:23.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/551: fsync d2/f44 0 2024-09-21T04:15:23.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/863: creat d4/d66/d7f/f112 x:0 0 0 2024-09-21T04:15:23.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/864: readlink d4/d12/d16/lee 0 2024-09-21T04:15:23.706 INFO:tasks.workunit.client.1.smithi148.stdout:4/865: chown d4/d66/d7f/d9a/dce 8351 1 2024-09-21T04:15:23.709 INFO:tasks.workunit.client.0.smithi110.stdout:0/967: link d4/dd/d60/de5/ca0 d4/d1c/d62/d69/c126 0 2024-09-21T04:15:23.710 INFO:tasks.workunit.client.0.smithi110.stdout:0/968: write d4/d1c/d62/d69/ffa [896090,18251] 0 2024-09-21T04:15:23.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/711: rename dc/d13/d39/l55 to dc/ldd 0 2024-09-21T04:15:23.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/712: dread dc/d13/d6a/fa5 [0,4194304] 0 2024-09-21T04:15:23.716 INFO:tasks.workunit.client.0.smithi110.stdout:5/694: rename d1/d49/d69/fca to d1/dc/fea 0 2024-09-21T04:15:23.717 INFO:tasks.workunit.client.1.smithi148.stdout:6/830: truncate d0/d1a/d23/dc/d68/f66 3018895 0 2024-09-21T04:15:23.717 INFO:tasks.workunit.client.1.smithi148.stdout:6/831: dread - d0/d1a/d23/dc/d81/fed zero size 2024-09-21T04:15:23.720 INFO:tasks.workunit.client.0.smithi110.stdout:2/552: mknod d2/d50/d7b/ca7 0 2024-09-21T04:15:23.720 INFO:tasks.workunit.client.0.smithi110.stdout:2/553: write d2/d1a/d1f/d2a/f61 [565393,11733] 0 2024-09-21T04:15:23.720 INFO:tasks.workunit.client.1.smithi148.stdout:3/703: truncate d3/d1f/d4d/d82/d21/fd2 2960461 0 2024-09-21T04:15:23.720 INFO:tasks.workunit.client.1.smithi148.stdout:3/704: readlink d3/l51 0 2024-09-21T04:15:23.720 INFO:tasks.workunit.client.1.smithi148.stdout:3/705: creat d3/d5/fe7 x:0 0 0 2024-09-21T04:15:23.721 INFO:tasks.workunit.client.0.smithi110.stdout:5/695: symlink d1/dc/leb 0 2024-09-21T04:15:23.721 INFO:tasks.workunit.client.0.smithi110.stdout:5/696: fdatasync d1/d22/f97 0 2024-09-21T04:15:23.724 INFO:tasks.workunit.client.1.smithi148.stdout:4/866: write d4/d87/f5c [3428366,58116] 0 2024-09-21T04:15:23.726 INFO:tasks.workunit.client.1.smithi148.stdout:8/713: dread dc/d13/d3c/f49 [0,4194304] 0 2024-09-21T04:15:23.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/706: creat d3/d1f/d4d/d82/fe8 x:0 0 0 2024-09-21T04:15:23.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/707: dread - d3/d5/d6/d2b/d96/f9d zero size 2024-09-21T04:15:23.727 INFO:tasks.workunit.client.1.smithi148.stdout:3/708: write d3/d5/d6/d12/d43/d54/fc6 [791982,9767] 0 2024-09-21T04:15:23.727 INFO:tasks.workunit.client.0.smithi110.stdout:0/969: dread d4/dd/d10/f13 [0,4194304] 0 2024-09-21T04:15:23.729 INFO:tasks.workunit.client.1.smithi148.stdout:7/757: sync 2024-09-21T04:15:23.729 INFO:tasks.workunit.client.1.smithi148.stdout:8/714: rename dc/d13/d3c/d81/d98/f99 to dc/d13/dbd/fde 0 2024-09-21T04:15:23.729 INFO:tasks.workunit.client.1.smithi148.stdout:3/709: truncate d3/d1f/f47 9641387 0 2024-09-21T04:15:23.729 INFO:tasks.workunit.client.1.smithi148.stdout:3/710: write d3/d1f/d4d/d82/d23/d28/d65/f8f [437553,110164] 0 2024-09-21T04:15:23.732 INFO:tasks.workunit.client.1.smithi148.stdout:3/711: creat d3/d5/db4/fe9 x:0 0 0 2024-09-21T04:15:23.732 INFO:tasks.workunit.client.1.smithi148.stdout:3/712: write d3/d5/d6/d2b/da2/fc8 [297288,73228] 0 2024-09-21T04:15:23.734 INFO:tasks.workunit.client.0.smithi110.stdout:5/697: dread d1/dc/f1a [0,4194304] 0 2024-09-21T04:15:23.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/758: write d5/f3d [4098316,126986] 0 2024-09-21T04:15:23.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/759: creat d5/d14/f10b x:0 0 0 2024-09-21T04:15:23.738 INFO:tasks.workunit.client.1.smithi148.stdout:3/713: getdents d3/d1f/d4d/d82/d52 0 2024-09-21T04:15:23.738 INFO:tasks.workunit.client.0.smithi110.stdout:0/970: write d4/dd/d10/fa1 [1521429,129566] 0 2024-09-21T04:15:23.739 INFO:tasks.workunit.client.1.smithi148.stdout:1/770: dwrite db/d24/d2e/f63 [0,4194304] 0 2024-09-21T04:15:23.740 INFO:tasks.workunit.client.1.smithi148.stdout:0/796: dwrite d5/d6/d2a/d2b/d99/db7/fbb [0,4194304] 0 2024-09-21T04:15:23.742 INFO:tasks.workunit.client.1.smithi148.stdout:1/771: mkdir db/d24/d2e/d71/dfc 0 2024-09-21T04:15:23.743 INFO:tasks.workunit.client.1.smithi148.stdout:0/797: mknod d5/d6/d7/dd0/cfa 0 2024-09-21T04:15:23.743 INFO:tasks.workunit.client.1.smithi148.stdout:0/798: read - d5/d6/d2a/dcd/df8/daf/f90 zero size 2024-09-21T04:15:23.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/760: rename d5/d14/d25/d79/d86/fca to d5/d8/f10c 0 2024-09-21T04:15:23.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/772: link db/l49 db/d24/d2e/d71/lfd 0 2024-09-21T04:15:23.746 INFO:tasks.workunit.client.1.smithi148.stdout:7/761: mknod d5/d14/d25/d40/dab/dad/c10d 0 2024-09-21T04:15:23.747 INFO:tasks.workunit.client.1.smithi148.stdout:1/773: mknod db/d46/d4c/cfe 0 2024-09-21T04:15:23.758 INFO:tasks.workunit.client.1.smithi148.stdout:3/714: dread d3/d84/f7e [0,4194304] 0 2024-09-21T04:15:23.760 INFO:tasks.workunit.client.0.smithi110.stdout:3/609: dwrite d0/d12/d3c/f45 [0,4194304] 0 2024-09-21T04:15:23.761 INFO:tasks.workunit.client.1.smithi148.stdout:1/774: read db/d24/d2e/d71/d94/d9a/f72 [947453,112243] 0 2024-09-21T04:15:23.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/898: dwrite d0/d2b/fdd [0,4194304] 0 2024-09-21T04:15:23.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/899: creat d0/d9/d16/f11c x:0 0 0 2024-09-21T04:15:23.775 INFO:tasks.workunit.client.0.smithi110.stdout:2/554: dwrite d2/d1a/d1f/d29/f4f [0,4194304] 0 2024-09-21T04:15:23.778 INFO:tasks.workunit.client.0.smithi110.stdout:2/555: mkdir d2/d81/da8 0 2024-09-21T04:15:23.778 INFO:tasks.workunit.client.0.smithi110.stdout:2/556: chown d2/d1a/d1f/d29/d19/l40 958651 1 2024-09-21T04:15:23.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/557: unlink d2/d1a/d1f/d5f/l79 0 2024-09-21T04:15:23.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/558: write d2/d1a/d1f/d29/f48 [1312875,12044] 0 2024-09-21T04:15:23.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/559: fsync d2/d50/d56/f94 0 2024-09-21T04:15:23.781 INFO:tasks.workunit.client.1.smithi148.stdout:2/900: mkdir d0/d2b/da3/da1/d11d 0 2024-09-21T04:15:23.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/775: truncate db/d24/f26 829676 0 2024-09-21T04:15:23.781 INFO:tasks.workunit.client.1.smithi148.stdout:1/776: creat db/d46/d4c/da3/fff x:0 0 0 2024-09-21T04:15:23.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/777: readlink db/dd/db9/lee 0 2024-09-21T04:15:23.782 INFO:tasks.workunit.client.1.smithi148.stdout:1/778: write db/d46/d4c/da3/fff [511663,13658] 0 2024-09-21T04:15:23.782 INFO:tasks.workunit.client.0.smithi110.stdout:2/560: mkdir d2/d1a/d8b/da5/da9 0 2024-09-21T04:15:23.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/901: symlink d0/d9/d16/l11e 0 2024-09-21T04:15:23.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/902: creat d0/d2f/f11f x:0 0 0 2024-09-21T04:15:23.785 INFO:tasks.workunit.client.0.smithi110.stdout:2/561: mknod d2/d1a/d1f/d5f/caa 0 2024-09-21T04:15:23.785 INFO:tasks.workunit.client.0.smithi110.stdout:2/562: write d2/f3b [1829225,76813] 0 2024-09-21T04:15:23.785 INFO:tasks.workunit.client.0.smithi110.stdout:2/563: chown d2/d1a/d1f/l96 44127683 1 2024-09-21T04:15:23.787 INFO:tasks.workunit.client.1.smithi148.stdout:1/779: rename db/d46/d4c/led to db/d24/d2e/daf/l100 0 2024-09-21T04:15:23.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/867: dwrite d4/d66/d7f/d2a/f36 [0,4194304] 0 2024-09-21T04:15:23.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/868: truncate d4/d12/d16/d29/d61/d6c/df3/f10d 982269 0 2024-09-21T04:15:23.788 INFO:tasks.workunit.client.1.smithi148.stdout:4/869: read - d4/d66/d7f/d9a/dce/ffe zero size 2024-09-21T04:15:23.789 INFO:tasks.workunit.client.0.smithi110.stdout:2/564: creat d2/d50/d9c/fab x:0 0 0 2024-09-21T04:15:23.790 INFO:tasks.workunit.client.1.smithi148.stdout:1/780: mknod db/d24/d2e/d71/d94/dde/de1/c101 0 2024-09-21T04:15:23.791 INFO:tasks.workunit.client.1.smithi148.stdout:3/715: dread d3/d1f/f47 [4194304,4194304] 0 2024-09-21T04:15:23.792 INFO:tasks.workunit.client.0.smithi110.stdout:2/565: link d2/c5c d2/d1a/d8b/cac 0 2024-09-21T04:15:23.792 INFO:tasks.workunit.client.0.smithi110.stdout:5/698: dwrite d1/d7/fa1 [0,4194304] 0 2024-09-21T04:15:23.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/781: dread db/d24/d2e/d71/d94/dde/ffb [0,4194304] 0 2024-09-21T04:15:23.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/832: dwrite d0/d1a/f83 [0,4194304] 0 2024-09-21T04:15:23.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/833: readlink d0/d8/d18/lc5 0 2024-09-21T04:15:23.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/834: creat d0/d34/f100 x:0 0 0 2024-09-21T04:15:23.794 INFO:tasks.workunit.client.1.smithi148.stdout:6/835: chown d0/d8/c7e 448 1 2024-09-21T04:15:23.795 INFO:tasks.workunit.client.1.smithi148.stdout:0/799: dwrite d5/d6/d2a/dcd/df8/daf/f90 [0,4194304] 0 2024-09-21T04:15:23.795 INFO:tasks.workunit.client.0.smithi110.stdout:3/610: dwrite d0/d22/d29/d76/fa6 [0,4194304] 0 2024-09-21T04:15:23.795 INFO:tasks.workunit.client.0.smithi110.stdout:0/971: dwrite d4/dd/d53/d7b/ffe [0,4194304] 0 2024-09-21T04:15:23.795 INFO:tasks.workunit.client.1.smithi148.stdout:3/716: rename d3/d5/d6/d12/d15/f6c to d3/d1f/d4d/d82/d23/d28/d46/db3/fea 0 2024-09-21T04:15:23.797 INFO:tasks.workunit.client.0.smithi110.stdout:2/566: mkdir d2/d1a/d1f/d5f/d9d/d86/dad 0 2024-09-21T04:15:23.802 INFO:tasks.workunit.client.1.smithi148.stdout:1/782: unlink db/d46/f48 0 2024-09-21T04:15:23.805 INFO:tasks.workunit.client.0.smithi110.stdout:5/699: dread d1/d49/d4d/d1e/d5a/d64/f73 [0,4194304] 0 2024-09-21T04:15:23.806 INFO:tasks.workunit.client.0.smithi110.stdout:5/700: write d1/dc/fd6 [915498,96409] 0 2024-09-21T04:15:23.806 INFO:tasks.workunit.client.0.smithi110.stdout:5/701: write d1/d22/f44 [1191540,79505] 0 2024-09-21T04:15:23.806 INFO:tasks.workunit.client.0.smithi110.stdout:5/702: read d1/d7/d18/d8f/fd3 [3601549,110426] 0 2024-09-21T04:15:23.806 INFO:tasks.workunit.client.0.smithi110.stdout:5/703: fdatasync d1/f35 0 2024-09-21T04:15:23.808 INFO:tasks.workunit.client.0.smithi110.stdout:5/704: write d1/d49/d6f/fa8 [43207,69070] 0 2024-09-21T04:15:23.812 INFO:tasks.workunit.client.1.smithi148.stdout:4/870: dwrite d4/fd8 [4194304,4194304] 0 2024-09-21T04:15:23.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/611: rename d0/d12/d16/d1a/la9 to d0/ld7 0 2024-09-21T04:15:23.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/612: fdatasync d0/d12/fd5 0 2024-09-21T04:15:23.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/613: fsync d0/d12/d16/f25 0 2024-09-21T04:15:23.813 INFO:tasks.workunit.client.0.smithi110.stdout:0/972: truncate d4/dd/d10/d8a/d92/fb6 442817 0 2024-09-21T04:15:23.813 INFO:tasks.workunit.client.1.smithi148.stdout:6/836: dread d0/d1a/d23/dc/d68/f50 [0,4194304] 0 2024-09-21T04:15:23.815 INFO:tasks.workunit.client.1.smithi148.stdout:3/717: dread d3/d5/d6/f17 [4194304,4194304] 0 2024-09-21T04:15:23.815 INFO:tasks.workunit.client.1.smithi148.stdout:3/718: chown d3/d5/d6/d2b/da2 23466 1 2024-09-21T04:15:23.815 INFO:tasks.workunit.client.1.smithi148.stdout:3/719: fsync d3/d5/d6/d12/d43/fbb 0 2024-09-21T04:15:23.815 INFO:tasks.workunit.client.1.smithi148.stdout:1/783: mknod db/d46/d4c/da3/c102 0 2024-09-21T04:15:23.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/614: creat d0/d12/d2a/d80/da2/db0/fd8 x:0 0 0 2024-09-21T04:15:23.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/615: chown d0/d22/l2e 25 1 2024-09-21T04:15:23.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/616: getdents d0/d12/d2a/db4 0 2024-09-21T04:15:23.817 INFO:tasks.workunit.client.0.smithi110.stdout:3/617: chown d0/d51/d7b 149768 1 2024-09-21T04:15:23.818 INFO:tasks.workunit.client.1.smithi148.stdout:2/903: dwrite d0/d9/d16/d55/ffd [0,4194304] 0 2024-09-21T04:15:23.818 INFO:tasks.workunit.client.1.smithi148.stdout:0/800: dread d5/d6/d2a/d2b/f2c [0,4194304] 0 2024-09-21T04:15:23.818 INFO:tasks.workunit.client.1.smithi148.stdout:2/904: chown d0/d9/f29 191 1 2024-09-21T04:15:23.818 INFO:tasks.workunit.client.0.smithi110.stdout:0/973: symlink d4/dd/d53/df4/l127 0 2024-09-21T04:15:23.818 INFO:tasks.workunit.client.0.smithi110.stdout:0/974: fdatasync f2 0 2024-09-21T04:15:23.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/871: mkdir d4/d12/d83/d103/d113 0 2024-09-21T04:15:23.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/872: readlink d4/d87/dc1/ld3 0 2024-09-21T04:15:23.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/873: write d4/d87/f5c [514052,43265] 0 2024-09-21T04:15:23.821 INFO:tasks.workunit.client.0.smithi110.stdout:3/618: mkdir d0/d12/d16/d1a/d6f/dd9 0 2024-09-21T04:15:23.823 INFO:tasks.workunit.client.1.smithi148.stdout:6/837: creat d0/d1a/d23/d27/d62/dc4/f101 x:0 0 0 2024-09-21T04:15:23.826 INFO:tasks.workunit.client.1.smithi148.stdout:6/838: fsync d0/d34/d38/f3a 0 2024-09-21T04:15:23.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/784: dread db/d24/d2e/d59/da7/fd8 [0,4194304] 0 2024-09-21T04:15:23.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/785: creat db/d24/d2e/d59/da7/f103 x:0 0 0 2024-09-21T04:15:23.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/720: rmdir d3/d1f/d4d/d82/d52 39 2024-09-21T04:15:23.827 INFO:tasks.workunit.client.1.smithi148.stdout:3/721: read d3/d5/d6/d12/d43/d54/fc6 [73541,116134] 0 2024-09-21T04:15:23.827 INFO:tasks.workunit.client.0.smithi110.stdout:0/975: link d4/d47/d2c/c65 d4/d1c/d101/c128 0 2024-09-21T04:15:23.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/619: symlink d0/d22/d29/d76/lda 0 2024-09-21T04:15:23.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/620: read - d0/d12/d3c/dad/fb7 zero size 2024-09-21T04:15:23.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/621: chown d0/d12/d3c/l9e 2467065 1 2024-09-21T04:15:23.830 INFO:tasks.workunit.client.0.smithi110.stdout:0/976: symlink d4/d1c/d62/d91/dd6/d114/l129 0 2024-09-21T04:15:23.834 INFO:tasks.workunit.client.0.smithi110.stdout:2/567: dwrite d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:23.835 INFO:tasks.workunit.client.1.smithi148.stdout:4/874: dread d4/d12/f41 [0,4194304] 0 2024-09-21T04:15:23.837 INFO:tasks.workunit.client.0.smithi110.stdout:2/568: getdents d2/d1a 0 2024-09-21T04:15:23.840 INFO:tasks.workunit.client.0.smithi110.stdout:3/622: write d0/d12/d3c/d5a/d70/f97 [1072240,44755] 0 2024-09-21T04:15:23.840 INFO:tasks.workunit.client.0.smithi110.stdout:9/609: sync 2024-09-21T04:15:23.840 INFO:tasks.workunit.client.0.smithi110.stdout:9/610: write d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a [1662514,16685] 0 2024-09-21T04:15:23.843 INFO:tasks.workunit.client.1.smithi148.stdout:6/839: symlink d0/d1a/d23/d27/dc2/l102 0 2024-09-21T04:15:23.844 INFO:tasks.workunit.client.0.smithi110.stdout:9/611: creat d7/d96/d99/d3c/d11/fcf x:0 0 0 2024-09-21T04:15:23.844 INFO:tasks.workunit.client.0.smithi110.stdout:0/977: write d4/fb [1299511,14227] 0 2024-09-21T04:15:23.844 INFO:tasks.workunit.client.0.smithi110.stdout:0/978: read d4/d47/f40 [114033,79120] 0 2024-09-21T04:15:23.848 INFO:tasks.workunit.client.1.smithi148.stdout:1/786: mkdir db/d24/d2e/d71/d94/de4/d104 0 2024-09-21T04:15:23.852 INFO:tasks.workunit.client.0.smithi110.stdout:0/979: unlink d4/d1c/d62/l123 0 2024-09-21T04:15:23.852 INFO:tasks.workunit.client.0.smithi110.stdout:0/980: creat d4/dd/d10/d29/f12a x:0 0 0 2024-09-21T04:15:23.852 INFO:tasks.workunit.client.0.smithi110.stdout:0/981: truncate d4/dd/fcb 4332017 0 2024-09-21T04:15:23.853 INFO:tasks.workunit.client.0.smithi110.stdout:5/705: dwrite d1/d49/d4d/d1e/f81 [0,4194304] 0 2024-09-21T04:15:23.854 INFO:tasks.workunit.client.0.smithi110.stdout:5/706: write d1/d22/db6/fdd [739863,77881] 0 2024-09-21T04:15:23.854 INFO:tasks.workunit.client.0.smithi110.stdout:9/612: write d7/d96/d99/d3c/d1f/d43/f6c [4051262,93358] 0 2024-09-21T04:15:23.854 INFO:tasks.workunit.client.1.smithi148.stdout:4/875: creat d4/d12/d16/d29/f114 x:0 0 0 2024-09-21T04:15:23.854 INFO:tasks.workunit.client.1.smithi148.stdout:4/876: write d4/d12/d16/d29/d61/d6c/d9d/fa2 [195232,126699] 0 2024-09-21T04:15:23.858 INFO:tasks.workunit.client.0.smithi110.stdout:2/569: dread d2/d1a/d1f/d5f/d9d/f78 [0,4194304] 0 2024-09-21T04:15:23.858 INFO:tasks.workunit.client.0.smithi110.stdout:2/570: write d2/d50/d56/f7e [219427,117726] 0 2024-09-21T04:15:23.858 INFO:tasks.workunit.client.0.smithi110.stdout:2/571: readlink d2/d1a/d1f/d5f/l55 0 2024-09-21T04:15:23.861 INFO:tasks.workunit.client.0.smithi110.stdout:9/613: mkdir d7/d96/d99/d3c/d1f/d43/db5/dbf/dd0 0 2024-09-21T04:15:23.863 INFO:tasks.workunit.client.0.smithi110.stdout:2/572: creat d2/d1a/d1f/d5f/d9d/d86/fae x:0 0 0 2024-09-21T04:15:23.865 INFO:tasks.workunit.client.0.smithi110.stdout:9/614: rename d7/fa4 to d7/d10/d88/d8f/fd1 0 2024-09-21T04:15:23.865 INFO:tasks.workunit.client.0.smithi110.stdout:9/615: chown d7/d10/d88/d8f/ccc 104 1 2024-09-21T04:15:23.865 INFO:tasks.workunit.client.0.smithi110.stdout:9/616: stat d7/d96/d99/d3c/d1f/d72/c69 0 2024-09-21T04:15:23.865 INFO:tasks.workunit.client.0.smithi110.stdout:9/617: chown d7/d96/d99/d3c/l4b 45 1 2024-09-21T04:15:23.866 INFO:tasks.workunit.client.0.smithi110.stdout:9/618: write d7/d10/d88/d8f/fd1 [1365627,97327] 0 2024-09-21T04:15:23.866 INFO:tasks.workunit.client.0.smithi110.stdout:9/619: read d7/d96/d99/f38 [97299,50686] 0 2024-09-21T04:15:23.866 INFO:tasks.workunit.client.0.smithi110.stdout:9/620: chown d7/d96/d99/d3c/d14/d31/la3 3641003 1 2024-09-21T04:15:23.866 INFO:tasks.workunit.client.0.smithi110.stdout:9/621: fsync d7/d96/d99/d3c/d1f/d43/fad 0 2024-09-21T04:15:23.867 INFO:tasks.workunit.client.1.smithi148.stdout:4/877: dread d4/f10 [4194304,4194304] 0 2024-09-21T04:15:23.869 INFO:tasks.workunit.client.1.smithi148.stdout:4/878: link d4/fe d4/d12/d16/f115 0 2024-09-21T04:15:23.870 INFO:tasks.workunit.client.0.smithi110.stdout:0/982: write d4/d47/f23 [467524,98176] 0 2024-09-21T04:15:23.870 INFO:tasks.workunit.client.0.smithi110.stdout:0/983: fsync d4/dd/d10/f50 0 2024-09-21T04:15:23.871 INFO:tasks.workunit.client.1.smithi148.stdout:7/762: dwrite d5/d14/f9f [0,4194304] 0 2024-09-21T04:15:23.871 INFO:tasks.workunit.client.1.smithi148.stdout:4/879: mkdir d4/d66/d7f/d116 0 2024-09-21T04:15:23.871 INFO:tasks.workunit.client.1.smithi148.stdout:7/763: dread - d5/d14/ff2 zero size 2024-09-21T04:15:23.872 INFO:tasks.workunit.client.1.smithi148.stdout:4/880: truncate d4/d87/d91/f104 120779 0 2024-09-21T04:15:23.872 INFO:tasks.workunit.client.1.smithi148.stdout:7/764: creat d5/d14/d25/d40/db9/f10e x:0 0 0 2024-09-21T04:15:23.872 INFO:tasks.workunit.client.1.smithi148.stdout:7/765: creat d5/d14/d25/d40/d7b/f10f x:0 0 0 2024-09-21T04:15:23.872 INFO:tasks.workunit.client.1.smithi148.stdout:0/801: dwrite d5/d6/d2a/dcd/df8/da5/fbf [0,4194304] 0 2024-09-21T04:15:23.872 INFO:tasks.workunit.client.1.smithi148.stdout:0/802: fdatasync d5/d6/d2a/dcd/df8/da5/f95 0 2024-09-21T04:15:23.873 INFO:tasks.workunit.client.1.smithi148.stdout:3/722: dwrite d3/d5/d6/d12/d15/d73/f9a [0,4194304] 0 2024-09-21T04:15:23.873 INFO:tasks.workunit.client.1.smithi148.stdout:4/881: write d4/d12/d16/d29/f46 [5033801,68731] 0 2024-09-21T04:15:23.873 INFO:tasks.workunit.client.0.smithi110.stdout:3/623: dwrite d0/d12/d3c/d5a/d70/fbf [0,4194304] 0 2024-09-21T04:15:23.873 INFO:tasks.workunit.client.0.smithi110.stdout:5/707: dread d1/d7/fa1 [0,4194304] 0 2024-09-21T04:15:23.874 INFO:tasks.workunit.client.0.smithi110.stdout:0/984: getdents d4/dd/d53 0 2024-09-21T04:15:23.874 INFO:tasks.workunit.client.0.smithi110.stdout:0/985: creat d4/dd/d10/d29/d45/f12b x:0 0 0 2024-09-21T04:15:23.876 INFO:tasks.workunit.client.1.smithi148.stdout:6/840: dwrite d0/d8/dba/fe4 [0,4194304] 0 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:0/986: dread d4/dd/d10/d29/f5d [0,4194304] 0 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:0/987: write d4/dd/d10/f2a [1893476,24991] 0 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:9/622: dread d7/d10/f3f [0,4194304] 0 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:9/623: chown d7/d96/d99/d3c/d4f/d9f/d6f/l70 26171 1 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:9/624: dread - d7/d96/d99/d3c/d4f/d9f/d6f/f77 zero size 2024-09-21T04:15:23.886 INFO:tasks.workunit.client.0.smithi110.stdout:9/625: fsync d7/d96/d99/f52 0 2024-09-21T04:15:23.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/766: mknod d5/d8/d6f/d7d/df5/c110 0 2024-09-21T04:15:23.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/767: readlink d5/d14/d25/d40/d7b/l97 0 2024-09-21T04:15:23.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/768: truncate d5/d8/f9 1439752 0 2024-09-21T04:15:23.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/769: write d5/d14/d25/d79/d86/fd9 [198272,10896] 0 2024-09-21T04:15:23.892 INFO:tasks.workunit.client.0.smithi110.stdout:9/626: write d7/d96/d99/d3c/d35/f9c [839041,25036] 0 2024-09-21T04:15:23.892 INFO:tasks.workunit.client.0.smithi110.stdout:9/627: write d7/d96/f9a [1022113,9058] 0 2024-09-21T04:15:23.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/803: rmdir d5/d6/d2a/dcd/df8/db4 39 2024-09-21T04:15:23.897 INFO:tasks.workunit.client.0.smithi110.stdout:0/988: mknod d4/d1c/d62/d69/c12c 0 2024-09-21T04:15:23.898 INFO:tasks.workunit.client.1.smithi148.stdout:1/787: rmdir db/d24/d2e/d59/da7 39 2024-09-21T04:15:23.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/723: symlink d3/d5/d6/d12/d15/leb 0 2024-09-21T04:15:23.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/724: write d3/d1f/f3e [1209191,12711] 0 2024-09-21T04:15:23.903 INFO:tasks.workunit.client.1.smithi148.stdout:4/882: rmdir d4/d87/d2d/dd0 39 2024-09-21T04:15:23.906 INFO:tasks.workunit.client.1.smithi148.stdout:6/841: dwrite d0/d1a/d23/d32/d93/fa8 [0,4194304] 0 2024-09-21T04:15:23.907 INFO:tasks.workunit.client.0.smithi110.stdout:2/573: dwrite d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:23.907 INFO:tasks.workunit.client.0.smithi110.stdout:5/708: dwrite d1/d7/d18/f6d [0,4194304] 0 2024-09-21T04:15:23.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/905: dwrite d0/d9/dc/f12 [0,4194304] 0 2024-09-21T04:15:23.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/906: stat d0/d9/d16/d55/d71 0 2024-09-21T04:15:23.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/907: chown d0/d2b/da3/da1/c3f 1627941 1 2024-09-21T04:15:23.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/908: fsync d0/d2b/fbb 0 2024-09-21T04:15:23.909 INFO:tasks.workunit.client.1.smithi148.stdout:7/770: unlink d5/d8/d6f/d7d/d90/l73 0 2024-09-21T04:15:23.909 INFO:tasks.workunit.client.1.smithi148.stdout:7/771: fdatasync d5/d8/d6f/d7d/d90/fce 0 2024-09-21T04:15:23.909 INFO:tasks.workunit.client.1.smithi148.stdout:7/772: chown d5/d14/d25/d40/d8b 27 1 2024-09-21T04:15:23.909 INFO:tasks.workunit.client.1.smithi148.stdout:7/773: stat d5/d8/d6f/cb6 0 2024-09-21T04:15:23.911 INFO:tasks.workunit.client.1.smithi148.stdout:0/804: symlink d5/d6/d2a/d45/d70/lfb 0 2024-09-21T04:15:23.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/725: creat d3/d1f/d4d/d82/d23/fec x:0 0 0 2024-09-21T04:15:23.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/883: creat d4/d87/d2d/f117 x:0 0 0 2024-09-21T04:15:23.915 INFO:tasks.workunit.client.0.smithi110.stdout:0/989: dread d4/dd/d53/fa2 [0,4194304] 0 2024-09-21T04:15:23.916 INFO:tasks.workunit.client.0.smithi110.stdout:3/624: dwrite d0/f10 [4194304,4194304] 0 2024-09-21T04:15:23.916 INFO:tasks.workunit.client.0.smithi110.stdout:0/990: fdatasync d4/dd/d10/d29/fe4 0 2024-09-21T04:15:23.916 INFO:tasks.workunit.client.0.smithi110.stdout:0/991: chown d4/d47/d2c/d3a/f43 145711 1 2024-09-21T04:15:23.916 INFO:tasks.workunit.client.0.smithi110.stdout:0/992: fdatasync d4/f56 0 2024-09-21T04:15:23.919 INFO:tasks.workunit.client.1.smithi148.stdout:6/842: mknod d0/d1a/d23/dc/d81/c103 0 2024-09-21T04:15:23.922 INFO:tasks.workunit.client.1.smithi148.stdout:4/884: dread d4/d12/f41 [4194304,4194304] 0 2024-09-21T04:15:23.923 INFO:tasks.workunit.client.0.smithi110.stdout:9/628: symlink d7/d96/d99/d3c/d1f/ld2 0 2024-09-21T04:15:23.923 INFO:tasks.workunit.client.0.smithi110.stdout:9/629: chown d7/d96/f9a 8028 1 2024-09-21T04:15:23.924 INFO:tasks.workunit.client.0.smithi110.stdout:2/574: symlink d2/d1a/d90/laf 0 2024-09-21T04:15:23.924 INFO:tasks.workunit.client.0.smithi110.stdout:2/575: write d2/d1a/d1f/d5f/f1d [2711885,4150] 0 2024-09-21T04:15:23.925 INFO:tasks.workunit.client.1.smithi148.stdout:2/909: dwrite d0/d9/dc/ddf/f118 [0,4194304] 0 2024-09-21T04:15:23.926 INFO:tasks.workunit.client.1.smithi148.stdout:0/805: symlink d5/d6/d2a/dcd/df8/daf/dc8/de8/df7/lfc 0 2024-09-21T04:15:23.929 INFO:tasks.workunit.client.1.smithi148.stdout:1/788: rename db/dd/db9/dc2/ldf to db/dd/l105 0 2024-09-21T04:15:23.930 INFO:tasks.workunit.client.1.smithi148.stdout:6/843: read d0/d1a/db8/dbe/fc3 [3351011,82728] 0 2024-09-21T04:15:23.931 INFO:tasks.workunit.client.1.smithi148.stdout:3/726: mknod d3/d5/d6/d2b/d96/db8/ced 0 2024-09-21T04:15:23.931 INFO:tasks.workunit.client.1.smithi148.stdout:3/727: creat d3/d5/d5c/fee x:0 0 0 2024-09-21T04:15:23.933 INFO:tasks.workunit.client.0.smithi110.stdout:2/576: dread d2/d1a/d1f/d29/f4f [0,4194304] 0 2024-09-21T04:15:23.933 INFO:tasks.workunit.client.0.smithi110.stdout:2/577: getdents d2/d1a/d1f/d2a/da1 0 2024-09-21T04:15:23.933 INFO:tasks.workunit.client.0.smithi110.stdout:5/709: creat d1/d49/d4d/d1e/d4a/dd4/dd5/fec x:0 0 0 2024-09-21T04:15:23.935 INFO:tasks.workunit.client.1.smithi148.stdout:4/885: mkdir d4/d66/d7f/d2a/d118 0 2024-09-21T04:15:23.941 INFO:tasks.workunit.client.1.smithi148.stdout:3/728: write d3/d1f/d4d/d82/d23/daf/fd3 [1174375,49822] 0 2024-09-21T04:15:23.941 INFO:tasks.workunit.client.1.smithi148.stdout:3/729: chown d3/d5/d6/d12/d43/c6d 1885 1 2024-09-21T04:15:23.944 INFO:tasks.workunit.client.0.smithi110.stdout:0/993: link d4/f5c d4/dd/d10/d29/f12d 0 2024-09-21T04:15:23.945 INFO:tasks.workunit.client.0.smithi110.stdout:0/994: read d4/d47/fab [455358,73646] 0 2024-09-21T04:15:23.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/630: symlink d7/d96/d99/d3c/ld3 0 2024-09-21T04:15:23.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/631: chown d7/d96/d99/d3c/d1f/d43 16 1 2024-09-21T04:15:23.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/632: truncate d7/d10/d88/d8f/fd1 2270423 0 2024-09-21T04:15:23.946 INFO:tasks.workunit.client.0.smithi110.stdout:0/995: read d4/dd/d10/d29/d75/fa9 [274311,38876] 0 2024-09-21T04:15:23.947 INFO:tasks.workunit.client.1.smithi148.stdout:7/774: rename d5/d8/d6f/dcd/df8 to d5/d14/d25/d40/dab/dad/d111 0 2024-09-21T04:15:23.950 INFO:tasks.workunit.client.1.smithi148.stdout:0/806: dwrite d5/d6/d7/d22/fa2 [0,4194304] 0 2024-09-21T04:15:23.950 INFO:tasks.workunit.client.1.smithi148.stdout:0/807: chown d5/d6/d2a/d2b/c53 399995 1 2024-09-21T04:15:23.951 INFO:tasks.workunit.client.1.smithi148.stdout:1/789: mkdir db/d46/d8f/d106 0 2024-09-21T04:15:23.955 INFO:tasks.workunit.client.1.smithi148.stdout:6/844: symlink d0/d1a/d23/d53/l104 0 2024-09-21T04:15:23.956 INFO:tasks.workunit.client.0.smithi110.stdout:9/633: dread d7/d96/d99/d3c/d35/f5e [0,4194304] 0 2024-09-21T04:15:23.956 INFO:tasks.workunit.client.0.smithi110.stdout:9/634: readlink d7/d96/d99/d3c/d1f/ld2 0 2024-09-21T04:15:23.958 INFO:tasks.workunit.client.1.smithi148.stdout:4/886: unlink d4/d87/d2d/fc3 0 2024-09-21T04:15:23.965 INFO:tasks.workunit.client.0.smithi110.stdout:5/710: mkdir d1/dc/ded 0 2024-09-21T04:15:23.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/625: rmdir d0/d12/d16/d1a/d95 39 2024-09-21T04:15:23.968 INFO:tasks.workunit.client.1.smithi148.stdout:9/812: sync 2024-09-21T04:15:23.968 INFO:tasks.workunit.client.1.smithi148.stdout:5/773: sync 2024-09-21T04:15:23.968 INFO:tasks.workunit.client.1.smithi148.stdout:8/715: sync 2024-09-21T04:15:23.969 INFO:tasks.workunit.client.1.smithi148.stdout:0/808: mknod d5/d6/d2a/dcd/df8/db4/cfd 0 2024-09-21T04:15:23.969 INFO:tasks.workunit.client.1.smithi148.stdout:7/775: dread d5/d8/d6f/d7d/d90/d28/f9c [0,4194304] 0 2024-09-21T04:15:23.971 INFO:tasks.workunit.client.1.smithi148.stdout:1/790: unlink db/dd/d21/fc6 0 2024-09-21T04:15:23.971 INFO:tasks.workunit.client.1.smithi148.stdout:1/791: write db/d24/d2e/d71/d94/dde/ffb [501062,64701] 0 2024-09-21T04:15:23.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/635: mknod d7/d96/d99/d3c/d1f/d43/db5/cd4 0 2024-09-21T04:15:23.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/636: write d7/d26/f7e [362985,6385] 0 2024-09-21T04:15:23.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/637: fsync d7/d96/d99/d3c/d1f/d43/f6c 0 2024-09-21T04:15:23.973 INFO:tasks.workunit.client.1.smithi148.stdout:6/845: mknod d0/d1a/d23/dc/d68/c105 0 2024-09-21T04:15:23.974 INFO:tasks.workunit.client.0.smithi110.stdout:5/711: rename d1/d39/f63 to d1/dc/de9/fee 0 2024-09-21T04:15:23.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/774: unlink d3/d14/d1f/d26/d2e/d3d/da0/fcc 0 2024-09-21T04:15:23.976 INFO:tasks.workunit.client.0.smithi110.stdout:3/626: mknod d0/d12/d16/d1a/dac/cdb 0 2024-09-21T04:15:23.977 INFO:tasks.workunit.client.1.smithi148.stdout:8/716: symlink dc/d3e/ldf 0 2024-09-21T04:15:23.977 INFO:tasks.workunit.client.0.smithi110.stdout:0/996: getdents d4/dd/d53/d7b 0 2024-09-21T04:15:23.978 INFO:tasks.workunit.client.1.smithi148.stdout:9/813: write d5/d60/d28/f6a [2935422,91719] 0 2024-09-21T04:15:23.978 INFO:tasks.workunit.client.1.smithi148.stdout:8/717: write dc/d13/d3c/d81/d85/da1/fca [351404,114979] 0 2024-09-21T04:15:23.979 INFO:tasks.workunit.client.1.smithi148.stdout:0/809: creat d5/d6/d2a/d3f/d8b/dab/ffe x:0 0 0 2024-09-21T04:15:23.981 INFO:tasks.workunit.client.1.smithi148.stdout:4/887: dwrite d4/d12/d75/d9c/fde [0,4194304] 0 2024-09-21T04:15:23.982 INFO:tasks.workunit.client.1.smithi148.stdout:1/792: write db/dd/f8c [2074372,88996] 0 2024-09-21T04:15:23.982 INFO:tasks.workunit.client.0.smithi110.stdout:5/712: mknod d1/d49/d4d/d1e/d4a/dd4/cef 0 2024-09-21T04:15:23.984 INFO:tasks.workunit.client.0.smithi110.stdout:5/713: dread d1/d22/db6/fdd [0,4194304] 0 2024-09-21T04:15:23.984 INFO:tasks.workunit.client.1.smithi148.stdout:5/775: creat d3/d14/d1f/d26/d2e/ddb/dde/f106 x:0 0 0 2024-09-21T04:15:23.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/627: creat d0/d12/d3c/dce/fdc x:0 0 0 2024-09-21T04:15:23.987 INFO:tasks.workunit.client.0.smithi110.stdout:0/997: rmdir d4/dd/d10/d29/d11f 0 2024-09-21T04:15:23.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/814: getdents d5/d1f/d81/dcf 0 2024-09-21T04:15:23.988 INFO:tasks.workunit.client.1.smithi148.stdout:9/815: write d5/d39/dba/f11d [378292,105359] 0 2024-09-21T04:15:23.989 INFO:tasks.workunit.client.1.smithi148.stdout:8/718: rename dc/d13/d3c/f4b to dc/d13/fe0 0 2024-09-21T04:15:23.990 INFO:tasks.workunit.client.1.smithi148.stdout:8/719: truncate dc/d13/dbd/fde 512701 0 2024-09-21T04:15:23.990 INFO:tasks.workunit.client.0.smithi110.stdout:0/998: truncate d4/dd/d10/fa1 148066 0 2024-09-21T04:15:23.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/810: getdents d5/d6/d2a/d2b 0 2024-09-21T04:15:23.993 INFO:tasks.workunit.client.0.smithi110.stdout:5/714: rename d1/d49/d4d/d1e/d5a/d64/f80 to d1/d7/ff0 0 2024-09-21T04:15:23.994 INFO:tasks.workunit.client.0.smithi110.stdout:9/638: write d7/d96/d99/d3c/d35/f76 [2001275,14022] 0 2024-09-21T04:15:23.994 INFO:tasks.workunit.client.0.smithi110.stdout:9/639: write d7/d10/f12 [759177,101449] 0 2024-09-21T04:15:23.994 INFO:tasks.workunit.client.0.smithi110.stdout:9/640: truncate d7/d10/d88/d8f/da0/fc8 253887 0 2024-09-21T04:15:23.994 INFO:tasks.workunit.client.0.smithi110.stdout:0/999: symlink d4/dd/d10/dee/l12e 0 2024-09-21T04:15:23.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/776: mknod d5/d14/d25/d79/d86/c112 0 2024-09-21T04:15:23.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/777: dread - d5/d8/d6f/d7d/df5/ffb zero size 2024-09-21T04:15:23.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/778: creat d5/d14/d25/d40/d8b/f113 x:0 0 0 2024-09-21T04:15:23.999 INFO:tasks.workunit.client.1.smithi148.stdout:1/793: mkdir db/d24/d107 0 2024-09-21T04:15:23.999 INFO:tasks.workunit.client.1.smithi148.stdout:1/794: dread - db/d24/d2e/d71/d94/ff3 zero size 2024-09-21T04:15:23.999 INFO:tasks.workunit.client.0.smithi110.stdout:3/628: dread d0/d12/d2a/f31 [0,4194304] 0 2024-09-21T04:15:24.000 INFO:tasks.workunit.client.1.smithi148.stdout:5/776: rmdir d3/d14/d1f/d26/d2e/ddb/deb 39 2024-09-21T04:15:24.000 INFO:tasks.workunit.client.1.smithi148.stdout:5/777: chown d3/d14/d1f/d26/d2e/d3d 20824 1 2024-09-21T04:15:24.000 INFO:tasks.workunit.client.1.smithi148.stdout:5/778: write d3/d34/d78/dbf/f4b [4777925,88466] 0 2024-09-21T04:15:24.001 INFO:tasks.workunit.client.1.smithi148.stdout:6/846: dwrite d0/d34/d38/d43/f98 [0,4194304] 0 2024-09-21T04:15:24.001 INFO:tasks.workunit.client.1.smithi148.stdout:6/847: write d0/d1a/d23/d27/fe1 [892171,102014] 0 2024-09-21T04:15:24.005 INFO:tasks.workunit.client.1.smithi148.stdout:9/816: symlink d5/d60/d28/d2f/dbf/dc2/l122 0 2024-09-21T04:15:24.006 INFO:tasks.workunit.client.1.smithi148.stdout:1/795: mknod db/d46/d8f/c108 0 2024-09-21T04:15:24.009 INFO:tasks.workunit.client.1.smithi148.stdout:0/811: dread d5/d6/d7/d22/fa2 [0,4194304] 0 2024-09-21T04:15:24.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/817: mkdir d5/d2b/d32/d103/d123 0 2024-09-21T04:15:24.011 INFO:tasks.workunit.client.1.smithi148.stdout:8/720: dread dc/d13/d3c/d81/d85/da1/fca [0,4194304] 0 2024-09-21T04:15:24.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/818: truncate d5/f96 603638 0 2024-09-21T04:15:24.011 INFO:tasks.workunit.client.1.smithi148.stdout:9/819: stat d5/d1f/d81/dcf/d106/c78 0 2024-09-21T04:15:24.011 INFO:tasks.workunit.client.1.smithi148.stdout:7/779: dread d5/d8/f1e [4194304,4194304] 0 2024-09-21T04:15:24.013 INFO:tasks.workunit.client.1.smithi148.stdout:9/820: mknod d5/d60/d28/d2f/d10a/c124 0 2024-09-21T04:15:24.015 INFO:tasks.workunit.client.1.smithi148.stdout:9/821: mknod d5/d60/d28/d73/d110/c125 0 2024-09-21T04:15:24.016 INFO:tasks.workunit.client.1.smithi148.stdout:4/888: dwrite d4/d87/f5c [0,4194304] 0 2024-09-21T04:15:24.019 INFO:tasks.workunit.client.1.smithi148.stdout:7/780: link d5/d8/cd d5/d14/d25/d40/dab/c114 0 2024-09-21T04:15:24.019 INFO:tasks.workunit.client.1.smithi148.stdout:9/822: symlink d5/d60/l126 0 2024-09-21T04:15:24.020 INFO:tasks.workunit.client.0.smithi110.stdout:3/629: write d0/d12/d3c/d5a/d70/fbf [598581,89344] 0 2024-09-21T04:15:24.020 INFO:tasks.workunit.client.0.smithi110.stdout:3/630: fsync d0/d12/d16/d1a/d5c/f60 0 2024-09-21T04:15:24.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/823: rmdir d5/d60/de1 39 2024-09-21T04:15:24.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/824: readlink d5/d60/d28/l70 0 2024-09-21T04:15:24.024 INFO:tasks.workunit.client.0.smithi110.stdout:3/631: creat d0/d12/d2a/fdd x:0 0 0 2024-09-21T04:15:24.024 INFO:tasks.workunit.client.0.smithi110.stdout:3/632: creat d0/d12/d2a/fde x:0 0 0 2024-09-21T04:15:24.026 INFO:tasks.workunit.client.1.smithi148.stdout:9/825: symlink d5/d2b/dbe/l127 0 2024-09-21T04:15:24.026 INFO:tasks.workunit.client.1.smithi148.stdout:9/826: write d5/f23 [826704,21491] 0 2024-09-21T04:15:24.026 INFO:tasks.workunit.client.1.smithi148.stdout:1/796: dread db/d24/d2e/d71/d94/d9a/f7d [0,4194304] 0 2024-09-21T04:15:24.028 INFO:tasks.workunit.client.0.smithi110.stdout:3/633: unlink d0/d12/d16/d1a/f4e 0 2024-09-21T04:15:24.032 INFO:tasks.workunit.client.1.smithi148.stdout:1/797: mknod db/c109 0 2024-09-21T04:15:24.033 INFO:tasks.workunit.client.0.smithi110.stdout:9/641: dwrite d7/d96/d99/d3c/d1f/d68/f7c [0,4194304] 0 2024-09-21T04:15:24.036 INFO:tasks.workunit.client.0.smithi110.stdout:9/642: rename d7/d96/d99/d3c/d14/la5 to d7/d96/d99/d3c/d4f/ld5 0 2024-09-21T04:15:24.041 INFO:tasks.workunit.client.0.smithi110.stdout:9/643: dread d7/d96/d99/f2a [4194304,4194304] 0 2024-09-21T04:15:24.042 INFO:tasks.workunit.client.1.smithi148.stdout:1/798: read db/d24/d2e/d59/da7/fd8 [1931807,15708] 0 2024-09-21T04:15:24.042 INFO:tasks.workunit.client.1.smithi148.stdout:9/827: dread d5/d1e/d37/d46/f6d [0,4194304] 0 2024-09-21T04:15:24.044 INFO:tasks.workunit.client.1.smithi148.stdout:1/799: mknod db/dd/d21/d89/c10a 0 2024-09-21T04:15:24.044 INFO:tasks.workunit.client.1.smithi148.stdout:1/800: chown db/dd/d21/d89/lce 112120 1 2024-09-21T04:15:24.044 INFO:tasks.workunit.client.1.smithi148.stdout:1/801: chown db/d24/c79 825247740 1 2024-09-21T04:15:24.045 INFO:tasks.workunit.client.1.smithi148.stdout:9/828: rename d5/d1e/d37/c114 to d5/d1f/d81/d9e/da2/c128 0 2024-09-21T04:15:24.046 INFO:tasks.workunit.client.1.smithi148.stdout:9/829: creat d5/d2b/dce/df6/f129 x:0 0 0 2024-09-21T04:15:24.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/830: rename d5/d1e/fea to d5/d1f/d81/f12a 0 2024-09-21T04:15:24.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/831: truncate d5/d1e/d37/d46/f6d 4895642 0 2024-09-21T04:15:24.049 INFO:tasks.workunit.client.1.smithi148.stdout:9/832: readlink d5/d1e/d37/l7c 0 2024-09-21T04:15:24.049 INFO:tasks.workunit.client.1.smithi148.stdout:9/833: chown d5/d2b/dbe/f52 82 1 2024-09-21T04:15:24.050 INFO:tasks.workunit.client.1.smithi148.stdout:1/802: dread db/d24/d2e/d71/d94/d9a/f72 [0,4194304] 0 2024-09-21T04:15:24.051 INFO:tasks.workunit.client.1.smithi148.stdout:1/803: creat db/dd/ddd/f10b x:0 0 0 2024-09-21T04:15:24.051 INFO:tasks.workunit.client.1.smithi148.stdout:1/804: dread - db/d24/d2e/d59/da7/f103 zero size 2024-09-21T04:15:24.051 INFO:tasks.workunit.client.1.smithi148.stdout:1/805: fdatasync db/f58 0 2024-09-21T04:15:24.052 INFO:tasks.workunit.client.0.smithi110.stdout:5/715: dwrite d1/d49/d4d/f1b [0,4194304] 0 2024-09-21T04:15:24.052 INFO:tasks.workunit.client.0.smithi110.stdout:6/733: sync 2024-09-21T04:15:24.052 INFO:tasks.workunit.client.0.smithi110.stdout:1/997: sync 2024-09-21T04:15:24.052 INFO:tasks.workunit.client.0.smithi110.stdout:1/998: chown d2/f94 8 1 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.1.smithi148.stdout:1/806: mkdir db/d24/d2e/d59/da7/dc1/dd2/d10c 0 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.0.smithi110.stdout:8/739: sync 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.0.smithi110.stdout:4/868: sync 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.0.smithi110.stdout:7/687: sync 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.0.smithi110.stdout:7/688: dread - dd/d1c/dad/db2/d56/d66/da6/d98/fb5 zero size 2024-09-21T04:15:24.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/644: dwrite d7/d10/d88/d8f/fd1 [0,4194304] 0 2024-09-21T04:15:24.054 INFO:tasks.workunit.client.0.smithi110.stdout:9/645: chown d7/d96/d99/d3c/d1f 160095554 1 2024-09-21T04:15:24.054 INFO:tasks.workunit.client.0.smithi110.stdout:6/734: link d4/d12/d17/d33/d76/c73 d4/d12/d16/d66/d7a/d4b/d7f/ced 0 2024-09-21T04:15:24.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/834: dread d5/f1b [0,4194304] 0 2024-09-21T04:15:24.055 INFO:tasks.workunit.client.0.smithi110.stdout:1/999: creat d2/df3/dfb/d11c/f142 x:0 0 0 2024-09-21T04:15:24.056 INFO:tasks.workunit.client.1.smithi148.stdout:1/807: truncate db/d24/f8b 1507625 0 2024-09-21T04:15:24.056 INFO:tasks.workunit.client.1.smithi148.stdout:1/808: stat db/d24/f2b 0 2024-09-21T04:15:24.056 INFO:tasks.workunit.client.1.smithi148.stdout:1/809: fdatasync db/ff 0 2024-09-21T04:15:24.056 INFO:tasks.workunit.client.0.smithi110.stdout:4/869: creat d17/d1e/f129 x:0 0 0 2024-09-21T04:15:24.056 INFO:tasks.workunit.client.0.smithi110.stdout:4/870: chown d17/d1e/d77/d10c/l67 1455 1 2024-09-21T04:15:24.057 INFO:tasks.workunit.client.1.smithi148.stdout:9/835: unlink d5/d60/d4c/l57 0 2024-09-21T04:15:24.058 INFO:tasks.workunit.client.0.smithi110.stdout:8/740: getdents d0/d7/d23/d6c 0 2024-09-21T04:15:24.058 INFO:tasks.workunit.client.0.smithi110.stdout:8/741: chown d0/d7/d13/d4b/f7e 66 1 2024-09-21T04:15:24.058 INFO:tasks.workunit.client.0.smithi110.stdout:8/742: chown d0/d7/d13/d46/df8 1 1 2024-09-21T04:15:24.058 INFO:tasks.workunit.client.0.smithi110.stdout:8/743: write d0/d30/f31 [1989181,45454] 0 2024-09-21T04:15:24.059 INFO:tasks.workunit.client.0.smithi110.stdout:8/744: creat d0/d7/d13/f10d x:0 0 0 2024-09-21T04:15:24.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/836: rename d5/d1e/c94 to d5/d1e/d37/d50/c12b 0 2024-09-21T04:15:24.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/837: dread - d5/d1e/d37/d46/fac zero size 2024-09-21T04:15:24.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/838: chown d5/d1f/l59 51 1 2024-09-21T04:15:24.061 INFO:tasks.workunit.client.0.smithi110.stdout:6/735: mknod d4/d12/d17/d33/dab/cee 0 2024-09-21T04:15:24.063 INFO:tasks.workunit.client.1.smithi148.stdout:9/839: rename d5/f16 to d5/d60/d28/d73/d108/f12c 0 2024-09-21T04:15:24.063 INFO:tasks.workunit.client.0.smithi110.stdout:8/745: mknod d0/d7/d13/d46/df8/c10e 0 2024-09-21T04:15:24.064 INFO:tasks.workunit.client.0.smithi110.stdout:9/646: dread d7/d96/d99/d3c/d14/f54 [0,4194304] 0 2024-09-21T04:15:24.065 INFO:tasks.workunit.client.0.smithi110.stdout:6/736: symlink d4/d12/d16/d66/d7a/d83/lef 0 2024-09-21T04:15:24.065 INFO:tasks.workunit.client.0.smithi110.stdout:9/647: symlink d7/d96/d99/d3c/d1f/d43/db5/dbf/ld6 0 2024-09-21T04:15:24.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/737: mknod d4/d12/d17/d33/cf0 0 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/738: dread - d4/d12/d17/d81/d90/fd9 zero size 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/739: chown d4/d12/f2b 1840331 1 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/740: readlink d4/d12/d17/d33/d76/la7 0 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/741: write d4/d12/d16/d6d/f87 [7123174,1655] 0 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:9/648: link d7/d96/d99/d3c/d11/f8c d7/d96/d99/fd7 0 2024-09-21T04:15:24.067 INFO:tasks.workunit.client.0.smithi110.stdout:9/649: creat d7/d10/fd8 x:0 0 0 2024-09-21T04:15:24.068 INFO:tasks.workunit.client.0.smithi110.stdout:6/742: getdents d4/d12/d17/d33/d3e 0 2024-09-21T04:15:24.072 INFO:tasks.workunit.client.0.smithi110.stdout:9/650: link d7/d96/d99/d3c/d1f/d72/l44 d7/d96/d99/d3c/d4f/d9f/d6f/ld9 0 2024-09-21T04:15:24.073 INFO:tasks.workunit.client.1.smithi148.stdout:1/810: read db/d24/d2e/d59/fa2 [3294445,81859] 0 2024-09-21T04:15:24.073 INFO:tasks.workunit.client.1.smithi148.stdout:6/848: dwrite d0/d1a/d23/d27/d62/d9a/fa3 [0,4194304] 0 2024-09-21T04:15:24.074 INFO:tasks.workunit.client.1.smithi148.stdout:1/811: creat db/d24/d2e/d71/d94/dde/f10d x:0 0 0 2024-09-21T04:15:24.074 INFO:tasks.workunit.client.1.smithi148.stdout:1/812: creat db/d24/d2a/f10e x:0 0 0 2024-09-21T04:15:24.074 INFO:tasks.workunit.client.1.smithi148.stdout:1/813: readlink db/dd/d21/db0/le9 0 2024-09-21T04:15:24.074 INFO:tasks.workunit.client.1.smithi148.stdout:1/814: chown db/l18 7263 1 2024-09-21T04:15:24.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/743: mkdir d4/d12/d17/d81/d90/dc3/df1 0 2024-09-21T04:15:24.076 INFO:tasks.workunit.client.0.smithi110.stdout:9/651: rename d7/d96/d99/d3c/d4f/d9f/d6f/l85 to d7/d96/d99/d3c/dae/lda 0 2024-09-21T04:15:24.079 INFO:tasks.workunit.client.0.smithi110.stdout:6/744: link d4/d12/d16/d66/fd5 d4/d12/d16/d66/d7a/d9e/ff2 0 2024-09-21T04:15:24.085 INFO:tasks.workunit.client.1.smithi148.stdout:7/781: dwrite d5/d14/d25/fa2 [0,4194304] 0 2024-09-21T04:15:24.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/721: dwrite dc/d13/d39/d6e/dac/faf [0,4194304] 0 2024-09-21T04:15:24.085 INFO:tasks.workunit.client.1.smithi148.stdout:7/782: write d5/d8/fea [565968,72791] 0 2024-09-21T04:15:24.086 INFO:tasks.workunit.client.1.smithi148.stdout:5/779: dwrite d3/d24/dad/fff [0,4194304] 0 2024-09-21T04:15:24.086 INFO:tasks.workunit.client.1.smithi148.stdout:5/780: chown d3/d14/d1f/d26/d2e/d56/da7/fab 659636 1 2024-09-21T04:15:24.087 INFO:tasks.workunit.client.0.smithi110.stdout:9/652: getdents d7/d10/d88 0 2024-09-21T04:15:24.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/745: dread d4/d12/d16/d66/d7a/d4b/f51 [0,4194304] 0 2024-09-21T04:15:24.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/746: stat d4/d12/d17/d2e/d38 0 2024-09-21T04:15:24.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/747: dread - d4/d78/daa/fd1 zero size 2024-09-21T04:15:24.092 INFO:tasks.workunit.client.0.smithi110.stdout:9/653: mknod d7/d10/cdb 0 2024-09-21T04:15:24.094 INFO:tasks.workunit.client.0.smithi110.stdout:6/748: symlink d4/d12/d17/d81/db6/lf3 0 2024-09-21T04:15:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/654: symlink d7/d96/d99/d3c/d4f/d9f/d6f/ldc 0 2024-09-21T04:15:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/655: chown d7/d96/d99/d3c/d1f/d72/d59/l65 427429 1 2024-09-21T04:15:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/656: write d7/d96/d99/d3c/d11/fcf [537908,12888] 0 2024-09-21T04:15:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/657: truncate d7/d96/d99/d3c/d35/f76 4232552 0 2024-09-21T04:15:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/658: chown d7/d96/d99/f2a 350668218 1 2024-09-21T04:15:24.098 INFO:tasks.workunit.client.1.smithi148.stdout:7/783: chown d5/lb2 1292 1 2024-09-21T04:15:24.098 INFO:tasks.workunit.client.1.smithi148.stdout:0/812: dwrite d5/d6/d2a/d3f/d6d/da3/fda [4194304,4194304] 0 2024-09-21T04:15:24.098 INFO:tasks.workunit.client.0.smithi110.stdout:3/634: dwrite d0/d12/d3c/d5a/d70/d81/fa8 [0,4194304] 0 2024-09-21T04:15:24.099 INFO:tasks.workunit.client.0.smithi110.stdout:6/749: rmdir d4/d12/d17/d33/d3e/dc8 0 2024-09-21T04:15:24.101 INFO:tasks.workunit.client.1.smithi148.stdout:5/781: unlink d3/d14/d1f/d26/d59/cf5 0 2024-09-21T04:15:24.103 INFO:tasks.workunit.client.0.smithi110.stdout:9/659: symlink d7/d96/d99/d3c/d4f/d9f/d6f/ldd 0 2024-09-21T04:15:24.108 INFO:tasks.workunit.client.0.smithi110.stdout:3/635: creat d0/d12/d3c/d5a/dc8/fdf x:0 0 0 2024-09-21T04:15:24.109 INFO:tasks.workunit.client.0.smithi110.stdout:5/716: dwrite d1/f8d [0,4194304] 0 2024-09-21T04:15:24.109 INFO:tasks.workunit.client.0.smithi110.stdout:7/689: dwrite dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:24.112 INFO:tasks.workunit.client.0.smithi110.stdout:3/636: link d0/f56 d0/d12/d16/d1a/dc6/fe0 0 2024-09-21T04:15:24.117 INFO:tasks.workunit.client.1.smithi148.stdout:8/722: dwrite dc/d13/d76/d5d/d78/f9f [0,4194304] 0 2024-09-21T04:15:24.118 INFO:tasks.workunit.client.1.smithi148.stdout:1/815: dwrite db/dd/d21/f93 [0,4194304] 0 2024-09-21T04:15:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/871: dwrite d17/d1e/d77/d10c/d20/d43/d51/d7a/fa7 [4194304,4194304] 0 2024-09-21T04:15:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/872: chown d17/d1e/d77/d10c/d20/l2d 493928656 1 2024-09-21T04:15:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:8/746: dwrite d0/d30/d54/d6b/fc3 [0,4194304] 0 2024-09-21T04:15:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:5/717: mkdir d1/d49/df1 0 2024-09-21T04:15:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:5/718: write d1/d22/f3c [6174595,7311] 0 2024-09-21T04:15:24.131 INFO:tasks.workunit.client.0.smithi110.stdout:8/747: unlink d0/d30/d34/l76 0 2024-09-21T04:15:24.132 INFO:tasks.workunit.client.0.smithi110.stdout:4/873: mknod d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/c12a 0 2024-09-21T04:15:24.132 INFO:tasks.workunit.client.0.smithi110.stdout:4/874: chown d17/d1e/d64/lf0 32108027 1 2024-09-21T04:15:24.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/784: symlink d5/d8/d6f/d7d/d90/d9b/l115 0 2024-09-21T04:15:24.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/785: chown d5/d8/d6f/c88 588971 1 2024-09-21T04:15:24.135 INFO:tasks.workunit.client.1.smithi148.stdout:0/813: symlink d5/lff 0 2024-09-21T04:15:24.135 INFO:tasks.workunit.client.0.smithi110.stdout:6/750: dwrite d4/d12/d17/d81/fcc [0,4194304] 0 2024-09-21T04:15:24.136 INFO:tasks.workunit.client.0.smithi110.stdout:9/660: dwrite d7/d10/d88/fac [0,4194304] 0 2024-09-21T04:15:24.137 INFO:tasks.workunit.client.0.smithi110.stdout:9/661: read d7/d96/f9a [20731,93255] 0 2024-09-21T04:15:24.138 INFO:tasks.workunit.client.0.smithi110.stdout:9/662: creat d7/d10/fde x:0 0 0 2024-09-21T04:15:24.138 INFO:tasks.workunit.client.0.smithi110.stdout:5/719: mkdir d1/d49/d4d/d1e/d4a/dae/df2 0 2024-09-21T04:15:24.139 INFO:tasks.workunit.client.0.smithi110.stdout:5/720: fsync d1/d39/f21 0 2024-09-21T04:15:24.140 INFO:tasks.workunit.client.0.smithi110.stdout:4/875: mkdir d17/d1e/d77/dcf/d12b 0 2024-09-21T04:15:24.141 INFO:tasks.workunit.client.0.smithi110.stdout:3/637: dwrite d0/fcb [0,4194304] 0 2024-09-21T04:15:24.141 INFO:tasks.workunit.client.0.smithi110.stdout:3/638: write d0/d12/d2a/fde [908570,41010] 0 2024-09-21T04:15:24.141 INFO:tasks.workunit.client.0.smithi110.stdout:3/639: creat d0/d12/d3c/dce/fe1 x:0 0 0 2024-09-21T04:15:24.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/814: dread d5/d6/d7/d11/f17 [4194304,4194304] 0 2024-09-21T04:15:24.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/815: write d5/d6/d2a/d3f/d6d/da3/dcc/fdf [428699,11196] 0 2024-09-21T04:15:24.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/816: truncate d5/d6/d7/d11/d7b/fba 115399 0 2024-09-21T04:15:24.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/817: readlink d5/d6/d2a/dcd/df8/daf/dc8/ld5 0 2024-09-21T04:15:24.144 INFO:tasks.workunit.client.0.smithi110.stdout:8/748: dread d0/d7/d13/d1c/f61 [0,4194304] 0 2024-09-21T04:15:24.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/818: dread d5/d6/d2a/d45/d70/f7f [4194304,4194304] 0 2024-09-21T04:15:24.145 INFO:tasks.workunit.client.0.smithi110.stdout:3/640: dread d0/d12/d16/d1a/d5c/f79 [0,4194304] 0 2024-09-21T04:15:24.148 INFO:tasks.workunit.client.0.smithi110.stdout:6/751: creat d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/ff4 x:0 0 0 2024-09-21T04:15:24.151 INFO:tasks.workunit.client.0.smithi110.stdout:5/721: dread d1/d7/f4e [0,4194304] 0 2024-09-21T04:15:24.151 INFO:tasks.workunit.client.0.smithi110.stdout:5/722: creat d1/d49/d4d/d1e/d4a/ff3 x:0 0 0 2024-09-21T04:15:24.152 INFO:tasks.workunit.client.0.smithi110.stdout:3/641: dread d0/d12/d16/d1a/d6f/f73 [0,4194304] 0 2024-09-21T04:15:24.155 INFO:tasks.workunit.client.0.smithi110.stdout:9/663: link d7/d96/d99/d3c/d11/f8c d7/d96/d99/d3c/fdf 0 2024-09-21T04:15:24.157 INFO:tasks.workunit.client.0.smithi110.stdout:4/876: rename d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/ffa to d17/d3e/d6b/def/f12c 0 2024-09-21T04:15:24.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/782: creat d3/d34/d78/f107 x:0 0 0 2024-09-21T04:15:24.165 INFO:tasks.workunit.client.1.smithi148.stdout:8/723: link dc/c11 dc/d13/d28/d62/ce1 0 2024-09-21T04:15:24.165 INFO:tasks.workunit.client.1.smithi148.stdout:8/724: chown dc/d13/d39/cc3 1041208627 1 2024-09-21T04:15:24.166 INFO:tasks.workunit.client.0.smithi110.stdout:8/749: unlink d0/d30/d54/fb0 0 2024-09-21T04:15:24.166 INFO:tasks.workunit.client.0.smithi110.stdout:8/750: write d0/d7/d13/d4b/ddc/f95 [745271,26381] 0 2024-09-21T04:15:24.169 INFO:tasks.workunit.client.1.smithi148.stdout:6/849: rename d0/d1a to d0/d34/d97/d106 0 2024-09-21T04:15:24.173 INFO:tasks.workunit.client.0.smithi110.stdout:5/723: rename d1/d49/d4d/d1e/d4a/dae/lcc to d1/d49/d4d/d1e/d4a/dd4/lf4 0 2024-09-21T04:15:24.174 INFO:tasks.workunit.client.0.smithi110.stdout:5/724: stat d1/d49/d4d/d1e/d5a/d64/d7b/fb5 0 2024-09-21T04:15:24.174 INFO:tasks.workunit.client.0.smithi110.stdout:5/725: write d1/d7/d18/d3f/f86 [935249,109027] 0 2024-09-21T04:15:24.175 INFO:tasks.workunit.client.0.smithi110.stdout:6/752: dwrite d4/d12/d17/d33/d76/fae [0,4194304] 0 2024-09-21T04:15:24.175 INFO:tasks.workunit.client.0.smithi110.stdout:6/753: write d4/d12/d16/d66/d7a/d4b/fa1 [4026258,34062] 0 2024-09-21T04:15:24.175 INFO:tasks.workunit.client.0.smithi110.stdout:6/754: creat d4/d12/d17/d33/d76/ff5 x:0 0 0 2024-09-21T04:15:24.176 INFO:tasks.workunit.client.1.smithi148.stdout:0/819: unlink d5/d6/d2a/d45/d70/f75 0 2024-09-21T04:15:24.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/664: dwrite d7/d96/d99/d3c/d4f/d9f/d6f/f77 [0,4194304] 0 2024-09-21T04:15:24.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/665: fsync d7/d10/fd8 0 2024-09-21T04:15:24.176 INFO:tasks.workunit.client.0.smithi110.stdout:4/877: mkdir d17/d1e/d77/d10c/d20/d43/d51/d52/d98/d12d 0 2024-09-21T04:15:24.177 INFO:tasks.workunit.client.1.smithi148.stdout:5/783: link d3/d34/d78/dbf/fd1 d3/d34/d78/dbf/d101/f108 0 2024-09-21T04:15:24.178 INFO:tasks.workunit.client.1.smithi148.stdout:6/850: write d0/d34/d97/d106/d23/d27/d80/fb2 [768024,98667] 0 2024-09-21T04:15:24.179 INFO:tasks.workunit.client.1.smithi148.stdout:8/725: unlink dc/d13/d3c/f49 0 2024-09-21T04:15:24.180 INFO:tasks.workunit.client.0.smithi110.stdout:8/751: mkdir d0/d7/d89/d10f 0 2024-09-21T04:15:24.181 INFO:tasks.workunit.client.0.smithi110.stdout:5/726: mkdir d1/d49/d69/df5 0 2024-09-21T04:15:24.189 INFO:tasks.workunit.client.0.smithi110.stdout:6/755: creat d4/d12/d16/ff6 x:0 0 0 2024-09-21T04:15:24.189 INFO:tasks.workunit.client.0.smithi110.stdout:5/727: write d1/f4c [2801737,37971] 0 2024-09-21T04:15:24.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/786: rmdir d5/d14 39 2024-09-21T04:15:24.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/787: truncate d5/d8/d6f/d7d/d90/d28/f70 426562 0 2024-09-21T04:15:24.193 INFO:tasks.workunit.client.0.smithi110.stdout:4/878: dwrite d17/d1e/f4c [4194304,4194304] 0 2024-09-21T04:15:24.193 INFO:tasks.workunit.client.0.smithi110.stdout:4/879: read - d17/d34/fb0 zero size 2024-09-21T04:15:24.193 INFO:tasks.workunit.client.0.smithi110.stdout:4/880: fsync d17/d1e/d77/d10c/fa3 0 2024-09-21T04:15:24.193 INFO:tasks.workunit.client.0.smithi110.stdout:4/881: read - d17/d1e/f129 zero size 2024-09-21T04:15:24.193 INFO:tasks.workunit.client.0.smithi110.stdout:4/882: chown d17/d1e/d7b/daf 1255710 1 2024-09-21T04:15:24.194 INFO:tasks.workunit.client.1.smithi148.stdout:0/820: creat d5/d6/d2a/d2b/d99/f100 x:0 0 0 2024-09-21T04:15:24.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/666: symlink d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/le0 0 2024-09-21T04:15:24.197 INFO:tasks.workunit.client.0.smithi110.stdout:5/728: dread d1/d7/d18/f41 [0,4194304] 0 2024-09-21T04:15:24.200 INFO:tasks.workunit.client.0.smithi110.stdout:8/752: symlink d0/d7/d13/d1c/l110 0 2024-09-21T04:15:24.204 INFO:tasks.workunit.client.1.smithi148.stdout:5/784: chown d3/l6d 0 1 2024-09-21T04:15:24.204 INFO:tasks.workunit.client.1.smithi148.stdout:5/785: chown d3/d14/d1f 59957 1 2024-09-21T04:15:24.205 INFO:tasks.workunit.client.0.smithi110.stdout:4/883: rename d17/d1e/d77/d10c/d20/d43/d51/ca0 to d17/d1e/d77/d10c/d57/db9/c12e 0 2024-09-21T04:15:24.206 INFO:tasks.workunit.client.1.smithi148.stdout:0/821: dread d5/d6/d7/f4f [0,4194304] 0 2024-09-21T04:15:24.206 INFO:tasks.workunit.client.0.smithi110.stdout:5/729: read d1/d22/fe3 [891397,45903] 0 2024-09-21T04:15:24.209 INFO:tasks.workunit.client.1.smithi148.stdout:6/851: mkdir d0/d34/d97/d106/d23/d27/ddd/d107 0 2024-09-21T04:15:24.209 INFO:tasks.workunit.client.1.smithi148.stdout:6/852: write d0/d34/d97/d4f/fa7 [2046196,47864] 0 2024-09-21T04:15:24.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/726: link dc/d13/d3c/d81/d85/f87 dc/d3e/db4/fe2 0 2024-09-21T04:15:24.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/727: chown dc/d13/d3c/f5f 767 1 2024-09-21T04:15:24.212 INFO:tasks.workunit.client.0.smithi110.stdout:4/884: dread d17/d3e/f9a [0,4194304] 0 2024-09-21T04:15:24.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/753: dread d0/f3d [0,4194304] 0 2024-09-21T04:15:24.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/754: write d0/d7/f2a [4405775,85509] 0 2024-09-21T04:15:24.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/755: truncate d0/d30/d54/d6b/da8/ff7 229134 0 2024-09-21T04:15:24.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/756: dread - d0/d7/d13/f10d zero size 2024-09-21T04:15:24.213 INFO:tasks.workunit.client.0.smithi110.stdout:5/730: symlink d1/dc/ded/lf6 0 2024-09-21T04:15:24.215 INFO:tasks.workunit.client.0.smithi110.stdout:4/885: truncate f4 1691948 0 2024-09-21T04:15:24.220 INFO:tasks.workunit.client.0.smithi110.stdout:8/757: rename d0/d7/d13/d4b/l25 to d0/d30/d9d/dad/l111 0 2024-09-21T04:15:24.222 INFO:tasks.workunit.client.0.smithi110.stdout:9/667: dwrite d7/d96/d99/d3c/f94 [0,4194304] 0 2024-09-21T04:15:24.227 INFO:tasks.workunit.client.1.smithi148.stdout:7/788: rename d5/d8/d6f/d7d/d90/l69 to d5/d8/d6f/d7d/l116 0 2024-09-21T04:15:24.227 INFO:tasks.workunit.client.0.smithi110.stdout:6/756: dwrite d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:24.228 INFO:tasks.workunit.client.1.smithi148.stdout:8/728: dread dc/d13/d76/f2c [0,4194304] 0 2024-09-21T04:15:24.229 INFO:tasks.workunit.client.1.smithi148.stdout:5/786: symlink d3/d34/d78/dbf/d6f/l109 0 2024-09-21T04:15:24.232 INFO:tasks.workunit.client.0.smithi110.stdout:8/758: mknod d0/d30/d9d/c112 0 2024-09-21T04:15:24.232 INFO:tasks.workunit.client.0.smithi110.stdout:8/759: chown d0/d7/d13/d1c/l53 0 1 2024-09-21T04:15:24.232 INFO:tasks.workunit.client.0.smithi110.stdout:8/760: fsync d0/d7/d13/d1c/f50 0 2024-09-21T04:15:24.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/822: creat d5/d6/d2a/d45/d70/f101 x:0 0 0 2024-09-21T04:15:24.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/823: chown d5/d6/d7/d11 0 1 2024-09-21T04:15:24.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/824: creat d5/d82/df0/f102 x:0 0 0 2024-09-21T04:15:24.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/825: chown d5/d82 155 1 2024-09-21T04:15:24.235 INFO:tasks.workunit.client.1.smithi148.stdout:0/826: write d5/d6/d2a/dcd/df8/daf/fc1 [5130372,93095] 0 2024-09-21T04:15:24.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/853: truncate d0/d34/d97/d106/d23/d27/d62/d9a/f7a 2202480 0 2024-09-21T04:15:24.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/854: readlink d0/d8/dba/lc7 0 2024-09-21T04:15:24.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/855: chown d0/d8/dba 72 1 2024-09-21T04:15:24.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/856: chown d0/d8/dba 260 1 2024-09-21T04:15:24.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/789: rename d5/d8/d64/ce1 to d5/d8/d6f/d7d/d90/c117 0 2024-09-21T04:15:24.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/790: write d5/d8/d6f/d7d/d90/dee/f104 [527360,127093] 0 2024-09-21T04:15:24.241 INFO:tasks.workunit.client.0.smithi110.stdout:9/668: mknod d7/d96/d99/d3c/d4f/d9f/ce1 0 2024-09-21T04:15:24.241 INFO:tasks.workunit.client.0.smithi110.stdout:9/669: creat d7/d10/d88/d8f/fe2 x:0 0 0 2024-09-21T04:15:24.243 INFO:tasks.workunit.client.0.smithi110.stdout:8/761: dread d0/d7/d13/d4b/f67 [0,4194304] 0 2024-09-21T04:15:24.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/757: mknod d4/d12/d17/d2e/d40/dad/cf7 0 2024-09-21T04:15:24.245 INFO:tasks.workunit.client.0.smithi110.stdout:5/731: dwrite d1/d39/f88 [0,4194304] 0 2024-09-21T04:15:24.247 INFO:tasks.workunit.client.0.smithi110.stdout:4/886: dwrite d17/d3e/dbd/fe6 [0,4194304] 0 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:0/827: mknod d5/d6/d2a/dcd/c103 0 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:4/889: sync 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:3/730: sync 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:9/840: sync 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:3/731: truncate d3/d1f/d4d/d82/d23/d28/d65/fbc 354216 0 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:2/910: sync 2024-09-21T04:15:24.248 INFO:tasks.workunit.client.1.smithi148.stdout:1/816: sync 2024-09-21T04:15:24.249 INFO:tasks.workunit.client.0.smithi110.stdout:9/670: link d7/d10/d88/d8f/ccc d7/d96/d99/d3c/d1f/d25/ce3 0 2024-09-21T04:15:24.249 INFO:tasks.workunit.client.0.smithi110.stdout:9/671: chown d7/d96/d99/d3c/d35/f75 192282 1 2024-09-21T04:15:24.249 INFO:tasks.workunit.client.0.smithi110.stdout:8/762: mknod d0/d7/c113 0 2024-09-21T04:15:24.250 INFO:tasks.workunit.client.0.smithi110.stdout:6/758: link d4/d12/f1d d4/d12/d17/d81/d90/ff8 0 2024-09-21T04:15:24.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/791: creat d5/d8/d6f/d7d/d90/dee/f118 x:0 0 0 2024-09-21T04:15:24.252 INFO:tasks.workunit.client.0.smithi110.stdout:5/732: link d1/d7/d18/d3f/caa d1/d22/db6/dc6/cf7 0 2024-09-21T04:15:24.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/787: dwrite d3/d34/d3c/d5d/f93 [4194304,4194304] 0 2024-09-21T04:15:24.253 INFO:tasks.workunit.client.1.smithi148.stdout:3/732: mkdir d3/d5/db4/def 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/911: rename d0/d9/d16/ffb to d0/d2b/da3/d8f/d9f/df5/f120 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/912: stat d0/d2b/da3/d8f/d9f/df5 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/913: fsync d0/d9/d16/fe1 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/914: truncate d0/d9/d16/d9e/f116 475440 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/915: fdatasync d0/d9/f17 0 2024-09-21T04:15:24.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/916: stat d0/ff 0 2024-09-21T04:15:24.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/792: write d5/d14/f10b [70103,79917] 0 2024-09-21T04:15:24.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/793: chown d5/d8/d6f/d7d/d90/l81 14 1 2024-09-21T04:15:24.258 INFO:tasks.workunit.client.0.smithi110.stdout:4/887: mkdir d17/d1e/d12f 0 2024-09-21T04:15:24.261 INFO:tasks.workunit.client.0.smithi110.stdout:9/672: rename d7/d96/d99/d3c/d14/l67 to d7/d96/d99/d3c/d11/le4 0 2024-09-21T04:15:24.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/729: dwrite dc/d13/d76/f57 [0,4194304] 0 2024-09-21T04:15:24.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/730: write dc/f46 [7244848,41526] 0 2024-09-21T04:15:24.263 INFO:tasks.workunit.client.0.smithi110.stdout:8/763: creat d0/d30/d105/f114 x:0 0 0 2024-09-21T04:15:24.264 INFO:tasks.workunit.client.1.smithi148.stdout:5/788: creat d3/d14/d1f/d26/d2e/f10a x:0 0 0 2024-09-21T04:15:24.265 INFO:tasks.workunit.client.1.smithi148.stdout:9/841: dread d5/d2b/d32/f5b [0,4194304] 0 2024-09-21T04:15:24.265 INFO:tasks.workunit.client.1.smithi148.stdout:9/842: chown d5/d60/d69/fbc 81138 1 2024-09-21T04:15:24.266 INFO:tasks.workunit.client.0.smithi110.stdout:6/759: symlink d4/d12/d17/d2e/d40/dad/lf9 0 2024-09-21T04:15:24.266 INFO:tasks.workunit.client.0.smithi110.stdout:6/760: fsync d4/d12/f34 0 2024-09-21T04:15:24.266 INFO:tasks.workunit.client.1.smithi148.stdout:3/733: truncate d3/d5/d6/f17 8366451 0 2024-09-21T04:15:24.266 INFO:tasks.workunit.client.1.smithi148.stdout:3/734: creat d3/d5/db4/ff0 x:0 0 0 2024-09-21T04:15:24.267 INFO:tasks.workunit.client.1.smithi148.stdout:2/917: mknod d0/d9/d16/d55/d6f/c121 0 2024-09-21T04:15:24.267 INFO:tasks.workunit.client.0.smithi110.stdout:4/888: mkdir d17/d1e/d64/d130 0 2024-09-21T04:15:24.267 INFO:tasks.workunit.client.0.smithi110.stdout:4/889: write d17/d3e/ffd [172541,19511] 0 2024-09-21T04:15:24.268 INFO:tasks.workunit.client.1.smithi148.stdout:7/794: dread d5/d8/d6f/d7d/f7e [0,4194304] 0 2024-09-21T04:15:24.269 INFO:tasks.workunit.client.1.smithi148.stdout:7/795: dread - d5/d8/d6f/d7d/d90/fce zero size 2024-09-21T04:15:24.269 INFO:tasks.workunit.client.0.smithi110.stdout:5/733: rename d1/d22/f4f to d1/d7/d18/ff8 0 2024-09-21T04:15:24.271 INFO:tasks.workunit.client.0.smithi110.stdout:8/764: link d0/d7/c19 d0/d30/d34/d3e/c115 0 2024-09-21T04:15:24.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/761: rmdir d4/d86 39 2024-09-21T04:15:24.275 INFO:tasks.workunit.client.0.smithi110.stdout:4/890: dread d17/d1e/d7b/fb8 [0,4194304] 0 2024-09-21T04:15:24.275 INFO:tasks.workunit.client.0.smithi110.stdout:4/891: dread - d17/d3e/d6b/dc7/f120 zero size 2024-09-21T04:15:24.276 INFO:tasks.workunit.client.1.smithi148.stdout:5/789: write d3/d34/d78/dbf/d6f/fc8 [747534,42749] 0 2024-09-21T04:15:24.276 INFO:tasks.workunit.client.1.smithi148.stdout:5/790: chown d3/d37 1636 1 2024-09-21T04:15:24.277 INFO:tasks.workunit.client.0.smithi110.stdout:5/734: unlink d1/d49/d4d/c96 0 2024-09-21T04:15:24.277 INFO:tasks.workunit.client.0.smithi110.stdout:5/735: chown d1/d49/c53 52 1 2024-09-21T04:15:24.280 INFO:tasks.workunit.client.0.smithi110.stdout:4/892: creat d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/f131 x:0 0 0 2024-09-21T04:15:24.280 INFO:tasks.workunit.client.0.smithi110.stdout:4/893: dread - d17/d34/fb0 zero size 2024-09-21T04:15:24.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/890: dwrite d4/d12/d16/d29/dbd/f3e [4194304,4194304] 0 2024-09-21T04:15:24.281 INFO:tasks.workunit.client.1.smithi148.stdout:8/731: link dc/d43/lbf dc/d13/d76/dd8/le3 0 2024-09-21T04:15:24.281 INFO:tasks.workunit.client.1.smithi148.stdout:9/843: unlink d5/d2b/d32/f82 0 2024-09-21T04:15:24.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/894: truncate d17/f32 1984760 0 2024-09-21T04:15:24.283 INFO:tasks.workunit.client.1.smithi148.stdout:3/735: mknod d3/d84/cf1 0 2024-09-21T04:15:24.284 INFO:tasks.workunit.client.1.smithi148.stdout:2/918: mknod d0/d2b/da3/da1/c122 0 2024-09-21T04:15:24.284 INFO:tasks.workunit.client.1.smithi148.stdout:2/919: write d0/d2b/f4b [8184,107112] 0 2024-09-21T04:15:24.285 INFO:tasks.workunit.client.1.smithi148.stdout:7/796: getdents d5/d8/d6f/d7d/d8a 0 2024-09-21T04:15:24.285 INFO:tasks.workunit.client.1.smithi148.stdout:7/797: truncate d5/d14/d25/d40/d7b/f10f 710910 0 2024-09-21T04:15:24.285 INFO:tasks.workunit.client.1.smithi148.stdout:7/798: write d5/d8/f1e [777937,29448] 0 2024-09-21T04:15:24.287 INFO:tasks.workunit.client.1.smithi148.stdout:5/791: mknod d3/d14/d1f/db1/db8/dba/c10b 0 2024-09-21T04:15:24.287 INFO:tasks.workunit.client.1.smithi148.stdout:5/792: fsync d3/d24/dad/fff 0 2024-09-21T04:15:24.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/828: dwrite d5/d82/f87 [0,4194304] 0 2024-09-21T04:15:24.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/829: creat d5/d82/f104 x:0 0 0 2024-09-21T04:15:24.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/736: write d3/d5/f41 [2375921,64336] 0 2024-09-21T04:15:24.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/737: fsync d3/d5/f1e 0 2024-09-21T04:15:24.291 INFO:tasks.workunit.client.1.smithi148.stdout:4/891: symlink d4/d87/l119 0 2024-09-21T04:15:24.291 INFO:tasks.workunit.client.1.smithi148.stdout:4/892: readlink d4/d66/d7f/d2a/da9/lfc 0 2024-09-21T04:15:24.291 INFO:tasks.workunit.client.1.smithi148.stdout:4/893: stat d4/d66/d7f/d9a/dce/f105 0 2024-09-21T04:15:24.292 INFO:tasks.workunit.client.1.smithi148.stdout:8/732: mkdir dc/d13/d28/de4 0 2024-09-21T04:15:24.293 INFO:tasks.workunit.client.1.smithi148.stdout:9/844: rename d5/d2b/dbe/cd4 to d5/d2b/d32/d103/c12d 0 2024-09-21T04:15:24.296 INFO:tasks.workunit.client.1.smithi148.stdout:2/920: mknod d0/d2b/da3/da1/dcd/c123 0 2024-09-21T04:15:24.299 INFO:tasks.workunit.client.1.smithi148.stdout:7/799: chown d5/lde 8748 1 2024-09-21T04:15:24.300 INFO:tasks.workunit.client.0.smithi110.stdout:9/673: dwrite d7/d10/d88/fac [0,4194304] 0 2024-09-21T04:15:24.300 INFO:tasks.workunit.client.0.smithi110.stdout:9/674: fsync d7/d96/d99/d3c/d35/f73 0 2024-09-21T04:15:24.301 INFO:tasks.workunit.client.1.smithi148.stdout:1/817: dwrite db/dd/d21/d89/dc9/fcd [0,4194304] 0 2024-09-21T04:15:24.301 INFO:tasks.workunit.client.1.smithi148.stdout:1/818: write db/d24/d2e/d71/d94/ff3 [490848,65373] 0 2024-09-21T04:15:24.301 INFO:tasks.workunit.client.1.smithi148.stdout:1/819: truncate db/d24/d2e/d71/d94/dde/fea 376800 0 2024-09-21T04:15:24.302 INFO:tasks.workunit.client.0.smithi110.stdout:4/895: write d17/d3e/f9c [762772,117839] 0 2024-09-21T04:15:24.302 INFO:tasks.workunit.client.0.smithi110.stdout:5/736: dwrite d1/d22/d51/f65 [0,4194304] 0 2024-09-21T04:15:24.302 INFO:tasks.workunit.client.0.smithi110.stdout:4/896: getdents d17/d1e/d77/dcf/d12b 0 2024-09-21T04:15:24.303 INFO:tasks.workunit.client.1.smithi148.stdout:3/738: mknod d3/d1f/d4d/d82/cf2 0 2024-09-21T04:15:24.307 INFO:tasks.workunit.client.1.smithi148.stdout:8/733: mkdir dc/d13/d3c/d81/d98/de5 0 2024-09-21T04:15:24.307 INFO:tasks.workunit.client.0.smithi110.stdout:9/675: rename d7/d96/d99/d3c/d1f/d72/c83 to d7/d96/d99/d3c/d4f/d9f/d6f/ce5 0 2024-09-21T04:15:24.309 INFO:tasks.workunit.client.1.smithi148.stdout:9/845: truncate d5/d2b/d4e/db3/ded/ff7 747418 0 2024-09-21T04:15:24.309 INFO:tasks.workunit.client.1.smithi148.stdout:9/846: write d5/d1e/f29 [1380874,41979] 0 2024-09-21T04:15:24.309 INFO:tasks.workunit.client.1.smithi148.stdout:9/847: chown d5/d2b/d32/cb6 256 1 2024-09-21T04:15:24.310 INFO:tasks.workunit.client.1.smithi148.stdout:2/921: creat d0/d2b/da3/da1/f124 x:0 0 0 2024-09-21T04:15:24.310 INFO:tasks.workunit.client.1.smithi148.stdout:2/922: write d0/d2b/f98 [733898,62678] 0 2024-09-21T04:15:24.314 INFO:tasks.workunit.client.1.smithi148.stdout:0/830: dwrite d5/d6/d2a/fe2 [0,4194304] 0 2024-09-21T04:15:24.315 INFO:tasks.workunit.client.1.smithi148.stdout:5/793: dwrite d3/d34/d3c/d5d/f93 [0,4194304] 0 2024-09-21T04:15:24.315 INFO:tasks.workunit.client.1.smithi148.stdout:5/794: creat d3/d34/d78/dbf/d6f/f10c x:0 0 0 2024-09-21T04:15:24.315 INFO:tasks.workunit.client.1.smithi148.stdout:5/795: creat d3/d14/d1a/f10d x:0 0 0 2024-09-21T04:15:24.315 INFO:tasks.workunit.client.1.smithi148.stdout:5/796: creat d3/d34/d78/dbf/f10e x:0 0 0 2024-09-21T04:15:24.318 INFO:tasks.workunit.client.0.smithi110.stdout:4/897: dread d17/d1e/d77/d10c/d57/db9/fc8 [0,4194304] 0 2024-09-21T04:15:24.318 INFO:tasks.workunit.client.0.smithi110.stdout:4/898: chown d17/d1e/d77/d10c/d20/d43/d51/d52/d98/ld7 41331899 1 2024-09-21T04:15:24.319 INFO:tasks.workunit.client.1.smithi148.stdout:1/820: mknod db/dd/d21/db0/c10f 0 2024-09-21T04:15:24.321 INFO:tasks.workunit.client.0.smithi110.stdout:5/737: mkdir d1/d49/de4/df9 0 2024-09-21T04:15:24.322 INFO:tasks.workunit.client.0.smithi110.stdout:9/676: creat d7/d10/d88/d8f/da0/fe6 x:0 0 0 2024-09-21T04:15:24.323 INFO:tasks.workunit.client.0.smithi110.stdout:4/899: rmdir d17/d1e/d77/d10c/d20/d43/d51/d52/d98 39 2024-09-21T04:15:24.324 INFO:tasks.workunit.client.0.smithi110.stdout:4/900: write d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/f131 [285740,30825] 0 2024-09-21T04:15:24.324 INFO:tasks.workunit.client.1.smithi148.stdout:3/739: dwrite d3/d5/f35 [0,4194304] 0 2024-09-21T04:15:24.325 INFO:tasks.workunit.client.1.smithi148.stdout:9/848: symlink d5/d60/d4c/l12e 0 2024-09-21T04:15:24.327 INFO:tasks.workunit.client.0.smithi110.stdout:5/738: rename d1/d49/d4d/d1e/d4a/dd4/dda/fe8 to d1/d22/ffa 0 2024-09-21T04:15:24.329 INFO:tasks.workunit.client.0.smithi110.stdout:9/677: mknod d7/d96/d99/d3c/d1f/d72/ce7 0 2024-09-21T04:15:24.329 INFO:tasks.workunit.client.1.smithi148.stdout:3/740: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:24.331 INFO:tasks.workunit.client.1.smithi148.stdout:2/923: mknod d0/d2b/da3/d8f/d9f/df6/c125 0 2024-09-21T04:15:24.332 INFO:tasks.workunit.client.0.smithi110.stdout:5/739: mkdir d1/d49/d4d/d1e/d4a/dfb 0 2024-09-21T04:15:24.334 INFO:tasks.workunit.client.0.smithi110.stdout:9/678: creat d7/d96/d99/d3c/d4f/d9f/d6f/fe8 x:0 0 0 2024-09-21T04:15:24.337 INFO:tasks.workunit.client.0.smithi110.stdout:9/679: unlink d7/d10/d88/fac 0 2024-09-21T04:15:24.338 INFO:tasks.workunit.client.0.smithi110.stdout:9/680: write d7/d96/d99/d3c/d11/fcf [1462214,101135] 0 2024-09-21T04:15:24.339 INFO:tasks.workunit.client.1.smithi148.stdout:0/831: symlink d5/d6/d7/d22/l105 0 2024-09-21T04:15:24.340 INFO:tasks.workunit.client.0.smithi110.stdout:5/740: symlink d1/d7/lfc 0 2024-09-21T04:15:24.342 INFO:tasks.workunit.client.0.smithi110.stdout:5/741: truncate d1/d7/d18/f74 904000 0 2024-09-21T04:15:24.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/797: rename d3/d34/d78/dbf/dc2 to d3/d4a/d10f 0 2024-09-21T04:15:24.347 INFO:tasks.workunit.client.0.smithi110.stdout:9/681: write d7/d96/d99/d3c/d1f/d43/f6e [2658141,9467] 0 2024-09-21T04:15:24.348 INFO:tasks.workunit.client.0.smithi110.stdout:9/682: fsync d7/d26/f5b 0 2024-09-21T04:15:24.348 INFO:tasks.workunit.client.0.smithi110.stdout:9/683: truncate d7/d96/d99/d3c/d4f/d9f/d6f/d78/f82 914055 0 2024-09-21T04:15:24.348 INFO:tasks.workunit.client.1.smithi148.stdout:8/734: dwrite dc/d43/d4c/d4f/f8a [0,4194304] 0 2024-09-21T04:15:24.348 INFO:tasks.workunit.client.1.smithi148.stdout:3/741: dwrite d3/d1f/d4d/d82/f1d [4194304,4194304] 0 2024-09-21T04:15:24.349 INFO:tasks.workunit.client.1.smithi148.stdout:1/821: unlink db/d46/f97 0 2024-09-21T04:15:24.349 INFO:tasks.workunit.client.1.smithi148.stdout:1/822: stat db/d24/d2e/d59 0 2024-09-21T04:15:24.351 INFO:tasks.workunit.client.0.smithi110.stdout:9/684: mknod d7/d96/d99/d3c/d1f/d68/ce9 0 2024-09-21T04:15:24.351 INFO:tasks.workunit.client.0.smithi110.stdout:9/685: dread - d7/d96/d99/d3c/d1f/d72/d59/fbc zero size 2024-09-21T04:15:24.353 INFO:tasks.workunit.client.1.smithi148.stdout:9/849: creat d5/d60/d28/d73/d108/f12f x:0 0 0 2024-09-21T04:15:24.353 INFO:tasks.workunit.client.1.smithi148.stdout:9/850: stat d5/d1e/f10d 0 2024-09-21T04:15:24.354 INFO:tasks.workunit.client.1.smithi148.stdout:0/832: dread d5/dcb/fe4 [0,4194304] 0 2024-09-21T04:15:24.356 INFO:tasks.workunit.client.0.smithi110.stdout:9/686: creat d7/d96/d99/d3c/d1f/d43/db5/dca/fea x:0 0 0 2024-09-21T04:15:24.356 INFO:tasks.workunit.client.0.smithi110.stdout:2/578: sync 2024-09-21T04:15:24.357 INFO:tasks.workunit.client.1.smithi148.stdout:3/742: write d3/d5/d6/d12/d15/d73/f7c [2004997,71603] 0 2024-09-21T04:15:24.360 INFO:tasks.workunit.client.0.smithi110.stdout:9/687: truncate d7/d10/d88/d8f/fd1 552990 0 2024-09-21T04:15:24.362 INFO:tasks.workunit.client.0.smithi110.stdout:2/579: symlink d2/d1a/d1f/d5f/d9d/d86/d9f/lb0 0 2024-09-21T04:15:24.366 INFO:tasks.workunit.client.0.smithi110.stdout:9/688: rename d7/d96/d99/d3c/d11/laf to d7/d96/d99/d3c/d4f/d9f/d6f/d78/leb 0 2024-09-21T04:15:24.367 INFO:tasks.workunit.client.0.smithi110.stdout:9/689: dread d7/d96/d99/fd7 [0,4194304] 0 2024-09-21T04:15:24.367 INFO:tasks.workunit.client.0.smithi110.stdout:9/690: read d7/d96/d99/d3c/d1f/d43/f6c [757519,35419] 0 2024-09-21T04:15:24.367 INFO:tasks.workunit.client.0.smithi110.stdout:5/742: dwrite d1/d49/d4d/d1e/d4a/dd4/dd5/fec [0,4194304] 0 2024-09-21T04:15:24.368 INFO:tasks.workunit.client.1.smithi148.stdout:2/924: truncate d0/d9/dc/f24 7616644 0 2024-09-21T04:15:24.372 INFO:tasks.workunit.client.1.smithi148.stdout:5/798: dwrite d3/d14/d1a/f33 [0,4194304] 0 2024-09-21T04:15:24.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/735: mkdir dc/de6 0 2024-09-21T04:15:24.374 INFO:tasks.workunit.client.1.smithi148.stdout:7/800: dwrite d5/d8/d6f/f9d [0,4194304] 0 2024-09-21T04:15:24.374 INFO:tasks.workunit.client.0.smithi110.stdout:9/691: mknod d7/d96/d99/d3c/d14/cec 0 2024-09-21T04:15:24.376 INFO:tasks.workunit.client.0.smithi110.stdout:5/743: getdents d1 0 2024-09-21T04:15:24.377 INFO:tasks.workunit.client.1.smithi148.stdout:1/823: mknod db/d24/d107/c110 0 2024-09-21T04:15:24.378 INFO:tasks.workunit.client.1.smithi148.stdout:1/824: readlink db/d24/d2e/d71/d94/d9a/l6f 0 2024-09-21T04:15:24.378 INFO:tasks.workunit.client.1.smithi148.stdout:1/825: write db/d46/d4c/da3/fbe [603714,81856] 0 2024-09-21T04:15:24.378 INFO:tasks.workunit.client.1.smithi148.stdout:9/851: symlink d5/d60/de1/l130 0 2024-09-21T04:15:24.378 INFO:tasks.workunit.client.1.smithi148.stdout:9/852: chown d5/d2b/d4e/db3/ded/c107 15 1 2024-09-21T04:15:24.379 INFO:tasks.workunit.client.0.smithi110.stdout:5/744: symlink d1/dc/lfd 0 2024-09-21T04:15:24.380 INFO:tasks.workunit.client.1.smithi148.stdout:0/833: rmdir d5/d6/d2a/dcd/df8/da5 39 2024-09-21T04:15:24.380 INFO:tasks.workunit.client.1.smithi148.stdout:0/834: write d5/d6/d7/d11/f29 [4528438,67551] 0 2024-09-21T04:15:24.382 INFO:tasks.workunit.client.1.smithi148.stdout:3/743: creat d3/d5/d6/ff3 x:0 0 0 2024-09-21T04:15:24.383 INFO:tasks.workunit.client.1.smithi148.stdout:5/799: mknod d3/d14/d1f/d72/c110 0 2024-09-21T04:15:24.384 INFO:tasks.workunit.client.1.smithi148.stdout:8/736: truncate dc/d13/d3c/d81/d85/f87 2760051 0 2024-09-21T04:15:24.384 INFO:tasks.workunit.client.1.smithi148.stdout:8/737: readlink dc/d13/d3c/d81/db0/ldb 0 2024-09-21T04:15:24.384 INFO:tasks.workunit.client.1.smithi148.stdout:8/738: creat dc/d3e/fe7 x:0 0 0 2024-09-21T04:15:24.386 INFO:tasks.workunit.client.1.smithi148.stdout:7/801: write d5/d8/d6f/d7d/d90/f6e [77766,78256] 0 2024-09-21T04:15:24.388 INFO:tasks.workunit.client.1.smithi148.stdout:0/835: creat d5/f106 x:0 0 0 2024-09-21T04:15:24.388 INFO:tasks.workunit.client.1.smithi148.stdout:0/836: write d5/d6/d2a/d2b/fd3 [842048,80450] 0 2024-09-21T04:15:24.389 INFO:tasks.workunit.client.1.smithi148.stdout:0/837: fdatasync d5/d82/fc4 0 2024-09-21T04:15:24.389 INFO:tasks.workunit.client.1.smithi148.stdout:0/838: readlink d5/d6/d2a/d2b/l68 0 2024-09-21T04:15:24.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/744: mkdir d3/d5/d6/d2b/d96/db8/df4 0 2024-09-21T04:15:24.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/745: truncate d3/d5/d6/d12/d15/da7/fd7 991640 0 2024-09-21T04:15:24.390 INFO:tasks.workunit.client.1.smithi148.stdout:5/800: symlink d3/d14/d1f/d26/d2e/ddb/dde/l111 0 2024-09-21T04:15:24.390 INFO:tasks.workunit.client.0.smithi110.stdout:2/580: dwrite d2/f3b [0,4194304] 0 2024-09-21T04:15:24.391 INFO:tasks.workunit.client.0.smithi110.stdout:2/581: truncate d2/d50/d9c/fab 319118 0 2024-09-21T04:15:24.391 INFO:tasks.workunit.client.1.smithi148.stdout:8/739: truncate dc/d43/f54 2960278 0 2024-09-21T04:15:24.392 INFO:tasks.workunit.client.0.smithi110.stdout:9/692: dread d7/d96/d99/d3c/f94 [0,4194304] 0 2024-09-21T04:15:24.392 INFO:tasks.workunit.client.0.smithi110.stdout:9/693: chown d7/d96/d99/d3c/d11/f8c 1 1 2024-09-21T04:15:24.393 INFO:tasks.workunit.client.1.smithi148.stdout:1/826: read db/f58 [4243990,94930] 0 2024-09-21T04:15:24.395 INFO:tasks.workunit.client.1.smithi148.stdout:0/839: getdents d5/d6/d2a/dcd/df8/daf/dc8/de8 0 2024-09-21T04:15:24.395 INFO:tasks.workunit.client.1.smithi148.stdout:0/840: chown d5/d6/d2a/la0 0 1 2024-09-21T04:15:24.397 INFO:tasks.workunit.client.0.smithi110.stdout:9/694: dread d7/d26/f7e [0,4194304] 0 2024-09-21T04:15:24.397 INFO:tasks.workunit.client.0.smithi110.stdout:9/695: read - d7/d96/d99/d3c/d14/fc2 zero size 2024-09-21T04:15:24.398 INFO:tasks.workunit.client.1.smithi148.stdout:3/746: unlink d3/d1f/d4d/d82/fe8 0 2024-09-21T04:15:24.400 INFO:tasks.workunit.client.0.smithi110.stdout:5/745: dwrite d1/d49/d4d/d1e/f61 [0,4194304] 0 2024-09-21T04:15:24.402 INFO:tasks.workunit.client.1.smithi148.stdout:5/801: rename d3/d14/d1f/db1/db8/dba/cd2 to d3/d4a/d10f/c112 0 2024-09-21T04:15:24.402 INFO:tasks.workunit.client.1.smithi148.stdout:5/802: dread - d3/d14/d1a/fe5 zero size 2024-09-21T04:15:24.406 INFO:tasks.workunit.client.0.smithi110.stdout:5/746: rename d1/d49/d4d/d1e/d4a/ld2 to d1/d49/d4d/d1e/lfe 0 2024-09-21T04:15:24.409 INFO:tasks.workunit.client.1.smithi148.stdout:7/802: dwrite d5/d8/ff [4194304,4194304] 0 2024-09-21T04:15:24.410 INFO:tasks.workunit.client.0.smithi110.stdout:5/747: symlink d1/d22/d6e/lff 0 2024-09-21T04:15:24.413 INFO:tasks.workunit.client.0.smithi110.stdout:5/748: mkdir d1/d49/d69/d100 0 2024-09-21T04:15:24.413 INFO:tasks.workunit.client.0.smithi110.stdout:5/749: readlink d1/d49/d4d/d1e/l34 0 2024-09-21T04:15:24.415 INFO:tasks.workunit.client.0.smithi110.stdout:5/750: mknod d1/d7/d58/d9e/c101 0 2024-09-21T04:15:24.417 INFO:tasks.workunit.client.1.smithi148.stdout:0/841: mknod d5/dcb/de6/c107 0 2024-09-21T04:15:24.418 INFO:tasks.workunit.client.0.smithi110.stdout:5/751: mknod d1/d39/c102 0 2024-09-21T04:15:24.418 INFO:tasks.workunit.client.0.smithi110.stdout:9/696: dwrite d7/d96/d99/d3c/d4f/d9f/d6f/d78/fb9 [0,4194304] 0 2024-09-21T04:15:24.419 INFO:tasks.workunit.client.1.smithi148.stdout:7/803: write d5/d14/d25/d40/d8b/fe9 [5305964,120832] 0 2024-09-21T04:15:24.419 INFO:tasks.workunit.client.1.smithi148.stdout:3/747: link d3/d1f/d4d/d82/d23/d28/fd9 d3/d1f/d4d/ff5 0 2024-09-21T04:15:24.420 INFO:tasks.workunit.client.1.smithi148.stdout:5/803: unlink d3/d34/d3c/c71 0 2024-09-21T04:15:24.423 INFO:tasks.workunit.client.0.smithi110.stdout:5/752: rmdir d1/d7/d18/d8f 39 2024-09-21T04:15:24.424 INFO:tasks.workunit.client.1.smithi148.stdout:8/740: rmdir dc/d43/d4c/d4f 39 2024-09-21T04:15:24.425 INFO:tasks.workunit.client.0.smithi110.stdout:9/697: mknod d7/d96/d99/d3c/ced 0 2024-09-21T04:15:24.425 INFO:tasks.workunit.client.0.smithi110.stdout:9/698: chown d7/d96/d99/d3c/d1f/d72/c97 468 1 2024-09-21T04:15:24.426 INFO:tasks.workunit.client.1.smithi148.stdout:1/827: rmdir db/dd/ddd 39 2024-09-21T04:15:24.427 INFO:tasks.workunit.client.0.smithi110.stdout:5/753: link d1/d22/d51/la0 d1/d49/d4d/d1e/d4a/dae/l103 0 2024-09-21T04:15:24.432 INFO:tasks.workunit.client.1.smithi148.stdout:7/804: dread d5/daa/fc5 [0,4194304] 0 2024-09-21T04:15:24.432 INFO:tasks.workunit.client.0.smithi110.stdout:5/754: unlink d1/d49/d4d/d1e/la3 0 2024-09-21T04:15:24.433 INFO:tasks.workunit.client.1.smithi148.stdout:7/805: read d5/d8/d6f/d7d/d90/dee/f104 [336571,84780] 0 2024-09-21T04:15:24.433 INFO:tasks.workunit.client.1.smithi148.stdout:6/857: sync 2024-09-21T04:15:24.434 INFO:tasks.workunit.client.1.smithi148.stdout:3/748: rename d3/d5/db4/dc9 to d3/d5/d5c/df6 0 2024-09-21T04:15:24.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/828: symlink db/d24/d2e/d71/d94/dde/de1/l111 0 2024-09-21T04:15:24.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/829: readlink db/d46/d76/lf9 0 2024-09-21T04:15:24.436 INFO:tasks.workunit.client.0.smithi110.stdout:5/755: symlink d1/d22/d51/l104 0 2024-09-21T04:15:24.436 INFO:tasks.workunit.client.0.smithi110.stdout:5/756: write d1/dc/f1a [2392206,12032] 0 2024-09-21T04:15:24.436 INFO:tasks.workunit.client.0.smithi110.stdout:5/757: truncate d1/d7/d58/f60 1461472 0 2024-09-21T04:15:24.437 INFO:tasks.workunit.client.0.smithi110.stdout:5/758: readlink d1/dc/lfd 0 2024-09-21T04:15:24.437 INFO:tasks.workunit.client.0.smithi110.stdout:5/759: creat d1/d7/d18/d3f/f105 x:0 0 0 2024-09-21T04:15:24.438 INFO:tasks.workunit.client.1.smithi148.stdout:6/858: unlink d0/d34/d97/d106/d23/d27/f95 0 2024-09-21T04:15:24.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/804: rename d3/d34/d78/l90 to d3/d14/d1f/d26/d59/d97/d102/l113 0 2024-09-21T04:15:24.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/805: creat d3/d34/d78/f114 x:0 0 0 2024-09-21T04:15:24.441 INFO:tasks.workunit.client.1.smithi148.stdout:3/749: mknod d3/d1f/d4d/d82/cf7 0 2024-09-21T04:15:24.441 INFO:tasks.workunit.client.1.smithi148.stdout:8/741: dread dc/d13/d3c/f5f [0,4194304] 0 2024-09-21T04:15:24.442 INFO:tasks.workunit.client.1.smithi148.stdout:5/806: dread d3/d14/d1f/d26/d59/fb2 [0,4194304] 0 2024-09-21T04:15:24.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/859: mkdir d0/d8/dba/d108 0 2024-09-21T04:15:24.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/860: creat d0/d34/d38/dc6/f109 x:0 0 0 2024-09-21T04:15:24.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/861: write d0/d34/d38/dc6/fd3 [706580,75999] 0 2024-09-21T04:15:24.444 INFO:tasks.workunit.client.1.smithi148.stdout:6/862: dread - d0/d34/d38/dc6/f109 zero size 2024-09-21T04:15:24.448 INFO:tasks.workunit.client.0.smithi110.stdout:9/699: dwrite d7/d96/d99/d3c/d1f/d43/f7b [0,4194304] 0 2024-09-21T04:15:24.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/750: mknod d3/d5/d6/d2b/da2/de0/cf8 0 2024-09-21T04:15:24.449 INFO:tasks.workunit.client.1.smithi148.stdout:1/830: getdents db/d24/d2e/d71/d94 0 2024-09-21T04:15:24.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/807: creat d3/d14/d1f/db1/db8/dba/f115 x:0 0 0 2024-09-21T04:15:24.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/808: stat d3/d14/d1f/d26/d2e/d56/l8c 0 2024-09-21T04:15:24.454 INFO:tasks.workunit.client.1.smithi148.stdout:6/863: stat d0/d34/d97/d106/d23/ca 0 2024-09-21T04:15:24.455 INFO:tasks.workunit.client.0.smithi110.stdout:9/700: unlink d7/d96/d99/d3c/d11/fcf 0 2024-09-21T04:15:24.455 INFO:tasks.workunit.client.1.smithi148.stdout:6/864: read d0/d34/d97/f30 [261677,54069] 0 2024-09-21T04:15:24.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/742: rmdir dc/d3e 39 2024-09-21T04:15:24.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/743: chown dc/d13/d28/de4 99844 1 2024-09-21T04:15:24.456 INFO:tasks.workunit.client.0.smithi110.stdout:5/760: dwrite d1/d49/d4d/f46 [4194304,4194304] 0 2024-09-21T04:15:24.458 INFO:tasks.workunit.client.1.smithi148.stdout:3/751: creat d3/d1f/d4d/ff9 x:0 0 0 2024-09-21T04:15:24.458 INFO:tasks.workunit.client.1.smithi148.stdout:4/894: sync 2024-09-21T04:15:24.459 INFO:tasks.workunit.client.0.smithi110.stdout:9/701: rename d7/d96/d99/d3c/d1f/d43/db5/cd4 to d7/d10/d88/cee 0 2024-09-21T04:15:24.460 INFO:tasks.workunit.client.1.smithi148.stdout:1/831: mkdir db/d24/d2a/d112 0 2024-09-21T04:15:24.462 INFO:tasks.workunit.client.1.smithi148.stdout:5/809: mkdir d3/d14/d1f/d26/d2e/ddb/deb/d105/d116 0 2024-09-21T04:15:24.464 INFO:tasks.workunit.client.0.smithi110.stdout:9/702: mknod d7/d96/d99/d3c/d1f/d43/cef 0 2024-09-21T04:15:24.466 INFO:tasks.workunit.client.1.smithi148.stdout:6/865: creat d0/d59/f10a x:0 0 0 2024-09-21T04:15:24.466 INFO:tasks.workunit.client.1.smithi148.stdout:6/866: chown d0/d34/d97/d106/d23/d32 986472 1 2024-09-21T04:15:24.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/752: mknod d3/d5/dac/cfa 0 2024-09-21T04:15:24.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/753: dread - d3/d1f/d4d/ff5 zero size 2024-09-21T04:15:24.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/754: write d3/d5/d6/f17 [3319519,15677] 0 2024-09-21T04:15:24.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/755: fdatasync d3/d5/d6/ff3 0 2024-09-21T04:15:24.469 INFO:tasks.workunit.client.1.smithi148.stdout:4/895: truncate d4/d87/f89 3437679 0 2024-09-21T04:15:24.472 INFO:tasks.workunit.client.1.smithi148.stdout:1/832: rename db/d46/d4c/da3/da9 to db/d24/d2e/d59/da7/dc1/dd2/d10c/d113 0 2024-09-21T04:15:24.473 INFO:tasks.workunit.client.1.smithi148.stdout:5/810: mknod d3/d14/d1f/d72/dc9/dd8/c117 0 2024-09-21T04:15:24.474 INFO:tasks.workunit.client.1.smithi148.stdout:3/756: mkdir d3/d5/db4/dfb 0 2024-09-21T04:15:24.475 INFO:tasks.workunit.client.1.smithi148.stdout:4/896: mkdir d4/d87/dc1/d11a 0 2024-09-21T04:15:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/833: truncate db/f1b 2009666 0 2024-09-21T04:15:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/834: write db/d24/d2e/d71/d94/d9a/f95 [612987,7519] 0 2024-09-21T04:15:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/835: write db/d24/d2e/d71/d94/d9a/f95 [452285,70135] 0 2024-09-21T04:15:24.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/811: mkdir d3/d14/d1f/d118 0 2024-09-21T04:15:24.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/812: write d3/d14/d1f/d26/d2e/ddb/ffe [114356,105753] 0 2024-09-21T04:15:24.480 INFO:tasks.workunit.client.1.smithi148.stdout:1/836: chown db/dd/l68 199 1 2024-09-21T04:15:24.482 INFO:tasks.workunit.client.0.smithi110.stdout:5/761: dwrite d1/d22/d51/f98 [0,4194304] 0 2024-09-21T04:15:24.482 INFO:tasks.workunit.client.1.smithi148.stdout:4/897: dread d4/d12/d75/d85/f99 [0,4194304] 0 2024-09-21T04:15:24.482 INFO:tasks.workunit.client.1.smithi148.stdout:4/898: readlink d4/d12/d16/d29/d61/d6c/l84 0 2024-09-21T04:15:24.485 INFO:tasks.workunit.client.1.smithi148.stdout:4/899: mkdir d4/d12/d16/d29/d61/d6c/df1/d11b 0 2024-09-21T04:15:24.485 INFO:tasks.workunit.client.1.smithi148.stdout:4/900: dread - d4/d66/d7f/d2a/da9/f10a zero size 2024-09-21T04:15:24.489 INFO:tasks.workunit.client.1.smithi148.stdout:8/744: dwrite dc/d13/d76/d5d/fa6 [0,4194304] 0 2024-09-21T04:15:24.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/837: dread db/d24/d2a/f45 [0,4194304] 0 2024-09-21T04:15:24.491 INFO:tasks.workunit.client.0.smithi110.stdout:5/762: rename d1/d22/d51/f65 to d1/d49/de4/df9/f106 0 2024-09-21T04:15:24.497 INFO:tasks.workunit.client.1.smithi148.stdout:1/838: unlink db/d46/f53 0 2024-09-21T04:15:24.497 INFO:tasks.workunit.client.0.smithi110.stdout:5/763: mknod d1/d7/d18/c107 0 2024-09-21T04:15:24.497 INFO:tasks.workunit.client.0.smithi110.stdout:5/764: chown d1/d49/d4d/d1e/l20 0 1 2024-09-21T04:15:24.499 INFO:tasks.workunit.client.0.smithi110.stdout:5/765: link d1/dc/f1a d1/dc/f108 0 2024-09-21T04:15:24.500 INFO:tasks.workunit.client.0.smithi110.stdout:5/766: getdents d1/d7/d58/d72 0 2024-09-21T04:15:24.500 INFO:tasks.workunit.client.0.smithi110.stdout:5/767: mkdir d1/dc/ded/d109 0 2024-09-21T04:15:24.500 INFO:tasks.workunit.client.0.smithi110.stdout:5/768: stat d1/d49/df1 0 2024-09-21T04:15:24.501 INFO:tasks.workunit.client.0.smithi110.stdout:5/769: rename d1/d39/l4b to d1/d49/l10a 0 2024-09-21T04:15:24.501 INFO:tasks.workunit.client.0.smithi110.stdout:5/770: stat d1/d7/l90 0 2024-09-21T04:15:24.502 INFO:tasks.workunit.client.0.smithi110.stdout:5/771: write d1/d22/f97 [205716,46954] 0 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/745: dread dc/d3e/f8b [0,4194304] 0 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/746: chown dc/d13/d39/cc3 103545 1 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/747: readlink dc/d13/d28/l36 0 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/748: fsync dc/d3e/fe7 0 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/749: stat dc/l27 0 2024-09-21T04:15:24.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/750: read - dc/d13/d39/fc6 zero size 2024-09-21T04:15:24.507 INFO:tasks.workunit.client.1.smithi148.stdout:8/751: link dc/d13/d28/d62/fb6 dc/d13/d28/d62/fe8 0 2024-09-21T04:15:24.508 INFO:tasks.workunit.client.0.smithi110.stdout:5/772: dread d1/d49/d4d/d1e/d5a/d64/f73 [0,4194304] 0 2024-09-21T04:15:24.508 INFO:tasks.workunit.client.0.smithi110.stdout:5/773: chown d1/d39 20 1 2024-09-21T04:15:24.508 INFO:tasks.workunit.client.1.smithi148.stdout:8/752: creat dc/d13/d76/dd8/fe9 x:0 0 0 2024-09-21T04:15:24.509 INFO:tasks.workunit.client.1.smithi148.stdout:8/753: read - dc/d43/d4c/f8c zero size 2024-09-21T04:15:24.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/867: dwrite d0/d34/d38/fd6 [0,4194304] 0 2024-09-21T04:15:24.512 INFO:tasks.workunit.client.1.smithi148.stdout:4/901: dwrite d4/d66/d7f/d2a/f36 [4194304,4194304] 0 2024-09-21T04:15:24.513 INFO:tasks.workunit.client.1.smithi148.stdout:3/757: dwrite d3/d1f/d4d/d82/d23/f26 [4194304,4194304] 0 2024-09-21T04:15:24.513 INFO:tasks.workunit.client.1.smithi148.stdout:3/758: readlink d3/d1f/d4d/l68 0 2024-09-21T04:15:24.513 INFO:tasks.workunit.client.1.smithi148.stdout:3/759: write d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 [5056794,33279] 0 2024-09-21T04:15:24.513 INFO:tasks.workunit.client.1.smithi148.stdout:5/813: dwrite d3/d14/d1f/d26/d2e/ddb/dde/f106 [0,4194304] 0 2024-09-21T04:15:24.519 INFO:tasks.workunit.client.1.smithi148.stdout:4/902: truncate d4/d12/d16/d29/d61/d7e/fa0 2153871 0 2024-09-21T04:15:24.522 INFO:tasks.workunit.client.1.smithi148.stdout:8/754: rename dc/d13/d39/f3b to dc/d13/d28/de4/fea 0 2024-09-21T04:15:24.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/814: dread d3/d34/f8d [0,4194304] 0 2024-09-21T04:15:24.530 INFO:tasks.workunit.client.1.smithi148.stdout:3/760: getdents d3/d5/d6/d2b/da2 0 2024-09-21T04:15:24.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/761: creat d3/d5/d6/d2b/da2/da3/de4/ffc x:0 0 0 2024-09-21T04:15:24.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/762: mknod d3/d5/d6/d12/d43/d54/d9b/dd6/cfd 0 2024-09-21T04:15:24.534 INFO:tasks.workunit.client.1.smithi148.stdout:6/868: dwrite d0/d34/d97/d106/d23/d32/faa [4194304,4194304] 0 2024-09-21T04:15:24.534 INFO:tasks.workunit.client.1.smithi148.stdout:3/763: rmdir d3/d5/dac 39 2024-09-21T04:15:24.534 INFO:tasks.workunit.client.1.smithi148.stdout:3/764: fdatasync d3/d5/d6/f17 0 2024-09-21T04:15:24.535 INFO:tasks.workunit.client.0.smithi110.stdout:5/774: dwrite d1/d39/f21 [0,4194304] 0 2024-09-21T04:15:24.536 INFO:tasks.workunit.client.1.smithi148.stdout:6/869: creat d0/d34/d97/d106/d23/d27/d80/db0/f10b x:0 0 0 2024-09-21T04:15:24.536 INFO:tasks.workunit.client.1.smithi148.stdout:5/815: dread d3/d14/d1f/d26/d2e/ddb/dde/f106 [0,4194304] 0 2024-09-21T04:15:24.539 INFO:tasks.workunit.client.0.smithi110.stdout:5/775: truncate d1/d22/d51/fa5 1252463 0 2024-09-21T04:15:24.540 INFO:tasks.workunit.client.1.smithi148.stdout:6/870: creat d0/d34/da0/dd4/f10c x:0 0 0 2024-09-21T04:15:24.541 INFO:tasks.workunit.client.1.smithi148.stdout:3/765: getdents d3/d1f/d4d/d82/d23 0 2024-09-21T04:15:24.542 INFO:tasks.workunit.client.0.smithi110.stdout:5/776: symlink d1/d7/d18/l10b 0 2024-09-21T04:15:24.543 INFO:tasks.workunit.client.1.smithi148.stdout:5/816: getdents d3/d14/d1f 0 2024-09-21T04:15:24.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/871: creat d0/d34/d97/d106/d23/d27/f10d x:0 0 0 2024-09-21T04:15:24.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/872: write d0/d34/d38/dc6/f109 [516050,63440] 0 2024-09-21T04:15:24.545 INFO:tasks.workunit.client.0.smithi110.stdout:5/777: creat d1/d49/de4/df9/f10c x:0 0 0 2024-09-21T04:15:24.546 INFO:tasks.workunit.client.1.smithi148.stdout:3/766: mknod d3/d5/d6/d2b/da2/da3/de4/cfe 0 2024-09-21T04:15:24.550 INFO:tasks.workunit.client.1.smithi148.stdout:5/817: getdents d3/d14/d1f/db1/db8/dba 0 2024-09-21T04:15:24.557 INFO:tasks.workunit.client.0.smithi110.stdout:5/778: dwrite d1/d49/d6f/f78 [0,4194304] 0 2024-09-21T04:15:24.560 INFO:tasks.workunit.client.1.smithi148.stdout:5/818: truncate d3/d14/d1f/d26/d2e/d56/f15 1619082 0 2024-09-21T04:15:24.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/873: rmdir d0/d34/d97 39 2024-09-21T04:15:24.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/819: getdents d3/d34/d3c 0 2024-09-21T04:15:24.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/874: mknod d0/d34/d97/d106/d23/d32/d93/d96/c10e 0 2024-09-21T04:15:24.565 INFO:tasks.workunit.client.1.smithi148.stdout:3/767: dwrite d3/d5/d5c/fc3 [0,4194304] 0 2024-09-21T04:15:24.566 INFO:tasks.workunit.client.1.smithi148.stdout:3/768: stat d3/d1f/d4d/d82/d23/d28/fd9 0 2024-09-21T04:15:24.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/875: mkdir d0/d34/d97/d106/d23/d27/d80/da4/d10f 0 2024-09-21T04:15:24.567 INFO:tasks.workunit.client.0.smithi110.stdout:5/779: dread d1/d49/d6f/f78 [0,4194304] 0 2024-09-21T04:15:24.568 INFO:tasks.workunit.client.0.smithi110.stdout:5/780: rename d1/d39/c102 to d1/d22/db6/dc6/c10d 0 2024-09-21T04:15:24.569 INFO:tasks.workunit.client.0.smithi110.stdout:5/781: getdents d1/d49/df1 0 2024-09-21T04:15:24.569 INFO:tasks.workunit.client.1.smithi148.stdout:3/769: read d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 [4947427,79398] 0 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/820: dread d3/d14/d1a/f33 [0,4194304] 0 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/821: chown d3/d14/d1f/d72/dc9/cea 11 1 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/822: truncate d3/d37/fdd 817891 0 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/876: mkdir d0/d34/d97/d106/d23/d27/d80/d110 0 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/877: fdatasync d0/d8/d18/f25 0 2024-09-21T04:15:24.570 INFO:tasks.workunit.client.0.smithi110.stdout:5/782: mkdir d1/d49/d4d/d1e/d4a/dd4/dd5/d10e 0 2024-09-21T04:15:24.572 INFO:tasks.workunit.client.1.smithi148.stdout:5/823: unlink d3/d14/l5a 0 2024-09-21T04:15:24.573 INFO:tasks.workunit.client.1.smithi148.stdout:6/878: creat d0/df9/f111 x:0 0 0 2024-09-21T04:15:24.573 INFO:tasks.workunit.client.1.smithi148.stdout:6/879: stat d0/d34/d97/d106/d23/d27/d62/d9a/fa3 0 2024-09-21T04:15:24.573 INFO:tasks.workunit.client.0.smithi110.stdout:5/783: mknod d1/d49/d4d/d1e/d4a/dd4/dda/c10f 0 2024-09-21T04:15:24.574 INFO:tasks.workunit.client.1.smithi148.stdout:5/824: creat d3/d14/d1f/d72/dc9/f119 x:0 0 0 2024-09-21T04:15:24.574 INFO:tasks.workunit.client.1.smithi148.stdout:5/825: creat d3/d14/d1f/d26/d2e/d56/f11a x:0 0 0 2024-09-21T04:15:24.574 INFO:tasks.workunit.client.1.smithi148.stdout:5/826: truncate d3/d14/d1f/d26/d2e/f10a 407901 0 2024-09-21T04:15:24.575 INFO:tasks.workunit.client.1.smithi148.stdout:6/880: symlink d0/d34/d97/d106/d23/d27/d62/dfb/l112 0 2024-09-21T04:15:24.575 INFO:tasks.workunit.client.1.smithi148.stdout:6/881: write d0/d34/d97/d106/d23/d27/d80/da4/fd9 [462992,90560] 0 2024-09-21T04:15:24.576 INFO:tasks.workunit.client.0.smithi110.stdout:5/784: creat d1/d49/d4d/d1e/d5a/f110 x:0 0 0 2024-09-21T04:15:24.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/827: mknod d3/d14/d1a/c11b 0 2024-09-21T04:15:24.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/828: dread - d3/d37/fe4 zero size 2024-09-21T04:15:24.579 INFO:tasks.workunit.client.1.smithi148.stdout:5/829: fdatasync d3/d14/d1f/db1/db8/dba/f115 0 2024-09-21T04:15:24.581 INFO:tasks.workunit.client.1.smithi148.stdout:6/882: rename d0/d34/d97/d106/d23/d27/d80/da4/d10f to d0/d8/dba/d113 0 2024-09-21T04:15:24.584 INFO:tasks.workunit.client.0.smithi110.stdout:5/785: write d1/d22/d51/f98 [1092464,118632] 0 2024-09-21T04:15:24.587 INFO:tasks.workunit.client.0.smithi110.stdout:5/786: symlink d1/d49/l111 0 2024-09-21T04:15:24.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/787: mknod d1/dc/c112 0 2024-09-21T04:15:24.588 INFO:tasks.workunit.client.0.smithi110.stdout:5/788: creat d1/d49/d4d/d1e/d4a/dd4/dd5/f113 x:0 0 0 2024-09-21T04:15:24.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/770: dwrite d3/d1f/d4d/d82/d74/f91 [0,4194304] 0 2024-09-21T04:15:24.590 INFO:tasks.workunit.client.1.smithi148.stdout:6/883: dread d0/d8/d18/f25 [4194304,4194304] 0 2024-09-21T04:15:24.591 INFO:tasks.workunit.client.0.smithi110.stdout:5/789: link d1/d49/d4d/d1e/d4a/faf d1/dc/ded/f114 0 2024-09-21T04:15:24.593 INFO:tasks.workunit.client.1.smithi148.stdout:6/884: creat d0/d8/f114 x:0 0 0 2024-09-21T04:15:24.594 INFO:tasks.workunit.client.0.smithi110.stdout:5/790: link d1/d39/c94 d1/d49/d4d/d1e/d5a/d64/d7b/c115 0 2024-09-21T04:15:24.594 INFO:tasks.workunit.client.0.smithi110.stdout:5/791: stat d1/d49/d4d/d1e/f2f 0 2024-09-21T04:15:24.594 INFO:tasks.workunit.client.0.smithi110.stdout:5/792: truncate d1/d49/d4d/d1e/d4a/ff3 1034781 0 2024-09-21T04:15:24.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/830: dwrite d3/d14/f84 [0,4194304] 0 2024-09-21T04:15:24.609 INFO:tasks.workunit.client.1.smithi148.stdout:3/771: dwrite d3/d5/d6/d2b/d96/f9d [0,4194304] 0 2024-09-21T04:15:24.610 INFO:tasks.workunit.client.0.smithi110.stdout:7/690: sync 2024-09-21T04:15:24.614 INFO:tasks.workunit.client.0.smithi110.stdout:7/691: mknod dd/d1c/dad/db2/d6d/dbc/ce0 0 2024-09-21T04:15:24.617 INFO:tasks.workunit.client.0.smithi110.stdout:7/692: creat dd/d1c/dad/db2/d56/d66/fe1 x:0 0 0 2024-09-21T04:15:24.619 INFO:tasks.workunit.client.0.smithi110.stdout:7/693: creat dd/d1c/d89/fe2 x:0 0 0 2024-09-21T04:15:24.619 INFO:tasks.workunit.client.0.smithi110.stdout:7/694: write dd/f1a [4025255,128988] 0 2024-09-21T04:15:24.620 INFO:tasks.workunit.client.0.smithi110.stdout:7/695: read dd/d14/d17/d8a/f8b [118006,39164] 0 2024-09-21T04:15:24.622 INFO:tasks.workunit.client.0.smithi110.stdout:7/696: rename dd/d1c/dad/db2/d56/d66/da6/d7a/f91 to dd/d14/d17/d8a/fe3 0 2024-09-21T04:15:24.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/793: dwrite d1/dc/fd6 [0,4194304] 0 2024-09-21T04:15:24.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/794: fdatasync d1/d7/f2b 0 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/831: dwrite d3/d34/d3c/d5d/fca [0,4194304] 0 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/832: chown d3/cbc 465614055 1 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/833: write d3/d34/d78/fa1 [1537015,129442] 0 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.1.smithi148.stdout:6/885: dwrite d0/d34/d97/d106/d23/d32/d93/fa8 [0,4194304] 0 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.1.smithi148.stdout:6/886: truncate d0/d34/d97/d106/fab 296516 0 2024-09-21T04:15:24.624 INFO:tasks.workunit.client.0.smithi110.stdout:7/697: read dd/d1c/dad/db2/d56/d66/fb6 [494683,112611] 0 2024-09-21T04:15:24.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/795: mkdir d1/d7/d58/d9e/d116 0 2024-09-21T04:15:24.626 INFO:tasks.workunit.client.0.smithi110.stdout:5/796: dread - d1/d49/d6f/fbd zero size 2024-09-21T04:15:24.626 INFO:tasks.workunit.client.1.smithi148.stdout:3/772: dwrite d3/d5/d6/d12/d43/d54/f6a [0,4194304] 0 2024-09-21T04:15:24.629 INFO:tasks.workunit.client.0.smithi110.stdout:7/698: unlink dd/d1c/dad/db2/d56/d66/da6/l70 0 2024-09-21T04:15:24.629 INFO:tasks.workunit.client.0.smithi110.stdout:7/699: chown dd/d1c/dad/db2/d6d/ddc 2800470 1 2024-09-21T04:15:24.630 INFO:tasks.workunit.client.0.smithi110.stdout:7/700: chown dd/d1c/dad/db2/d56/d66/fe1 45298 1 2024-09-21T04:15:24.631 INFO:tasks.workunit.client.0.smithi110.stdout:5/797: mkdir d1/d7/d18/d3f/d117 0 2024-09-21T04:15:24.631 INFO:tasks.workunit.client.0.smithi110.stdout:5/798: creat d1/d49/d4d/d1e/d4a/dd4/dd5/f118 x:0 0 0 2024-09-21T04:15:24.633 INFO:tasks.workunit.client.1.smithi148.stdout:5/834: mkdir d3/d4a/d11c 0 2024-09-21T04:15:24.633 INFO:tasks.workunit.client.1.smithi148.stdout:5/835: creat d3/d34/f11d x:0 0 0 2024-09-21T04:15:24.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/701: creat dd/d4d/dc1/fe4 x:0 0 0 2024-09-21T04:15:24.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/702: write dd/d14/f41 [944488,35954] 0 2024-09-21T04:15:24.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/703: truncate dd/d1c/dad/db2/d6d/fd3 252628 0 2024-09-21T04:15:24.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/887: mkdir d0/d34/d97/d106/d23/d27/d80/da4/df1/d115 0 2024-09-21T04:15:24.639 INFO:tasks.workunit.client.0.smithi110.stdout:7/704: creat dd/d14/fe5 x:0 0 0 2024-09-21T04:15:24.640 INFO:tasks.workunit.client.1.smithi148.stdout:2/925: sync 2024-09-21T04:15:24.643 INFO:tasks.workunit.client.1.smithi148.stdout:5/836: truncate d3/d14/d1f/d26/d2e/d56/fc4 1679604 0 2024-09-21T04:15:24.644 INFO:tasks.workunit.client.0.smithi110.stdout:5/799: dread d1/d7/f4e [0,4194304] 0 2024-09-21T04:15:24.644 INFO:tasks.workunit.client.1.smithi148.stdout:6/888: dread d0/d34/d97/d106/d23/d27/d80/fb2 [0,4194304] 0 2024-09-21T04:15:24.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/837: symlink d3/d14/d1f/d26/d2e/ddb/dde/l11e 0 2024-09-21T04:15:24.647 INFO:tasks.workunit.client.1.smithi148.stdout:6/889: dread d0/d34/d38/dc6/fd3 [0,4194304] 0 2024-09-21T04:15:24.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/838: truncate d3/d14/d1f/db1/db8/dba/fc1 2563846 0 2024-09-21T04:15:24.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/890: rename d0/d34/d97/d106/d23/d27/d80/df0 to d0/df9/d116 0 2024-09-21T04:15:24.651 INFO:tasks.workunit.client.1.smithi148.stdout:6/891: mknod d0/d34/d38/d57/d58/c117 0 2024-09-21T04:15:24.656 INFO:tasks.workunit.client.1.smithi148.stdout:6/892: getdents d0 0 2024-09-21T04:15:24.657 INFO:tasks.workunit.client.1.smithi148.stdout:9/853: sync 2024-09-21T04:15:24.657 INFO:tasks.workunit.client.1.smithi148.stdout:6/893: rmdir d0/d34/d97/d106/d23/d27/dc2 39 2024-09-21T04:15:24.658 INFO:tasks.workunit.client.0.smithi110.stdout:7/705: dwrite dd/d14/fcb [0,4194304] 0 2024-09-21T04:15:24.658 INFO:tasks.workunit.client.0.smithi110.stdout:7/706: chown dd/d1c/dad/db2/f4f 28061529 1 2024-09-21T04:15:24.658 INFO:tasks.workunit.client.1.smithi148.stdout:9/854: creat d5/d60/d28/d2f/dbf/f131 x:0 0 0 2024-09-21T04:15:24.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/773: dwrite d3/d5/d5c/fc4 [0,4194304] 0 2024-09-21T04:15:24.658 INFO:tasks.workunit.client.1.smithi148.stdout:5/839: write d3/d14/d1f/d26/d2e/ffb [197603,24038] 0 2024-09-21T04:15:24.659 INFO:tasks.workunit.client.1.smithi148.stdout:6/894: rename d0/d59/c84 to d0/d34/d97/d106/d23/d27/d80/da4/c118 0 2024-09-21T04:15:24.664 INFO:tasks.workunit.client.1.smithi148.stdout:6/895: stat d0/d34/d97/d106/lb5 0 2024-09-21T04:15:24.665 INFO:tasks.workunit.client.1.smithi148.stdout:6/896: read d0/d34/d97/d106/d23/d27/fe1 [850577,20393] 0 2024-09-21T04:15:24.665 INFO:tasks.workunit.client.1.smithi148.stdout:2/926: dwrite d0/d9/da2/ff2 [0,4194304] 0 2024-09-21T04:15:24.665 INFO:tasks.workunit.client.0.smithi110.stdout:7/707: link dd/c63 dd/d1c/dad/db2/d56/d66/da6/d98/ce6 0 2024-09-21T04:15:24.667 INFO:tasks.workunit.client.1.smithi148.stdout:9/855: unlink d5/d1f/d81/dcf/d106/fe8 0 2024-09-21T04:15:24.667 INFO:tasks.workunit.client.1.smithi148.stdout:9/856: chown d5/f96 5021 1 2024-09-21T04:15:24.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/897: write d0/d34/d38/d43/f5e [7038736,58809] 0 2024-09-21T04:15:24.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/898: creat d0/d34/d38/dc6/f119 x:0 0 0 2024-09-21T04:15:24.671 INFO:tasks.workunit.client.1.smithi148.stdout:3/774: mknod d3/d5/d6/d12/d43/cff 0 2024-09-21T04:15:24.672 INFO:tasks.workunit.client.1.smithi148.stdout:5/840: mknod d3/d14/d1f/d118/c11f 0 2024-09-21T04:15:24.674 INFO:tasks.workunit.client.1.smithi148.stdout:2/927: link d0/d2b/da3/cd2 d0/d9/dc/d13/d50/c126 0 2024-09-21T04:15:24.675 INFO:tasks.workunit.client.1.smithi148.stdout:9/857: mknod d5/d1f/d81/dcf/c132 0 2024-09-21T04:15:24.676 INFO:tasks.workunit.client.1.smithi148.stdout:6/899: mknod d0/c11a 0 2024-09-21T04:15:24.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/775: mkdir d3/d1f/d4d/d82/d23/daf/d100 0 2024-09-21T04:15:24.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/776: chown d3/d5/d6/d2b/da2/da3/de4 17 1 2024-09-21T04:15:24.677 INFO:tasks.workunit.client.1.smithi148.stdout:2/928: symlink d0/d2b/da3/da1/d11d/l127 0 2024-09-21T04:15:24.677 INFO:tasks.workunit.client.1.smithi148.stdout:2/929: write d0/d9/d16/f60 [1855574,111142] 0 2024-09-21T04:15:24.679 INFO:tasks.workunit.client.1.smithi148.stdout:3/777: mknod d3/d5/c101 0 2024-09-21T04:15:24.680 INFO:tasks.workunit.client.1.smithi148.stdout:6/900: mkdir d0/d34/d38/d43/d63/d11b 0 2024-09-21T04:15:24.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/778: mknod d3/d5/d6/d12/d15/c102 0 2024-09-21T04:15:24.710 INFO:tasks.workunit.client.0.smithi110.stdout:3/642: sync 2024-09-21T04:15:24.710 INFO:tasks.workunit.client.0.smithi110.stdout:3/643: write d0/d12/d2a/f3e [2528388,117880] 0 2024-09-21T04:15:24.710 INFO:tasks.workunit.client.0.smithi110.stdout:3/644: chown d0/d12/d16/c3f 50398810 1 2024-09-21T04:15:24.711 INFO:tasks.workunit.client.0.smithi110.stdout:3/645: creat d0/d12/d16/d1a/d6f/fe2 x:0 0 0 2024-09-21T04:15:24.713 INFO:tasks.workunit.client.0.smithi110.stdout:3/646: unlink d0/d51/d7b/f8c 0 2024-09-21T04:15:24.714 INFO:tasks.workunit.client.0.smithi110.stdout:7/708: dwrite dd/d1c/dad/db2/d56/d66/da6/d7a/fa4 [0,4194304] 0 2024-09-21T04:15:24.714 INFO:tasks.workunit.client.0.smithi110.stdout:3/647: getdents d0/d12/d2a/d80/d9c 0 2024-09-21T04:15:24.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/858: dwrite d5/d2b/d4e/fa3 [0,4194304] 0 2024-09-21T04:15:24.718 INFO:tasks.workunit.client.1.smithi148.stdout:2/930: dwrite d0/d9/dc/d13/f8b [0,4194304] 0 2024-09-21T04:15:24.718 INFO:tasks.workunit.client.0.smithi110.stdout:7/709: link dd/d1c/dad/db2/d4a/d52/f58 dd/d14/d17/d8a/dd2/fe7 0 2024-09-21T04:15:24.719 INFO:tasks.workunit.client.0.smithi110.stdout:7/710: fsync dd/d1c/dad/db2/d56/f86 0 2024-09-21T04:15:24.724 INFO:tasks.workunit.client.0.smithi110.stdout:3/648: link d0/d12/d16/l2b d0/d12/d3c/d5a/d65/le3 0 2024-09-21T04:15:24.730 INFO:tasks.workunit.client.0.smithi110.stdout:7/711: write dd/d1c/dad/db2/d4a/d52/f9c [1883144,48690] 0 2024-09-21T04:15:24.730 INFO:tasks.workunit.client.1.smithi148.stdout:3/779: dwrite d3/d5/d5c/fc4 [0,4194304] 0 2024-09-21T04:15:24.731 INFO:tasks.workunit.client.0.smithi110.stdout:7/712: truncate dd/f7e 3274966 0 2024-09-21T04:15:24.732 INFO:tasks.workunit.client.1.smithi148.stdout:6/901: rmdir d0/d34/d38/d43 39 2024-09-21T04:15:24.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/859: dwrite d5/d1e/f65 [0,4194304] 0 2024-09-21T04:15:24.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/780: mknod d3/d5/d6/d12/c103 0 2024-09-21T04:15:24.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/781: creat d3/d5/db4/f104 x:0 0 0 2024-09-21T04:15:24.737 INFO:tasks.workunit.client.0.smithi110.stdout:7/713: mkdir dd/d1c/de8 0 2024-09-21T04:15:24.738 INFO:tasks.workunit.client.1.smithi148.stdout:6/902: mknod d0/d34/da0/df5/c11c 0 2024-09-21T04:15:24.738 INFO:tasks.workunit.client.1.smithi148.stdout:6/903: fsync d0/d34/d97/d106/d23/d27/d80/fb2 0 2024-09-21T04:15:24.741 INFO:tasks.workunit.client.0.smithi110.stdout:7/714: mkdir dd/d48/d9f/de9 0 2024-09-21T04:15:24.741 INFO:tasks.workunit.client.0.smithi110.stdout:7/715: readlink dd/d1c/dad/db2/d56/d66/da6/d98/ld6 0 2024-09-21T04:15:24.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/860: rename d5/d1e/d37/fb0 to d5/d1e/d37/d50/f133 0 2024-09-21T04:15:24.748 INFO:tasks.workunit.client.1.smithi148.stdout:2/931: dwrite d0/d9/da2/ff2 [0,4194304] 0 2024-09-21T04:15:24.750 INFO:tasks.workunit.client.0.smithi110.stdout:3/649: dwrite d0/d12/d3c/d5a/dc8/fdf [0,4194304] 0 2024-09-21T04:15:24.750 INFO:tasks.workunit.client.0.smithi110.stdout:3/650: write d0/d12/d16/f25 [1133694,26720] 0 2024-09-21T04:15:24.750 INFO:tasks.workunit.client.0.smithi110.stdout:3/651: fdatasync d0/d12/d3c/d5a/f64 0 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:3/782: rename d3/d1f/d4d/d82/d23/d28/d65 to d3/d1f/d4d/d82/d23/db5/d105 0 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:3/783: fsync d3/d5/d6/f17 0 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/755: sync 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:1/839: sync 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/756: truncate dc/d13/d3c/d81/d85/fcc 744065 0 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/903: sync 2024-09-21T04:15:24.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/904: chown d4/d87/d91 1465 1 2024-09-21T04:15:24.752 INFO:tasks.workunit.client.1.smithi148.stdout:0/842: sync 2024-09-21T04:15:24.752 INFO:tasks.workunit.client.1.smithi148.stdout:7/806: sync 2024-09-21T04:15:24.752 INFO:tasks.workunit.client.1.smithi148.stdout:0/843: fdatasync d5/d6/f5a 0 2024-09-21T04:15:24.752 INFO:tasks.workunit.client.1.smithi148.stdout:7/807: fdatasync d5/d14/f78 0 2024-09-21T04:15:24.753 INFO:tasks.workunit.client.1.smithi148.stdout:2/932: creat d0/d9/d16/d85/f128 x:0 0 0 2024-09-21T04:15:24.754 INFO:tasks.workunit.client.1.smithi148.stdout:6/904: read d0/d34/d97/d106/d23/ff [287115,22905] 0 2024-09-21T04:15:24.756 INFO:tasks.workunit.client.1.smithi148.stdout:8/757: truncate dc/d13/dbd/fde 669332 0 2024-09-21T04:15:24.758 INFO:tasks.workunit.client.1.smithi148.stdout:1/840: creat db/d24/d2e/d59/da7/dc1/dd2/d10c/f114 x:0 0 0 2024-09-21T04:15:24.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/841: write db/d24/d2a/ff7 [632015,63109] 0 2024-09-21T04:15:24.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/842: creat db/d24/d2a/da4/f115 x:0 0 0 2024-09-21T04:15:24.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/808: creat d5/d14/d25/d40/d7b/f119 x:0 0 0 2024-09-21T04:15:24.760 INFO:tasks.workunit.client.0.smithi110.stdout:7/716: dwrite dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:24.761 INFO:tasks.workunit.client.0.smithi110.stdout:7/717: chown dd/d1c/dad/db2/d56/d66/da6/d98 5595458 1 2024-09-21T04:15:24.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/933: mkdir d0/d9/d59/dda/d129 0 2024-09-21T04:15:24.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/934: truncate d0/d9/dc/d13/d25/f10e 985647 0 2024-09-21T04:15:24.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/935: fdatasync d0/d9/dc/d13/d4c/f8a 0 2024-09-21T04:15:24.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/905: mkdir d0/d8/dba/d113/d11d 0 2024-09-21T04:15:24.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/906: fdatasync d0/d34/d38/dc6/f119 0 2024-09-21T04:15:24.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/907: chown d0/d8/dba/dbc 19021 1 2024-09-21T04:15:24.767 INFO:tasks.workunit.client.1.smithi148.stdout:6/908: dread d0/d34/d97/d106/fab [0,4194304] 0 2024-09-21T04:15:24.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/758: link dc/d43/f47 dc/d13/d3c/d81/d98/dc1/feb 0 2024-09-21T04:15:24.770 INFO:tasks.workunit.client.1.smithi148.stdout:1/843: symlink db/d24/d2e/d71/dfc/l116 0 2024-09-21T04:15:24.772 INFO:tasks.workunit.client.1.smithi148.stdout:9/861: rename d5/d39/dba to d5/d60/d4c/d5e/d134 0 2024-09-21T04:15:24.773 INFO:tasks.workunit.client.1.smithi148.stdout:6/909: creat d0/d34/d97/d106/d23/dc/f11e x:0 0 0 2024-09-21T04:15:24.775 INFO:tasks.workunit.client.1.smithi148.stdout:8/759: creat dc/d13/d76/d50/db1/fec x:0 0 0 2024-09-21T04:15:24.775 INFO:tasks.workunit.client.1.smithi148.stdout:8/760: readlink dc/l27 0 2024-09-21T04:15:24.775 INFO:tasks.workunit.client.1.smithi148.stdout:8/761: dread - dc/d13/d39/fc6 zero size 2024-09-21T04:15:24.775 INFO:tasks.workunit.client.1.smithi148.stdout:0/844: dread d5/d6/d7/ff [0,4194304] 0 2024-09-21T04:15:24.775 INFO:tasks.workunit.client.1.smithi148.stdout:6/910: mkdir d0/d34/d38/d11f 0 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:0/845: mkdir d5/d6/d7/d11/d108 0 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:7/809: read d5/d14/d25/fc2 [936726,129475] 0 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:6/911: dread - d0/d34/d38/d43/f8a zero size 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:6/912: stat d0/d34/d97/d106/d23/d53/fc1 0 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:6/913: readlink d0/d34/d97/d106/d23/d32/l46 0 2024-09-21T04:15:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:6/914: creat d0/d34/f120 x:0 0 0 2024-09-21T04:15:24.778 INFO:tasks.workunit.client.1.smithi148.stdout:0/846: mknod d5/d6/d7/d11/c109 0 2024-09-21T04:15:24.778 INFO:tasks.workunit.client.1.smithi148.stdout:7/810: truncate d5/d8/ff 7063479 0 2024-09-21T04:15:24.778 INFO:tasks.workunit.client.1.smithi148.stdout:7/811: stat d5/d14/d25/d40/d7b/f9a 0 2024-09-21T04:15:24.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/936: write d0/ff [1771564,124552] 0 2024-09-21T04:15:24.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/937: stat d0/d9/d16/f5b 0 2024-09-21T04:15:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/938: stat d0/d9/d16/d55/c77 0 2024-09-21T04:15:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:9/862: dread d5/f1b [0,4194304] 0 2024-09-21T04:15:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/847: dread d5/d6/d2a/dcd/df8/da5/f86 [0,4194304] 0 2024-09-21T04:15:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/848: fsync d5/d6/d2a/d3f/d8b/dab/fc9 0 2024-09-21T04:15:24.786 INFO:tasks.workunit.client.1.smithi148.stdout:2/939: creat d0/d9/d100/f12a x:0 0 0 2024-09-21T04:15:24.788 INFO:tasks.workunit.client.0.smithi110.stdout:7/718: dread dd/d4d/f68 [0,4194304] 0 2024-09-21T04:15:24.788 INFO:tasks.workunit.client.1.smithi148.stdout:0/849: rename d5/l44 to d5/d6/d2a/dcd/df8/daf/dc8/de8/l10a 0 2024-09-21T04:15:24.790 INFO:tasks.workunit.client.1.smithi148.stdout:2/940: mknod d0/d9/dc/d13/daf/db2/c12b 0 2024-09-21T04:15:24.792 INFO:tasks.workunit.client.1.smithi148.stdout:0/850: creat d5/d6/d2a/dcd/df8/db4/f10b x:0 0 0 2024-09-21T04:15:24.792 INFO:tasks.workunit.client.1.smithi148.stdout:0/851: fdatasync d5/d6/d2a/d3f/f8d 0 2024-09-21T04:15:24.794 INFO:tasks.workunit.client.1.smithi148.stdout:2/941: rename d0/d9/d16/d55/d71/dc2/ld5 to d0/d2f/db1/l12c 0 2024-09-21T04:15:24.795 INFO:tasks.workunit.client.0.smithi110.stdout:3/652: dwrite d0/d12/d16/d1a/d5c/f79 [0,4194304] 0 2024-09-21T04:15:24.795 INFO:tasks.workunit.client.0.smithi110.stdout:7/719: mkdir dd/d1c/dad/db2/d56/d66/dea 0 2024-09-21T04:15:24.796 INFO:tasks.workunit.client.1.smithi148.stdout:2/942: symlink d0/d9/d59/d10b/l12d 0 2024-09-21T04:15:24.797 INFO:tasks.workunit.client.1.smithi148.stdout:2/943: creat d0/d9/dc/d13/d25/f12e x:0 0 0 2024-09-21T04:15:24.797 INFO:tasks.workunit.client.1.smithi148.stdout:2/944: write d0/d9/dc/d13/d25/f79 [3432496,27592] 0 2024-09-21T04:15:24.798 INFO:tasks.workunit.client.1.smithi148.stdout:0/852: rename d5/d6/d2a/d45/cbe to d5/d6/d2a/dcd/df8/daf/dc8/de8/df7/c10c 0 2024-09-21T04:15:24.799 INFO:tasks.workunit.client.0.smithi110.stdout:3/653: getdents d0/d12/d3c/dad 0 2024-09-21T04:15:24.800 INFO:tasks.workunit.client.1.smithi148.stdout:2/945: truncate d0/d9/d16/d55/d71/fa5 3889512 0 2024-09-21T04:15:24.802 INFO:tasks.workunit.client.1.smithi148.stdout:0/853: creat d5/d6/d2a/dcd/df8/daf/dc8/de8/df7/f10d x:0 0 0 2024-09-21T04:15:24.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/854: creat d5/d6/d7/d11/d48/f10e x:0 0 0 2024-09-21T04:15:24.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/855: readlink d5/d6/d7/dd0/lef 0 2024-09-21T04:15:24.806 INFO:tasks.workunit.client.1.smithi148.stdout:3/784: dwrite d3/d1f/d4d/d82/d23/d28/d46/db3/fea [0,4194304] 0 2024-09-21T04:15:24.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/905: dwrite d4/d87/f5c [4194304,4194304] 0 2024-09-21T04:15:24.821 INFO:tasks.workunit.client.1.smithi148.stdout:3/785: dread d3/d1f/d4d/d82/f11 [0,4194304] 0 2024-09-21T04:15:24.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/906: write d4/d66/d7f/f4a [886355,56860] 0 2024-09-21T04:15:24.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/786: mkdir d3/d1f/d4d/d82/d52/d106 0 2024-09-21T04:15:24.824 INFO:tasks.workunit.client.1.smithi148.stdout:3/787: creat d3/d1f/d4d/d82/d23/d28/d46/f107 x:0 0 0 2024-09-21T04:15:24.824 INFO:tasks.workunit.client.1.smithi148.stdout:3/788: chown f1 156449 1 2024-09-21T04:15:24.825 INFO:tasks.workunit.client.1.smithi148.stdout:3/789: creat d3/d1f/d4d/d82/d23/d28/f108 x:0 0 0 2024-09-21T04:15:24.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/654: write d0/d12/d16/f62 [896076,75233] 0 2024-09-21T04:15:24.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/655: creat d0/d12/fe4 x:0 0 0 2024-09-21T04:15:24.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/856: dwrite d5/f106 [0,4194304] 0 2024-09-21T04:15:24.829 INFO:tasks.workunit.client.1.smithi148.stdout:7/812: dwrite d5/d8/d6f/fec [0,4194304] 0 2024-09-21T04:15:24.829 INFO:tasks.workunit.client.1.smithi148.stdout:6/915: dwrite d0/d34/fd7 [0,4194304] 0 2024-09-21T04:15:24.830 INFO:tasks.workunit.client.1.smithi148.stdout:3/790: mkdir d3/d5/d6/d109 0 2024-09-21T04:15:24.830 INFO:tasks.workunit.client.1.smithi148.stdout:3/791: truncate d3/d5/d6/d2b/da2/da3/fc5 922040 0 2024-09-21T04:15:24.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/916: write d0/d34/d97/d106/d23/d32/d93/f9c [655856,8650] 0 2024-09-21T04:15:24.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/917: truncate d0/d34/d97/d106/d23/d32/d93/fca 813317 0 2024-09-21T04:15:24.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/656: mknod d0/d12/d2a/db4/ce5 0 2024-09-21T04:15:24.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/657: chown d0/d12/d16/d1a/dc6/fe0 1837654133 1 2024-09-21T04:15:24.835 INFO:tasks.workunit.client.1.smithi148.stdout:0/857: mknod d5/d6/d2a/d3f/d6d/da3/df4/c10f 0 2024-09-21T04:15:24.835 INFO:tasks.workunit.client.1.smithi148.stdout:8/762: dwrite dc/d13/d28/d62/f97 [0,4194304] 0 2024-09-21T04:15:24.839 INFO:tasks.workunit.client.0.smithi110.stdout:7/720: dwrite dd/fc6 [0,4194304] 0 2024-09-21T04:15:24.839 INFO:tasks.workunit.client.0.smithi110.stdout:7/721: write dd/d14/f41 [971881,10755] 0 2024-09-21T04:15:24.839 INFO:tasks.workunit.client.0.smithi110.stdout:7/722: getdents dd/d1c/dad/db2/d56/d66/da6/da7 0 2024-09-21T04:15:24.840 INFO:tasks.workunit.client.1.smithi148.stdout:2/946: dwrite d0/d2b/f32 [0,4194304] 0 2024-09-21T04:15:24.840 INFO:tasks.workunit.client.1.smithi148.stdout:2/947: stat d0/d2b/da3/d8f/d9f/dc9/fe9 0 2024-09-21T04:15:24.840 INFO:tasks.workunit.client.1.smithi148.stdout:2/948: creat d0/d9/d16/d9e/f12f x:0 0 0 2024-09-21T04:15:24.841 INFO:tasks.workunit.client.1.smithi148.stdout:3/792: mknod d3/d5/d6/d12/d43/d54/c10a 0 2024-09-21T04:15:24.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/863: dwrite d5/d1e/d37/d50/f55 [0,4194304] 0 2024-09-21T04:15:24.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/864: fsync d5/fa5 0 2024-09-21T04:15:24.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/865: write d5/d1e/f118 [816969,97390] 0 2024-09-21T04:15:24.846 INFO:tasks.workunit.client.1.smithi148.stdout:0/858: unlink d5/d6/d2a/c97 0 2024-09-21T04:15:24.846 INFO:tasks.workunit.client.1.smithi148.stdout:0/859: fdatasync d5/d6/d2a/dcd/df8/daf/dc8/fe3 0 2024-09-21T04:15:24.846 INFO:tasks.workunit.client.0.smithi110.stdout:3/658: dread d0/d12/d2a/f30 [0,4194304] 0 2024-09-21T04:15:24.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/659: write d0/d22/d29/f8b [220284,80676] 0 2024-09-21T04:15:24.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/660: read - d0/d12/d3c/d5a/d70/fcf zero size 2024-09-21T04:15:24.850 INFO:tasks.workunit.client.1.smithi148.stdout:6/918: dwrite d0/d34/d97/d106/d23/d27/d80/da4/fe7 [4194304,4194304] 0 2024-09-21T04:15:24.850 INFO:tasks.workunit.client.1.smithi148.stdout:6/919: write d0/d34/d38/d43/feb [264471,42384] 0 2024-09-21T04:15:24.851 INFO:tasks.workunit.client.0.smithi110.stdout:3/661: unlink d0/d22/f9b 0 2024-09-21T04:15:24.852 INFO:tasks.workunit.client.1.smithi148.stdout:8/763: truncate dc/d43/f44 1911767 0 2024-09-21T04:15:24.852 INFO:tasks.workunit.client.1.smithi148.stdout:2/949: write d0/d9/d16/d55/ffc [3142332,88643] 0 2024-09-21T04:15:24.852 INFO:tasks.workunit.client.1.smithi148.stdout:2/950: creat d0/d9/d16/f130 x:0 0 0 2024-09-21T04:15:24.855 INFO:tasks.workunit.client.0.smithi110.stdout:3/662: rename d0/d12/d3c/c3d to d0/d12/d16/d1a/ce6 0 2024-09-21T04:15:24.855 INFO:tasks.workunit.client.0.smithi110.stdout:3/663: dread - d0/d12/d3c/dad/fb7 zero size 2024-09-21T04:15:24.855 INFO:tasks.workunit.client.0.smithi110.stdout:3/664: write d0/d12/d2a/f61 [931150,89018] 0 2024-09-21T04:15:24.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/793: mkdir d3/d5/d6/d2b/d96/db8/d10b 0 2024-09-21T04:15:24.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/794: read - d3/d5/fe5 zero size 2024-09-21T04:15:24.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/795: readlink d3/d1f/d4d/d82/d52/l2a 0 2024-09-21T04:15:24.859 INFO:tasks.workunit.client.0.smithi110.stdout:3/665: rename d0/d12/d3c/d5a/d65/lb2 to d0/d22/d29/le7 0 2024-09-21T04:15:24.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/666: fsync d0/d12/d16/d1a/f35 0 2024-09-21T04:15:24.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/667: write d0/d22/d29/fc5 [1889031,121886] 0 2024-09-21T04:15:24.863 INFO:tasks.workunit.client.0.smithi110.stdout:7/723: dwrite dd/d14/f73 [0,4194304] 0 2024-09-21T04:15:24.863 INFO:tasks.workunit.client.0.smithi110.stdout:7/724: stat dd/d48/fc8 0 2024-09-21T04:15:24.864 INFO:tasks.workunit.client.1.smithi148.stdout:2/951: dread d0/d9/dc/d13/d50/f70 [0,4194304] 0 2024-09-21T04:15:24.864 INFO:tasks.workunit.client.1.smithi148.stdout:9/866: dwrite d5/d2b/dbe/f9a [0,4194304] 0 2024-09-21T04:15:24.864 INFO:tasks.workunit.client.0.smithi110.stdout:3/668: getdents d0/d12/d3c/dce 0 2024-09-21T04:15:24.867 INFO:tasks.workunit.client.1.smithi148.stdout:0/860: write d5/d6/d2a/d2b/f35 [1730404,63671] 0 2024-09-21T04:15:24.867 INFO:tasks.workunit.client.1.smithi148.stdout:0/861: chown d5/d6/d7/d22/l105 0 1 2024-09-21T04:15:24.869 INFO:tasks.workunit.client.0.smithi110.stdout:7/725: mknod dd/d1c/dad/db2/d56/d66/da6/ceb 0 2024-09-21T04:15:24.869 INFO:tasks.workunit.client.0.smithi110.stdout:7/726: fsync dd/d48/d9f/fdb 0 2024-09-21T04:15:24.869 INFO:tasks.workunit.client.1.smithi148.stdout:5/841: sync 2024-09-21T04:15:24.869 INFO:tasks.workunit.client.1.smithi148.stdout:5/842: chown d3/d14/d1f/d72/dc9/dd8 118208024 1 2024-09-21T04:15:24.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/920: rename d0/df9/d116/d89 to d0/d8/dba/d113/d11d/d121 0 2024-09-21T04:15:24.871 INFO:tasks.workunit.client.1.smithi148.stdout:6/921: write d0/d34/d97/d106/d23/d27/d80/db0/fff [369167,117139] 0 2024-09-21T04:15:24.871 INFO:tasks.workunit.client.1.smithi148.stdout:6/922: stat d0/d34/d97/d106/d23/dc 0 2024-09-21T04:15:24.874 INFO:tasks.workunit.client.1.smithi148.stdout:3/796: unlink d3/d5/d6/d12/d15/leb 0 2024-09-21T04:15:24.877 INFO:tasks.workunit.client.1.smithi148.stdout:2/952: getdents d0/d9/d16/d55 0 2024-09-21T04:15:24.877 INFO:tasks.workunit.client.0.smithi110.stdout:8/765: sync 2024-09-21T04:15:24.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/843: mknod d3/d14/d1a/c120 0 2024-09-21T04:15:24.882 INFO:tasks.workunit.client.0.smithi110.stdout:7/727: creat dd/d1c/dad/db2/d6d/ddc/fec x:0 0 0 2024-09-21T04:15:24.882 INFO:tasks.workunit.client.0.smithi110.stdout:7/728: creat dd/d1c/dad/db2/d6d/dbc/fed x:0 0 0 2024-09-21T04:15:24.883 INFO:tasks.workunit.client.1.smithi148.stdout:0/862: dwrite d5/d82/f104 [0,4194304] 0 2024-09-21T04:15:24.883 INFO:tasks.workunit.client.0.smithi110.stdout:3/669: rmdir d0/d12/d2a/d80/da2/db0 39 2024-09-21T04:15:24.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/670: write d0/d22/d29/f8b [957386,65186] 0 2024-09-21T04:15:24.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/671: write d0/d12/d3c/d5a/dc8/fdf [5181051,86885] 0 2024-09-21T04:15:24.886 INFO:tasks.workunit.client.0.smithi110.stdout:8/766: unlink d0/d30/d34/d3e/d5d/cee 0 2024-09-21T04:15:24.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/764: rename dc/l2d to dc/d43/d4c/led 0 2024-09-21T04:15:24.890 INFO:tasks.workunit.client.1.smithi148.stdout:6/923: mknod d0/d34/d97/d106/d23/d27/d80/da4/df1/d115/c122 0 2024-09-21T04:15:24.892 INFO:tasks.workunit.client.1.smithi148.stdout:3/797: mknod d3/d5/db4/def/c10c 0 2024-09-21T04:15:24.894 INFO:tasks.workunit.client.1.smithi148.stdout:2/953: symlink d0/d9/d59/l131 0 2024-09-21T04:15:24.895 INFO:tasks.workunit.client.0.smithi110.stdout:3/672: creat d0/d12/d3c/fe8 x:0 0 0 2024-09-21T04:15:24.897 INFO:tasks.workunit.client.1.smithi148.stdout:5/844: mkdir d3/d34/d3c/d5d/d75/d121 0 2024-09-21T04:15:24.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/924: dread d0/d34/d97/d106/d23/d27/d62/d9a/fa3 [0,4194304] 0 2024-09-21T04:15:24.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/925: chown d0/d34/d97/d106/d23/d32/d93/cf8 2583191 1 2024-09-21T04:15:24.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/926: chown d0/d34/d97/d106/d23/d27/d80/da4/df1/d115/c122 17935906 1 2024-09-21T04:15:24.904 INFO:tasks.workunit.client.0.smithi110.stdout:8/767: dwrite d0/d7/d13/fbc [0,4194304] 0 2024-09-21T04:15:24.904 INFO:tasks.workunit.client.1.smithi148.stdout:2/954: read d0/d9/d16/d55/d71/dc2/dcb/fd6 [3552514,39873] 0 2024-09-21T04:15:24.905 INFO:tasks.workunit.client.0.smithi110.stdout:3/673: creat d0/d12/d2a/d80/da2/da7/fe9 x:0 0 0 2024-09-21T04:15:24.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/674: link d0/d12/d2a/d80/d9c/ld0 d0/d12/d3c/d5a/lea 0 2024-09-21T04:15:24.911 INFO:tasks.workunit.client.1.smithi148.stdout:8/765: creat dc/d13/d3c/d81/d98/de5/fee x:0 0 0 2024-09-21T04:15:24.912 INFO:tasks.workunit.client.1.smithi148.stdout:8/766: truncate dc/d13/d76/d50/db1/fec 897732 0 2024-09-21T04:15:24.912 INFO:tasks.workunit.client.0.smithi110.stdout:4/901: sync 2024-09-21T04:15:24.913 INFO:tasks.workunit.client.0.smithi110.stdout:6/762: sync 2024-09-21T04:15:24.913 INFO:tasks.workunit.client.0.smithi110.stdout:3/675: truncate d0/d12/d2a/f30 2563262 0 2024-09-21T04:15:24.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/798: mkdir d3/d5/d6/d2b/d96/d10d 0 2024-09-21T04:15:24.915 INFO:tasks.workunit.client.0.smithi110.stdout:4/902: mkdir d17/d1e/d77/d10c/d20/d43/d51/d52/d132 0 2024-09-21T04:15:24.915 INFO:tasks.workunit.client.0.smithi110.stdout:4/903: write d17/d1e/d7b/daf/fff [967822,128121] 0 2024-09-21T04:15:24.916 INFO:tasks.workunit.client.1.smithi148.stdout:6/927: mkdir d0/d8/dba/dbc/d123 0 2024-09-21T04:15:24.917 INFO:tasks.workunit.client.0.smithi110.stdout:6/763: mkdir d4/d12/d17/d9a/dfa 0 2024-09-21T04:15:24.917 INFO:tasks.workunit.client.0.smithi110.stdout:6/764: dread - d4/d12/d17/d2e/d40/fd8 zero size 2024-09-21T04:15:24.918 INFO:tasks.workunit.client.1.smithi148.stdout:2/955: mkdir d0/d9/d59/dda/d129/d132 0 2024-09-21T04:15:24.918 INFO:tasks.workunit.client.0.smithi110.stdout:3/676: rmdir d0/d12/d3c/d5a 39 2024-09-21T04:15:24.922 INFO:tasks.workunit.client.1.smithi148.stdout:5/845: dwrite d3/d34/d78/dbf/fd1 [0,4194304] 0 2024-09-21T04:15:24.922 INFO:tasks.workunit.client.1.smithi148.stdout:8/767: dread dc/d13/d76/f2c [0,4194304] 0 2024-09-21T04:15:24.922 INFO:tasks.workunit.client.1.smithi148.stdout:8/768: chown dc/fa8 2443827 1 2024-09-21T04:15:24.922 INFO:tasks.workunit.client.1.smithi148.stdout:8/769: chown dc/d13/d6a/l7b 2246 1 2024-09-21T04:15:24.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/904: creat d17/d1e/d77/d10c/d20/dca/df3/f133 x:0 0 0 2024-09-21T04:15:24.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/905: dread - d17/d1e/d77/f97 zero size 2024-09-21T04:15:24.924 INFO:tasks.workunit.client.1.smithi148.stdout:6/928: symlink d0/d34/d38/d57/d58/l124 0 2024-09-21T04:15:24.924 INFO:tasks.workunit.client.1.smithi148.stdout:6/929: write d0/d34/d97/d106/d23/dc/d68/f86 [2282337,90780] 0 2024-09-21T04:15:24.925 INFO:tasks.workunit.client.0.smithi110.stdout:6/765: mkdir d4/d12/d17/d2e/dfb 0 2024-09-21T04:15:24.925 INFO:tasks.workunit.client.0.smithi110.stdout:6/766: creat d4/d12/d16/d66/ffc x:0 0 0 2024-09-21T04:15:24.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/767: chown d4/d86/lc0 24 1 2024-09-21T04:15:24.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/768: creat d4/d78/daa/ffd x:0 0 0 2024-09-21T04:15:24.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/769: getdents d4/d12/d17 0 2024-09-21T04:15:24.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/770: read - d4/d86/d8f/fc7 zero size 2024-09-21T04:15:24.927 INFO:tasks.workunit.client.1.smithi148.stdout:2/956: creat d0/d9/d16/d55/d71/f133 x:0 0 0 2024-09-21T04:15:24.928 INFO:tasks.workunit.client.1.smithi148.stdout:9/867: rename d5/d2b/d4e/d95 to d5/d1e/d37/d50/d135 0 2024-09-21T04:15:24.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/868: chown d5/f117 222123885 1 2024-09-21T04:15:24.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/869: fsync d5/d60/f45 0 2024-09-21T04:15:24.930 INFO:tasks.workunit.client.0.smithi110.stdout:7/729: rename dd/d1c/dad/db2/d6d to dd/d4d/dee 0 2024-09-21T04:15:24.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/846: dread d3/d24/f76 [4194304,4194304] 0 2024-09-21T04:15:24.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/847: dread - d3/d34/ff7 zero size 2024-09-21T04:15:24.934 INFO:tasks.workunit.client.1.smithi148.stdout:3/799: dwrite d3/d1f/d4d/d82/d23/db5/d105/fce [0,4194304] 0 2024-09-21T04:15:24.934 INFO:tasks.workunit.client.1.smithi148.stdout:3/800: creat d3/d5/d5c/f10e x:0 0 0 2024-09-21T04:15:24.935 INFO:tasks.workunit.client.0.smithi110.stdout:3/677: getdents d0/d22 0 2024-09-21T04:15:24.936 INFO:tasks.workunit.client.0.smithi110.stdout:6/771: write d4/d12/d16/d66/d7a/d9e/ff2 [2199222,74449] 0 2024-09-21T04:15:24.938 INFO:tasks.workunit.client.0.smithi110.stdout:4/906: write d17/d1e/d21/de1/ff1 [3802101,84817] 0 2024-09-21T04:15:24.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/730: write dd/d14/d17/f3a [5118684,35897] 0 2024-09-21T04:15:24.940 INFO:tasks.workunit.client.0.smithi110.stdout:8/768: rename d0/d30/d54/d6b/da8/cd9 to d0/d30/d34/d3e/d100/c116 0 2024-09-21T04:15:24.944 INFO:tasks.workunit.client.0.smithi110.stdout:3/678: creat d0/d12/d3c/d5a/d65/feb x:0 0 0 2024-09-21T04:15:24.944 INFO:tasks.workunit.client.0.smithi110.stdout:3/679: fsync d0/d12/d16/d1a/d95/fc9 0 2024-09-21T04:15:24.944 INFO:tasks.workunit.client.0.smithi110.stdout:3/680: readlink d0/d12/d16/d1a/l42 0 2024-09-21T04:15:24.945 INFO:tasks.workunit.client.1.smithi148.stdout:9/870: read d5/d1f/d81/f10c [3708565,108612] 0 2024-09-21T04:15:24.945 INFO:tasks.workunit.client.1.smithi148.stdout:8/770: creat dc/d13/dbd/fef x:0 0 0 2024-09-21T04:15:24.949 INFO:tasks.workunit.client.1.smithi148.stdout:8/771: dread dc/d13/d76/d5d/fa7 [0,4194304] 0 2024-09-21T04:15:24.950 INFO:tasks.workunit.client.1.smithi148.stdout:6/930: creat d0/d34/d97/d106/d23/d27/d62/dfb/f125 x:0 0 0 2024-09-21T04:15:24.950 INFO:tasks.workunit.client.1.smithi148.stdout:6/931: dread - d0/d34/d38/dc6/fce zero size 2024-09-21T04:15:24.952 INFO:tasks.workunit.client.1.smithi148.stdout:2/957: mknod d0/d9/dc/d13/d4c/c134 0 2024-09-21T04:15:24.957 INFO:tasks.workunit.client.0.smithi110.stdout:7/731: dwrite dd/d1c/dad/db2/d56/d66/da6/f8e [0,4194304] 0 2024-09-21T04:15:24.959 INFO:tasks.workunit.client.0.smithi110.stdout:6/772: dwrite d4/f79 [0,4194304] 0 2024-09-21T04:15:24.959 INFO:tasks.workunit.client.0.smithi110.stdout:6/773: truncate d4/d12/d17/d2e/d40/fd8 330042 0 2024-09-21T04:15:24.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/772: dread dc/d13/fe0 [0,4194304] 0 2024-09-21T04:15:24.961 INFO:tasks.workunit.client.1.smithi148.stdout:8/773: stat dc/d13/dbd/fef 0 2024-09-21T04:15:24.961 INFO:tasks.workunit.client.1.smithi148.stdout:1/844: sync 2024-09-21T04:15:24.962 INFO:tasks.workunit.client.1.smithi148.stdout:4/907: sync 2024-09-21T04:15:24.964 INFO:tasks.workunit.client.0.smithi110.stdout:4/907: creat d17/d1e/d77/dcf/d12b/f134 x:0 0 0 2024-09-21T04:15:24.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/801: mknod d3/d5/d6/d12/d43/d54/c10f 0 2024-09-21T04:15:24.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/802: write d3/d1f/f3e [1748344,23637] 0 2024-09-21T04:15:24.965 INFO:tasks.workunit.client.1.smithi148.stdout:8/774: read dc/f21 [705592,124777] 0 2024-09-21T04:15:24.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/871: truncate d5/d60/d28/fc7 164874 0 2024-09-21T04:15:24.967 INFO:tasks.workunit.client.0.smithi110.stdout:3/681: symlink d0/d22/lec 0 2024-09-21T04:15:24.969 INFO:tasks.workunit.client.1.smithi148.stdout:6/932: mknod d0/d34/d97/d106/d23/d53/c126 0 2024-09-21T04:15:24.969 INFO:tasks.workunit.client.0.smithi110.stdout:3/682: dread d0/d12/f9d [0,4194304] 0 2024-09-21T04:15:24.974 INFO:tasks.workunit.client.0.smithi110.stdout:4/908: dread d17/d1e/d77/d10c/d20/d43/fe3 [0,4194304] 0 2024-09-21T04:15:24.974 INFO:tasks.workunit.client.0.smithi110.stdout:4/909: write d17/d1e/d77/d10c/d20/dca/df3/f133 [465679,82839] 0 2024-09-21T04:15:24.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/958: dwrite d0/fff [0,4194304] 0 2024-09-21T04:15:24.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/959: write d0/d9/dc/d13/daf/db2/ff9 [988432,39378] 0 2024-09-21T04:15:24.975 INFO:tasks.workunit.client.1.smithi148.stdout:2/960: write d0/d9/dc/d13/daf/db2/ff9 [1763073,119879] 0 2024-09-21T04:15:24.975 INFO:tasks.workunit.client.0.smithi110.stdout:7/732: truncate dd/d1c/dad/db2/d56/d66/da6/d98/fdf 141865 0 2024-09-21T04:15:24.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/582: sync 2024-09-21T04:15:24.977 INFO:tasks.workunit.client.0.smithi110.stdout:5/800: sync 2024-09-21T04:15:24.977 INFO:tasks.workunit.client.0.smithi110.stdout:9/703: sync 2024-09-21T04:15:24.977 INFO:tasks.workunit.client.0.smithi110.stdout:9/704: creat d7/d96/d99/d3c/d14/ff0 x:0 0 0 2024-09-21T04:15:24.978 INFO:tasks.workunit.client.1.smithi148.stdout:8/775: dread dc/d13/d3c/d81/d85/f87 [0,4194304] 0 2024-09-21T04:15:24.979 INFO:tasks.workunit.client.0.smithi110.stdout:8/769: rmdir d0/d7/d23/d52 39 2024-09-21T04:15:24.979 INFO:tasks.workunit.client.0.smithi110.stdout:8/770: chown d0/d7/d23/d82/cbb 21 1 2024-09-21T04:15:24.980 INFO:tasks.workunit.client.0.smithi110.stdout:3/683: creat d0/d22/dd2/fed x:0 0 0 2024-09-21T04:15:24.980 INFO:tasks.workunit.client.0.smithi110.stdout:3/684: stat d0/d12/d2a/d80/d9c/lc2 0 2024-09-21T04:15:24.983 INFO:tasks.workunit.client.1.smithi148.stdout:3/803: dwrite d3/d1f/d4d/d82/d52/fab [0,4194304] 0 2024-09-21T04:15:24.984 INFO:tasks.workunit.client.1.smithi148.stdout:3/804: chown d3/d1f/d4d/d82/d23/fba 164710 1 2024-09-21T04:15:24.984 INFO:tasks.workunit.client.1.smithi148.stdout:3/805: truncate d3/d1f/f8e 1837426 0 2024-09-21T04:15:24.984 INFO:tasks.workunit.client.0.smithi110.stdout:4/910: symlink d17/d3e/d6b/def/l135 0 2024-09-21T04:15:24.984 INFO:tasks.workunit.client.0.smithi110.stdout:4/911: write d17/d1e/d21/f4d [220904,57009] 0 2024-09-21T04:15:24.985 INFO:tasks.workunit.client.0.smithi110.stdout:6/774: write d4/d12/d16/d66/d7a/d83/fa3 [6723452,65589] 0 2024-09-21T04:15:24.986 INFO:tasks.workunit.client.0.smithi110.stdout:2/583: creat d2/d1a/da3/fb1 x:0 0 0 2024-09-21T04:15:24.987 INFO:tasks.workunit.client.0.smithi110.stdout:7/733: write dd/fc6 [2242808,102045] 0 2024-09-21T04:15:24.987 INFO:tasks.workunit.client.0.smithi110.stdout:7/734: creat dd/d48/d9f/fef x:0 0 0 2024-09-21T04:15:24.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/705: mknod d7/d96/d99/d3c/d35/d71/cf1 0 2024-09-21T04:15:24.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/706: fsync d7/d26/fb1 0 2024-09-21T04:15:24.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/707: truncate d7/d96/f9a 1938110 0 2024-09-21T04:15:24.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/806: dread d3/d5/d6/d12/d43/d54/f6a [0,4194304] 0 2024-09-21T04:15:24.997 INFO:tasks.workunit.client.0.smithi110.stdout:5/801: dwrite d1/d49/d4d/d1e/d4a/dd4/dd5/f118 [0,4194304] 0 2024-09-21T04:15:25.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/708: write d7/d96/d99/d3c/d1f/d43/f60 [2138007,112545] 0 2024-09-21T04:15:25.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/709: write d7/d96/d99/d3c/d14/d31/f53 [964604,14724] 0 2024-09-21T04:15:25.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/710: read d7/d96/d99/d3c/d1f/d43/f6c [849364,4543] 0 2024-09-21T04:15:25.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/711: truncate d7/d96/d99/d3c/d11/f81 4685436 0 2024-09-21T04:15:25.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/712: creat d7/d96/ff2 x:0 0 0 2024-09-21T04:15:25.012 INFO:tasks.workunit.client.1.smithi148.stdout:3/807: dread d3/d1f/d4d/d82/d23/d28/d46/f9f [0,4194304] 0 2024-09-21T04:15:25.015 INFO:tasks.workunit.client.0.smithi110.stdout:7/735: dwrite dd/f90 [0,4194304] 0 2024-09-21T04:15:25.015 INFO:tasks.workunit.client.0.smithi110.stdout:7/736: truncate dd/d48/d9f/fef 431430 0 2024-09-21T04:15:25.016 INFO:tasks.workunit.client.0.smithi110.stdout:5/802: dwrite d1/d22/d51/f98 [0,4194304] 0 2024-09-21T04:15:25.016 INFO:tasks.workunit.client.0.smithi110.stdout:5/803: creat d1/d39/f119 x:0 0 0 2024-09-21T04:15:25.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/804: write d1/dc/f33 [270214,100350] 0 2024-09-21T04:15:25.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/805: chown d1/d22/d6e 601023 1 2024-09-21T04:15:25.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/806: fsync d1/d49/d4d/d1e/f3b 0 2024-09-21T04:15:25.059 INFO:tasks.workunit.client.0.smithi110.stdout:8/771: mkdir d0/d30/d34/d3e/d32/da3/d117 0 2024-09-21T04:15:25.061 INFO:tasks.workunit.client.0.smithi110.stdout:3/685: truncate d0/d12/d3c/d5a/d70/f97 3405919 0 2024-09-21T04:15:25.064 INFO:tasks.workunit.client.0.smithi110.stdout:4/912: creat d17/d1e/d64/f136 x:0 0 0 2024-09-21T04:15:25.071 INFO:tasks.workunit.client.0.smithi110.stdout:3/686: write d0/d12/d3c/f4f [2357562,53280] 0 2024-09-21T04:15:25.071 INFO:tasks.workunit.client.0.smithi110.stdout:3/687: dread - d0/d22/dd2/fed zero size 2024-09-21T04:15:25.072 INFO:tasks.workunit.client.0.smithi110.stdout:3/688: write d0/d12/d3c/f7f [273437,9674] 0 2024-09-21T04:15:25.072 INFO:tasks.workunit.client.0.smithi110.stdout:6/775: readlink d4/d12/d16/d66/d7a/d83/la9 0 2024-09-21T04:15:25.072 INFO:tasks.workunit.client.0.smithi110.stdout:6/776: write d4/d12/d16/d66/ffc [439432,93490] 0 2024-09-21T04:15:25.075 INFO:tasks.workunit.client.0.smithi110.stdout:2/584: truncate d2/d1a/f88 3116929 0 2024-09-21T04:15:25.079 INFO:tasks.workunit.client.1.smithi148.stdout:4/908: mkdir d4/d12/d78/de4/d10c/d11c 0 2024-09-21T04:15:25.081 INFO:tasks.workunit.client.1.smithi148.stdout:1/845: creat db/dd/d21/db0/f117 x:0 0 0 2024-09-21T04:15:25.082 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:24 smithi148 ceph-mon[34957]: pgmap v9: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 252 MiB/s rd, 309 MiB/s wr, 460 op/s 2024-09-21T04:15:25.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/933: mkdir d0/d34/d97/d106/d23/d27/d80/d110/d127 0 2024-09-21T04:15:25.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/934: readlink d0/d34/d97/d106/d23/dc/d68/l9e 0 2024-09-21T04:15:25.088 INFO:tasks.workunit.client.0.smithi110.stdout:3/689: dwrite d0/fd3 [0,4194304] 0 2024-09-21T04:15:25.088 INFO:tasks.workunit.client.0.smithi110.stdout:3/690: truncate d0/d12/d16/d1a/d95/fc9 562250 0 2024-09-21T04:15:25.088 INFO:tasks.workunit.client.0.smithi110.stdout:3/691: truncate d0/d12/d3c/dce/fe1 672371 0 2024-09-21T04:15:25.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/713: rmdir d7/d96/d99/d3c/d11/dc1 0 2024-09-21T04:15:25.104 INFO:tasks.workunit.client.1.smithi148.stdout:3/808: creat d3/d5/d6/d2b/d96/db8/f110 x:0 0 0 2024-09-21T04:15:25.106 INFO:tasks.workunit.client.1.smithi148.stdout:3/809: dread d3/f85 [0,4194304] 0 2024-09-21T04:15:25.115 INFO:tasks.workunit.client.0.smithi110.stdout:7/737: mknod dd/d14/d17/d8a/cf0 0 2024-09-21T04:15:25.122 INFO:tasks.workunit.client.0.smithi110.stdout:7/738: write dd/d14/f73 [4160484,20369] 0 2024-09-21T04:15:25.123 INFO:tasks.workunit.client.0.smithi110.stdout:5/807: mknod d1/d22/db6/c11a 0 2024-09-21T04:15:25.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/810: dwrite d3/d5/d6/d2b/da2/da3/de4/ffc [0,4194304] 0 2024-09-21T04:15:25.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:24 smithi110 ceph-mon[27723]: pgmap v9: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 252 MiB/s rd, 309 MiB/s wr, 460 op/s 2024-09-21T04:15:25.155 INFO:tasks.workunit.client.1.smithi148.stdout:8/776: mkdir dc/d13/d76/d50/df0 0 2024-09-21T04:15:25.156 INFO:tasks.workunit.client.1.smithi148.stdout:8/777: chown dc/f14 61 1 2024-09-21T04:15:25.180 INFO:tasks.workunit.client.1.smithi148.stdout:2/961: creat d0/d2b/da3/d8f/f135 x:0 0 0 2024-09-21T04:15:25.217 INFO:tasks.workunit.client.0.smithi110.stdout:8/772: creat d0/d30/d54/d10c/f118 x:0 0 0 2024-09-21T04:15:25.219 INFO:tasks.workunit.client.0.smithi110.stdout:4/913: truncate d17/d1e/d64/f8f 10950 0 2024-09-21T04:15:25.219 INFO:tasks.workunit.client.0.smithi110.stdout:4/914: fsync d17/d34/fb0 0 2024-09-21T04:15:25.219 INFO:tasks.workunit.client.0.smithi110.stdout:4/915: dread - d17/d1e/d77/d10c/d20/d43/fe2 zero size 2024-09-21T04:15:25.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/863: rename d5/d6 to d5/dcb/de6/d110 0 2024-09-21T04:15:25.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/864: write d5/dcb/de6/d110/d2a/d3f/d6d/da3/fda [179454,116365] 0 2024-09-21T04:15:25.223 INFO:tasks.workunit.client.0.smithi110.stdout:2/585: mknod d2/d1a/d1f/cb2 0 2024-09-21T04:15:25.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/909: rmdir d4/d12/d83/d103/d113 0 2024-09-21T04:15:25.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/910: creat d4/d87/d2d/f11d x:0 0 0 2024-09-21T04:15:25.225 INFO:tasks.workunit.client.1.smithi148.stdout:4/911: truncate d4/d12/d16/d29/f5a 8140069 0 2024-09-21T04:15:25.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/846: mkdir db/d24/d2e/d59/da7/dc1/dd2/d118 0 2024-09-21T04:15:25.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/847: dread - db/dd/ddd/f10b zero size 2024-09-21T04:15:25.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/848: creat db/d24/d2e/d71/d94/dde/f119 x:0 0 0 2024-09-21T04:15:25.227 INFO:tasks.workunit.client.1.smithi148.stdout:1/849: write db/d24/d2e/d59/f7e [4371520,98678] 0 2024-09-21T04:15:25.230 INFO:tasks.workunit.client.0.smithi110.stdout:3/692: unlink d0/d12/d16/d1a/d95/ca5 0 2024-09-21T04:15:25.230 INFO:tasks.workunit.client.0.smithi110.stdout:3/693: chown d0/d12/d16 116333 1 2024-09-21T04:15:25.231 INFO:tasks.workunit.client.1.smithi148.stdout:6/935: creat d0/d34/d38/d43/d63/d11b/f128 x:0 0 0 2024-09-21T04:15:25.235 INFO:tasks.workunit.client.0.smithi110.stdout:9/714: mknod d7/d96/d99/d3c/d1f/cf3 0 2024-09-21T04:15:25.237 INFO:tasks.workunit.client.0.smithi110.stdout:7/739: creat dd/db1/ff1 x:0 0 0 2024-09-21T04:15:25.239 INFO:tasks.workunit.client.0.smithi110.stdout:5/808: creat d1/d7/d18/d3f/d117/f11b x:0 0 0 2024-09-21T04:15:25.241 INFO:tasks.workunit.client.1.smithi148.stdout:3/811: symlink d3/d5/dac/l111 0 2024-09-21T04:15:25.241 INFO:tasks.workunit.client.1.smithi148.stdout:8/778: truncate dc/d13/d76/d5d/f83 2571566 0 2024-09-21T04:15:25.241 INFO:tasks.workunit.client.1.smithi148.stdout:8/779: chown dc/d43/d4c/d4f/fce 223783 1 2024-09-21T04:15:25.242 INFO:tasks.workunit.client.0.smithi110.stdout:8/773: dwrite d0/d30/d34/d3e/d32/f7c [0,4194304] 0 2024-09-21T04:15:25.242 INFO:tasks.workunit.client.1.smithi148.stdout:0/865: dwrite d5/dcb/de6/d110/d2a/d3f/d6d/da3/fae [0,4194304] 0 2024-09-21T04:15:25.244 INFO:tasks.workunit.client.0.smithi110.stdout:4/916: truncate d17/d1e/d77/d10c/fa3 8297088 0 2024-09-21T04:15:25.247 INFO:tasks.workunit.client.0.smithi110.stdout:2/586: symlink d2/d1a/d8b/lb3 0 2024-09-21T04:15:25.254 INFO:tasks.workunit.client.0.smithi110.stdout:7/740: mknod dd/d14/d17/d8a/dd2/cf2 0 2024-09-21T04:15:25.255 INFO:tasks.workunit.client.1.smithi148.stdout:2/962: symlink d0/d9/dc/d13/d4c/l136 0 2024-09-21T04:15:25.256 INFO:tasks.workunit.client.1.smithi148.stdout:1/850: link db/d46/d4c/da3/c102 db/d24/d2a/dbd/df5/c11a 0 2024-09-21T04:15:25.257 INFO:tasks.workunit.client.1.smithi148.stdout:4/912: mknod d4/d66/c11e 0 2024-09-21T04:15:25.257 INFO:tasks.workunit.client.1.smithi148.stdout:4/913: readlink d4/d12/d16/d29/d61/d6c/l84 0 2024-09-21T04:15:25.257 INFO:tasks.workunit.client.0.smithi110.stdout:5/809: link d1/d49/d4d/d1e/d4a/dd4/cdb d1/d7/d58/d9e/d116/c11c 0 2024-09-21T04:15:25.260 INFO:tasks.workunit.client.0.smithi110.stdout:4/917: read d17/d1e/d77/d10c/d20/d43/d51/d52/d98/f99 [6494619,100081] 0 2024-09-21T04:15:25.260 INFO:tasks.workunit.client.1.smithi148.stdout:6/936: symlink d0/d34/d97/d106/l129 0 2024-09-21T04:15:25.262 INFO:tasks.workunit.client.0.smithi110.stdout:2/587: creat d2/d1a/d1f/d5f/d9d/d86/d9b/fb4 x:0 0 0 2024-09-21T04:15:25.262 INFO:tasks.workunit.client.0.smithi110.stdout:2/588: write d2/d50/d56/f94 [184899,34257] 0 2024-09-21T04:15:25.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/812: creat d3/d5/d6/d2b/d96/db8/df4/f112 x:0 0 0 2024-09-21T04:15:25.268 INFO:tasks.workunit.client.0.smithi110.stdout:5/810: link d1/d49/d4d/d1e/d4a/dd4/dd5/f113 d1/d49/d4d/d1e/d4a/dd4/dd5/f11d 0 2024-09-21T04:15:25.268 INFO:tasks.workunit.client.0.smithi110.stdout:5/811: write d1/d49/d4d/d1e/f5f [4958986,25594] 0 2024-09-21T04:15:25.269 INFO:tasks.workunit.client.0.smithi110.stdout:8/774: dwrite d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:25.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/780: dwrite dc/d13/d76/d42/f90 [0,4194304] 0 2024-09-21T04:15:25.271 INFO:tasks.workunit.client.0.smithi110.stdout:4/918: symlink d17/d3e/dbd/l137 0 2024-09-21T04:15:25.271 INFO:tasks.workunit.client.0.smithi110.stdout:4/919: read - d17/d1e/d77/d10c/d57/db9/f11f zero size 2024-09-21T04:15:25.273 INFO:tasks.workunit.client.0.smithi110.stdout:2/589: link d2/d1a/d1f/d29/d19/l40 d2/d1a/d8b/lb5 0 2024-09-21T04:15:25.276 INFO:tasks.workunit.client.1.smithi148.stdout:1/851: mknod db/d24/c11b 0 2024-09-21T04:15:25.277 INFO:tasks.workunit.client.0.smithi110.stdout:5/812: truncate d1/d7/d58/f60 1206051 0 2024-09-21T04:15:25.277 INFO:tasks.workunit.client.0.smithi110.stdout:8/775: write d0/d7/d23/f33 [2168691,21824] 0 2024-09-21T04:15:25.278 INFO:tasks.workunit.client.1.smithi148.stdout:4/914: mkdir d4/d12/d16/d29/d61/d6c/df1/d11f 0 2024-09-21T04:15:25.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/781: dread dc/d13/d76/d42/f71 [0,4194304] 0 2024-09-21T04:15:25.280 INFO:tasks.workunit.client.1.smithi148.stdout:6/937: truncate d0/d8/f49 3041672 0 2024-09-21T04:15:25.280 INFO:tasks.workunit.client.1.smithi148.stdout:6/938: fdatasync d0/d34/d97/d106/d23/d27/fe1 0 2024-09-21T04:15:25.280 INFO:tasks.workunit.client.1.smithi148.stdout:6/939: write d0/d8/f114 [823026,37642] 0 2024-09-21T04:15:25.281 INFO:tasks.workunit.client.0.smithi110.stdout:4/920: rename d17/d1e/d21/l76 to d17/d1e/d64/l138 0 2024-09-21T04:15:25.282 INFO:tasks.workunit.client.1.smithi148.stdout:2/963: dwrite d0/d9/f6e [0,4194304] 0 2024-09-21T04:15:25.282 INFO:tasks.workunit.client.1.smithi148.stdout:3/813: mkdir d3/d5/db4/d113 0 2024-09-21T04:15:25.283 INFO:tasks.workunit.client.0.smithi110.stdout:2/590: mkdir d2/d50/d9c/db6 0 2024-09-21T04:15:25.284 INFO:tasks.workunit.client.0.smithi110.stdout:2/591: chown d2/d1a/d90/laf 41339 1 2024-09-21T04:15:25.284 INFO:tasks.workunit.client.0.smithi110.stdout:2/592: chown d2/d1a/d1f/d5f/f1d 8880 1 2024-09-21T04:15:25.285 INFO:tasks.workunit.client.0.smithi110.stdout:6/777: mkdir d4/d12/d17/dfe 0 2024-09-21T04:15:25.285 INFO:tasks.workunit.client.0.smithi110.stdout:6/778: readlink d4/d12/d17/d9a/le1 0 2024-09-21T04:15:25.288 INFO:tasks.workunit.client.0.smithi110.stdout:5/813: mkdir d1/d49/d4d/d1e/d5a/d11e 0 2024-09-21T04:15:25.288 INFO:tasks.workunit.client.0.smithi110.stdout:5/814: stat d1/d22/f27 0 2024-09-21T04:15:25.291 INFO:tasks.workunit.client.1.smithi148.stdout:1/852: creat db/d46/d8f/f11c x:0 0 0 2024-09-21T04:15:25.292 INFO:tasks.workunit.client.0.smithi110.stdout:4/921: read d17/d1e/d77/d10c/d57/f9f [979425,87979] 0 2024-09-21T04:15:25.292 INFO:tasks.workunit.client.0.smithi110.stdout:4/922: creat d17/d1e/d7b/daf/f139 x:0 0 0 2024-09-21T04:15:25.292 INFO:tasks.workunit.client.0.smithi110.stdout:4/923: fsync d17/d1e/d7b/fb8 0 2024-09-21T04:15:25.292 INFO:tasks.workunit.client.1.smithi148.stdout:3/814: write d3/d1f/d4d/d82/d23/db5/d105/f67 [6903261,84101] 0 2024-09-21T04:15:25.293 INFO:tasks.workunit.client.1.smithi148.stdout:3/815: creat d3/d1f/d4d/d82/d23/d28/d46/f114 x:0 0 0 2024-09-21T04:15:25.295 INFO:tasks.workunit.client.1.smithi148.stdout:4/915: rename d4/d66/d7f/d2a/l64 to d4/d66/d7f/d9a/dc2/l120 0 2024-09-21T04:15:25.297 INFO:tasks.workunit.client.0.smithi110.stdout:8/776: mkdir d0/d30/d34/d3e/d5d/dd7/dda/d119 0 2024-09-21T04:15:25.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/782: rmdir dc/d13/d3c 39 2024-09-21T04:15:25.297 INFO:tasks.workunit.client.0.smithi110.stdout:8/777: dread - d0/d7/d6f/f93 zero size 2024-09-21T04:15:25.298 INFO:tasks.workunit.client.0.smithi110.stdout:2/593: creat d2/d1a/d1f/d29/d19/fb7 x:0 0 0 2024-09-21T04:15:25.300 INFO:tasks.workunit.client.0.smithi110.stdout:6/779: creat d4/d12/d17/d81/dbf/fff x:0 0 0 2024-09-21T04:15:25.300 INFO:tasks.workunit.client.0.smithi110.stdout:4/924: dread d17/d1e/d7b/f5e [0,4194304] 0 2024-09-21T04:15:25.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/964: mknod d0/d9/c137 0 2024-09-21T04:15:25.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/965: dread - d0/d9/d16/d55/d71/dc2/dcb/f11a zero size 2024-09-21T04:15:25.307 INFO:tasks.workunit.client.1.smithi148.stdout:2/966: stat d0/c90 0 2024-09-21T04:15:25.307 INFO:tasks.workunit.client.0.smithi110.stdout:2/594: symlink d2/d1a/d1f/d5f/d9d/d86/dad/lb8 0 2024-09-21T04:15:25.309 INFO:tasks.workunit.client.1.smithi148.stdout:2/967: dread d0/d2b/da3/d8f/f9d [0,4194304] 0 2024-09-21T04:15:25.309 INFO:tasks.workunit.client.0.smithi110.stdout:6/780: dread d4/d12/d16/d66/d7a/d9e/ff2 [0,4194304] 0 2024-09-21T04:15:25.311 INFO:tasks.workunit.client.0.smithi110.stdout:4/925: truncate d17/d1e/f33 438104 0 2024-09-21T04:15:25.313 INFO:tasks.workunit.client.0.smithi110.stdout:5/815: dwrite d1/d49/f71 [0,4194304] 0 2024-09-21T04:15:25.314 INFO:tasks.workunit.client.0.smithi110.stdout:5/816: fsync d1/d49/d4d/d1e/d4a/dd4/dd5/fec 0 2024-09-21T04:15:25.314 INFO:tasks.workunit.client.0.smithi110.stdout:6/781: mknod d4/d12/d17/d2e/d40/dbc/c100 0 2024-09-21T04:15:25.314 INFO:tasks.workunit.client.0.smithi110.stdout:6/782: truncate d4/d86/d8f/fc7 146347 0 2024-09-21T04:15:25.316 INFO:tasks.workunit.client.0.smithi110.stdout:6/783: dread d4/d12/d16/d66/d7a/d83/fcf [0,4194304] 0 2024-09-21T04:15:25.317 INFO:tasks.workunit.client.1.smithi148.stdout:3/816: symlink d3/d5/d6/d2b/d96/d99/l115 0 2024-09-21T04:15:25.319 INFO:tasks.workunit.client.0.smithi110.stdout:4/926: symlink d17/d1e/d77/d10c/d20/d43/l13a 0 2024-09-21T04:15:25.319 INFO:tasks.workunit.client.0.smithi110.stdout:4/927: readlink d17/d1e/d7b/daf/l10b 0 2024-09-21T04:15:25.321 INFO:tasks.workunit.client.1.smithi148.stdout:6/940: rename d0/df9/f111 to d0/d34/d97/d106/d23/d27/dc2/de0/f12a 0 2024-09-21T04:15:25.323 INFO:tasks.workunit.client.0.smithi110.stdout:4/928: dread d17/d3e/dbd/f110 [0,4194304] 0 2024-09-21T04:15:25.324 INFO:tasks.workunit.client.0.smithi110.stdout:2/595: dwrite d2/d1a/d60/f8a [0,4194304] 0 2024-09-21T04:15:25.324 INFO:tasks.workunit.client.1.smithi148.stdout:4/916: symlink d4/d66/d7f/d116/l121 0 2024-09-21T04:15:25.325 INFO:tasks.workunit.client.0.smithi110.stdout:5/817: mkdir d1/d11f 0 2024-09-21T04:15:25.327 INFO:tasks.workunit.client.1.smithi148.stdout:8/783: symlink dc/d13/d76/d50/lf1 0 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/784: creat d4/d12/d16/d66/d7a/d4b/d7f/d89/f101 x:0 0 0 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/785: dread - d4/d12/d17/d2e/d38/f98 zero size 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/786: write d4/d12/d17/d33/d76/fc1 [305120,40656] 0 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/787: write d4/d78/fe9 [508211,55577] 0 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/788: chown d4/d78/fea 57 1 2024-09-21T04:15:25.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/789: fsync d4/d78/daa/f11 0 2024-09-21T04:15:25.330 INFO:tasks.workunit.client.0.smithi110.stdout:4/929: getdents d17/d1e/d64/d123 0 2024-09-21T04:15:25.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/853: getdents db/d24/d2a/da4 0 2024-09-21T04:15:25.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/854: chown db/d24/d2e/d59/cf0 0 1 2024-09-21T04:15:25.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/855: write db/d24/d2e/d71/d94/ff3 [765517,98914] 0 2024-09-21T04:15:25.332 INFO:tasks.workunit.client.1.smithi148.stdout:2/968: dwrite d0/d9/dc/d13/f8b [0,4194304] 0 2024-09-21T04:15:25.333 INFO:tasks.workunit.client.1.smithi148.stdout:3/817: mknod d3/d1f/d4d/d82/d23/d28/d46/db3/c116 0 2024-09-21T04:15:25.333 INFO:tasks.workunit.client.1.smithi148.stdout:3/818: fsync d3/d5/d6/d12/d43/d54/fc6 0 2024-09-21T04:15:25.335 INFO:tasks.workunit.client.0.smithi110.stdout:2/596: mkdir d2/d1a/da3/db9 0 2024-09-21T04:15:25.336 INFO:tasks.workunit.client.1.smithi148.stdout:6/941: mkdir d0/df9/d116/d12b 0 2024-09-21T04:15:25.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/790: symlink d4/d12/d17/d9a/dfa/l102 0 2024-09-21T04:15:25.342 INFO:tasks.workunit.client.1.smithi148.stdout:1/856: rename db/d24/d2e/d59/da7/dc1/dd2/d10c/f114 to db/d24/d2e/d59/f11d 0 2024-09-21T04:15:25.342 INFO:tasks.workunit.client.1.smithi148.stdout:1/857: readlink db/dd/l3f 0 2024-09-21T04:15:25.345 INFO:tasks.workunit.client.1.smithi148.stdout:6/942: dread d0/d34/d97/d106/d23/d27/d80/fb2 [0,4194304] 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.1.smithi148.stdout:2/969: link d0/d2b/da3/d95/fa9 d0/f138 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:2/597: creat d2/d1a/d1f/d5f/fba x:0 0 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:2/598: write d2/d1a/d1f/d65/fa0 [517863,28188] 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:4/930: dread d17/d3e/f8b [0,4194304] 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:6/791: truncate d4/d12/d16/d6d/f87 8032331 0 2024-09-21T04:15:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:6/792: readlink d4/d12/d17/d2e/d40/dad/ldc 0 2024-09-21T04:15:25.348 INFO:tasks.workunit.client.1.smithi148.stdout:3/819: mkdir d3/d5/d6/d2b/da2/da3/de4/d117 0 2024-09-21T04:15:25.353 INFO:tasks.workunit.client.1.smithi148.stdout:6/943: mkdir d0/d34/d97/d106/d23/dc/d68/d12c 0 2024-09-21T04:15:25.353 INFO:tasks.workunit.client.1.smithi148.stdout:6/944: dread - d0/d34/d97/d106/d23/dc/f8e zero size 2024-09-21T04:15:25.353 INFO:tasks.workunit.client.0.smithi110.stdout:4/931: rename d17/d3e/d6b/def to d17/d1e/d77/d10c/d20/d43/d13b 0 2024-09-21T04:15:25.353 INFO:tasks.workunit.client.0.smithi110.stdout:4/932: creat d17/d3e/d6b/dc7/f13c x:0 0 0 2024-09-21T04:15:25.354 INFO:tasks.workunit.client.0.smithi110.stdout:4/933: write d17/d3e/dbd/f110 [860823,69240] 0 2024-09-21T04:15:25.358 INFO:tasks.workunit.client.0.smithi110.stdout:6/793: unlink d4/d12/d17/d81/dbf/cdd 0 2024-09-21T04:15:25.358 INFO:tasks.workunit.client.0.smithi110.stdout:6/794: chown d4/d12/d16/d66/d7a/d83/ldf 33356329 1 2024-09-21T04:15:25.358 INFO:tasks.workunit.client.0.smithi110.stdout:6/795: read - d4/d12/d16/d66/d7a/d83/fe0 zero size 2024-09-21T04:15:25.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/917: dwrite d4/d87/d2d/dd0/fd6 [0,4194304] 0 2024-09-21T04:15:25.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/918: chown d4/d87/fba 5920 1 2024-09-21T04:15:25.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/970: link d0/d9/dc/d13/f1a d0/d9/d16/f139 0 2024-09-21T04:15:25.360 INFO:tasks.workunit.client.1.smithi148.stdout:2/971: creat d0/d2b/da3/d8f/f13a x:0 0 0 2024-09-21T04:15:25.361 INFO:tasks.workunit.client.0.smithi110.stdout:2/599: read d2/d1a/f88 [2962372,70083] 0 2024-09-21T04:15:25.361 INFO:tasks.workunit.client.0.smithi110.stdout:2/600: fdatasync d2/d1a/d1f/d65/f68 0 2024-09-21T04:15:25.361 INFO:tasks.workunit.client.0.smithi110.stdout:2/601: dread - d2/d1a/da3/fb1 zero size 2024-09-21T04:15:25.361 INFO:tasks.workunit.client.0.smithi110.stdout:2/602: fdatasync d2/d1a/d1f/d29/f4c 0 2024-09-21T04:15:25.362 INFO:tasks.workunit.client.1.smithi148.stdout:3/820: dread d3/d1f/d4d/d82/f1d [4194304,4194304] 0 2024-09-21T04:15:25.363 INFO:tasks.workunit.client.1.smithi148.stdout:8/784: dwrite dc/d13/f9b [0,4194304] 0 2024-09-21T04:15:25.363 INFO:tasks.workunit.client.1.smithi148.stdout:8/785: readlink dc/d3e/ldf 0 2024-09-21T04:15:25.363 INFO:tasks.workunit.client.1.smithi148.stdout:8/786: write dc/d13/d3c/f93 [1157919,48113] 0 2024-09-21T04:15:25.368 INFO:tasks.workunit.client.0.smithi110.stdout:6/796: rename d4/d12/d16/l49 to d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/l103 0 2024-09-21T04:15:25.371 INFO:tasks.workunit.client.1.smithi148.stdout:4/919: unlink d4/d12/d16/d29/dbd/f3e 0 2024-09-21T04:15:25.371 INFO:tasks.workunit.client.1.smithi148.stdout:4/920: dread - d4/d12/d16/d29/dbd/f68 zero size 2024-09-21T04:15:25.374 INFO:tasks.workunit.client.0.smithi110.stdout:5/818: dwrite d1/fb1 [0,4194304] 0 2024-09-21T04:15:25.375 INFO:tasks.workunit.client.1.smithi148.stdout:2/972: creat d0/d9/dc/d13/d25/dea/f13b x:0 0 0 2024-09-21T04:15:25.375 INFO:tasks.workunit.client.1.smithi148.stdout:2/973: fdatasync d0/d9/d16/d9e/f107 0 2024-09-21T04:15:25.377 INFO:tasks.workunit.client.0.smithi110.stdout:9/715: creat d7/d96/d99/d3c/ff4 x:0 0 0 2024-09-21T04:15:25.377 INFO:tasks.workunit.client.0.smithi110.stdout:9/716: chown d7/d96/d99/d3c/dae 0 1 2024-09-21T04:15:25.380 INFO:tasks.workunit.client.1.smithi148.stdout:3/821: symlink d3/d5/d6/d2b/da2/da3/de4/d117/l118 0 2024-09-21T04:15:25.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/858: dwrite db/d46/d4c/f5b [0,4194304] 0 2024-09-21T04:15:25.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/859: dread - db/d24/fe7 zero size 2024-09-21T04:15:25.382 INFO:tasks.workunit.client.1.smithi148.stdout:2/974: dread d0/d9/d16/d55/ffc [0,4194304] 0 2024-09-21T04:15:25.386 INFO:tasks.workunit.client.0.smithi110.stdout:9/717: dread d7/d96/d99/d3c/d11/f2e [0,4194304] 0 2024-09-21T04:15:25.387 INFO:tasks.workunit.client.1.smithi148.stdout:4/921: mknod d4/d12/d16/d29/d61/d6c/df1/c122 0 2024-09-21T04:15:25.387 INFO:tasks.workunit.client.1.smithi148.stdout:4/922: write d4/d12/d16/d29/d61/fda [635430,38685] 0 2024-09-21T04:15:25.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/822: dread d3/d1f/d4d/d82/d52/f4a [0,4194304] 0 2024-09-21T04:15:25.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/823: fdatasync d3/d5/d6/d2b/d96/db8/f110 0 2024-09-21T04:15:25.391 INFO:tasks.workunit.client.0.smithi110.stdout:9/718: symlink d7/d96/d99/lf5 0 2024-09-21T04:15:25.395 INFO:tasks.workunit.client.0.smithi110.stdout:2/603: dwrite d2/d1a/d1f/d5f/f2b [0,4194304] 0 2024-09-21T04:15:25.395 INFO:tasks.workunit.client.0.smithi110.stdout:9/719: symlink d7/d96/d99/d3c/d1f/d72/lf6 0 2024-09-21T04:15:25.395 INFO:tasks.workunit.client.0.smithi110.stdout:9/720: chown d7/d96/d99/d3c/d1f 368743517 1 2024-09-21T04:15:25.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/923: creat d4/d12/d83/d103/f123 x:0 0 0 2024-09-21T04:15:25.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/924: dread - d4/d66/d7f/d2a/f95 zero size 2024-09-21T04:15:25.401 INFO:tasks.workunit.client.1.smithi148.stdout:3/824: link d3/d5/c93 d3/d1f/d4d/d82/d23/d28/c119 0 2024-09-21T04:15:25.401 INFO:tasks.workunit.client.1.smithi148.stdout:4/925: creat d4/d12/da7/f124 x:0 0 0 2024-09-21T04:15:25.401 INFO:tasks.workunit.client.1.smithi148.stdout:4/926: getdents d4/d87/d2d/df4 0 2024-09-21T04:15:25.404 INFO:tasks.workunit.client.0.smithi110.stdout:6/797: dwrite d4/d78/daa/f1f [0,4194304] 0 2024-09-21T04:15:25.406 INFO:tasks.workunit.client.1.smithi148.stdout:2/975: dwrite d0/d9/d38/f6d [0,4194304] 0 2024-09-21T04:15:25.407 INFO:tasks.workunit.client.1.smithi148.stdout:2/976: chown d0/d2b/l64 3674 1 2024-09-21T04:15:25.407 INFO:tasks.workunit.client.1.smithi148.stdout:7/813: sync 2024-09-21T04:15:25.408 INFO:tasks.workunit.client.0.smithi110.stdout:4/934: dwrite d17/d1e/d21/de1/ff1 [0,4194304] 0 2024-09-21T04:15:25.413 INFO:tasks.workunit.client.1.smithi148.stdout:7/814: fsync d5/fa7 0 2024-09-21T04:15:25.417 INFO:tasks.workunit.client.1.smithi148.stdout:7/815: mknod d5/d8/d6f/d7d/d90/d9b/c11a 0 2024-09-21T04:15:25.418 INFO:tasks.workunit.client.1.smithi148.stdout:2/977: write d0/d9/d16/d55/d71/dc4/fe2 [22370,61332] 0 2024-09-21T04:15:25.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/935: dread d17/d1e/d77/d10c/d20/f95 [0,4194304] 0 2024-09-21T04:15:25.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/936: getdents d17/d1e/d77/d10c/d20/d43/d51/d52/d98/d12d 0 2024-09-21T04:15:25.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/937: chown d17/d1e/d77/d10c/d20/dca/fd6 2 1 2024-09-21T04:15:25.421 INFO:tasks.workunit.client.1.smithi148.stdout:7/816: rmdir d5/d14/d25/d40/dab 39 2024-09-21T04:15:25.425 INFO:tasks.workunit.client.1.smithi148.stdout:2/978: link d0/d2f/l5f d0/d2b/d48/l13c 0 2024-09-21T04:15:25.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/817: rename d5/d8/lf7 to d5/d8/d6f/d7d/d90/l11b 0 2024-09-21T04:15:25.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/818: chown d5/d8/d6f/fec 7196 1 2024-09-21T04:15:25.429 INFO:tasks.workunit.client.1.smithi148.stdout:2/979: rename d0/d9/c1c to d0/d9/d16/d55/c13d 0 2024-09-21T04:15:25.431 INFO:tasks.workunit.client.1.smithi148.stdout:7/819: link d5/d14/d25/d40/d7b/f9a d5/daa/f11c 0 2024-09-21T04:15:25.431 INFO:tasks.workunit.client.1.smithi148.stdout:7/820: truncate d5/d8/d6f/d7d/d90/dee/f118 48973 0 2024-09-21T04:15:25.432 INFO:tasks.workunit.client.1.smithi148.stdout:2/980: mkdir d0/d9/dc/ddf/d13e 0 2024-09-21T04:15:25.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/821: creat d5/d8/d6f/d7d/d90/f11d x:0 0 0 2024-09-21T04:15:25.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/822: truncate d5/d8/f10c 900042 0 2024-09-21T04:15:25.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/823: creat d5/daa/f11e x:0 0 0 2024-09-21T04:15:25.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/824: read - d5/d14/d25/d40/d8b/f113 zero size 2024-09-21T04:15:25.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/825: write d5/d14/d25/d40/db9/fbd [1182280,77347] 0 2024-09-21T04:15:25.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/826: fdatasync d5/d8/d6f/d7d/d90/f10 0 2024-09-21T04:15:25.437 INFO:tasks.workunit.client.1.smithi148.stdout:2/981: link d0/d2b/da3/d82/cb6 d0/d9/dc/d13/dec/c13f 0 2024-09-21T04:15:25.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/982: mkdir d0/d9/d100/d140 0 2024-09-21T04:15:25.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/983: creat d0/d2f/db1/f141 x:0 0 0 2024-09-21T04:15:25.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/984: write d0/d9/d16/f11c [854015,28223] 0 2024-09-21T04:15:25.440 INFO:tasks.workunit.client.0.smithi110.stdout:2/604: dwrite d2/d1a/d60/f8a [0,4194304] 0 2024-09-21T04:15:25.442 INFO:tasks.workunit.client.1.smithi148.stdout:3/825: dwrite d3/d1f/d4d/d82/d23/d28/d46/f107 [0,4194304] 0 2024-09-21T04:15:25.443 INFO:tasks.workunit.client.1.smithi148.stdout:4/927: dwrite d4/d12/d16/d29/d61/d6c/d9d/fb0 [0,4194304] 0 2024-09-21T04:15:25.443 INFO:tasks.workunit.client.0.smithi110.stdout:7/741: mkdir dd/df3 0 2024-09-21T04:15:25.443 INFO:tasks.workunit.client.0.smithi110.stdout:7/742: fsync dd/d4d/dee/dbc/fc2 0 2024-09-21T04:15:25.447 INFO:tasks.workunit.client.0.smithi110.stdout:6/798: dwrite d4/d78/fe9 [0,4194304] 0 2024-09-21T04:15:25.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/743: mknod dd/d4d/dc1/cf4 0 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/799: getdents d4/d12/d17/d2e/d40/dad 0 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/800: chown d4/d12/d17/d2e/d40/dbc 20691 1 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/801: stat d4/f79 0 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:7/744: dread dd/d48/d9f/fef [0,4194304] 0 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/802: write d4/f6 [263893,117213] 0 2024-09-21T04:15:25.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/803: write d4/d12/f2b [2070565,14399] 0 2024-09-21T04:15:25.453 INFO:tasks.workunit.client.0.smithi110.stdout:3/694: mknod d0/d12/cee 0 2024-09-21T04:15:25.453 INFO:tasks.workunit.client.1.smithi148.stdout:4/928: dread d4/d12/d16/d29/d61/d7e/fa0 [0,4194304] 0 2024-09-21T04:15:25.459 INFO:tasks.workunit.client.0.smithi110.stdout:7/745: mkdir dd/d14/d17/dd1/df5 0 2024-09-21T04:15:25.460 INFO:tasks.workunit.client.0.smithi110.stdout:7/746: truncate dd/d48/fb7 248893 0 2024-09-21T04:15:25.460 INFO:tasks.workunit.client.0.smithi110.stdout:7/747: truncate dd/db1/ff1 32830 0 2024-09-21T04:15:25.460 INFO:tasks.workunit.client.0.smithi110.stdout:7/748: chown dd/d1c/dad/db2/d56/d66/da6/d7a/d9b 13146 1 2024-09-21T04:15:25.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/695: creat d0/d12/d3c/d5a/d70/d81/fef x:0 0 0 2024-09-21T04:15:25.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/826: mknod d3/d5/db4/c11a 0 2024-09-21T04:15:25.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/827: write d3/d5/d5c/f10e [987082,58562] 0 2024-09-21T04:15:25.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/828: chown d3/d5/d6/d12/d43/l50 133963058 1 2024-09-21T04:15:25.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/829: fdatasync d3/d1f/d4d/d82/d23/d28/d46/f114 0 2024-09-21T04:15:25.465 INFO:tasks.workunit.client.0.smithi110.stdout:3/696: mkdir d0/d12/d3c/d5a/d65/df0 0 2024-09-21T04:15:25.466 INFO:tasks.workunit.client.1.smithi148.stdout:7/827: dwrite d5/d14/d25/d40/dab/dad/fe5 [0,4194304] 0 2024-09-21T04:15:25.466 INFO:tasks.workunit.client.1.smithi148.stdout:7/828: readlink d5/d8/d6f/d7d/d90/lb7 0 2024-09-21T04:15:25.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/830: unlink d3/d5/d6/d12/d43/d54/c10f 0 2024-09-21T04:15:25.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/929: read d4/d12/d16/d29/d61/d6c/f88 [4170522,113202] 0 2024-09-21T04:15:25.467 INFO:tasks.workunit.client.0.smithi110.stdout:3/697: creat d0/d12/d2a/d80/d9c/ff1 x:0 0 0 2024-09-21T04:15:25.467 INFO:tasks.workunit.client.1.smithi148.stdout:2/985: dwrite d0/d9/d16/d55/fd9 [0,4194304] 0 2024-09-21T04:15:25.468 INFO:tasks.workunit.client.0.smithi110.stdout:2/605: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:25.472 INFO:tasks.workunit.client.1.smithi148.stdout:7/829: rename d5/d14/d25/c43 to d5/d14/d25/d40/db9/dff/c11f 0 2024-09-21T04:15:25.474 INFO:tasks.workunit.client.1.smithi148.stdout:3/831: link d3/d5/d5c/fc4 d3/d5/d6/f11b 0 2024-09-21T04:15:25.475 INFO:tasks.workunit.client.0.smithi110.stdout:2/606: truncate d2/d50/d7b/f91 2279112 0 2024-09-21T04:15:25.477 INFO:tasks.workunit.client.1.smithi148.stdout:4/930: symlink d4/d12/d16/d29/d61/d6c/d9d/dfb/l125 0 2024-09-21T04:15:25.477 INFO:tasks.workunit.client.1.smithi148.stdout:4/931: stat d4/d12/d75/d9c/fc4 0 2024-09-21T04:15:25.477 INFO:tasks.workunit.client.1.smithi148.stdout:4/932: write d4/d66/d7f/d2a/da9/f10a [1040784,101534] 0 2024-09-21T04:15:25.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/986: dwrite d0/d9/d16/d55/d71/f133 [0,4194304] 0 2024-09-21T04:15:25.481 INFO:tasks.workunit.client.1.smithi148.stdout:7/830: mkdir d5/d8/d6f/d7d/d120 0 2024-09-21T04:15:25.483 INFO:tasks.workunit.client.1.smithi148.stdout:4/933: rename d4/l43 to d4/d12/d78/de4/l126 0 2024-09-21T04:15:25.484 INFO:tasks.workunit.client.1.smithi148.stdout:4/934: write d4/d12/d75/d85/f99 [27222,54033] 0 2024-09-21T04:15:25.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/831: rename d5/d8/d64/ce7 to d5/d8/d6f/d7d/df5/c121 0 2024-09-21T04:15:25.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/832: mknod d5/d14/d25/d40/c122 0 2024-09-21T04:15:25.490 INFO:tasks.workunit.client.1.smithi148.stdout:7/833: unlink d5/d8/d64/f96 0 2024-09-21T04:15:25.491 INFO:tasks.workunit.client.1.smithi148.stdout:7/834: link d5/d8/d6f/d7d/d90/cc1 d5/d8/d6f/d7d/df5/c123 0 2024-09-21T04:15:25.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/872: mkdir d5/d60/d28/d2f/d136 0 2024-09-21T04:15:25.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/835: symlink d5/d8/d6f/d7d/df5/l124 0 2024-09-21T04:15:25.494 INFO:tasks.workunit.client.1.smithi148.stdout:9/873: unlink d5/d2b/d32/cb6 0 2024-09-21T04:15:25.495 INFO:tasks.workunit.client.1.smithi148.stdout:9/874: chown d5/d60/d28/f30 35144055 1 2024-09-21T04:15:25.496 INFO:tasks.workunit.client.1.smithi148.stdout:9/875: rmdir d5/d2b/dbe 39 2024-09-21T04:15:25.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/935: read d4/d87/d91/fcf [2920269,99213] 0 2024-09-21T04:15:25.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/936: chown d4/d87/l119 30282 1 2024-09-21T04:15:25.499 INFO:tasks.workunit.client.0.smithi110.stdout:4/938: rename d17/d1e/d77/d10c/d20/d43/d51/d7a/d86 to d17/d1e/d13d 0 2024-09-21T04:15:25.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/939: link d17/f3c d17/d3e/d6b/dc7/f13e 0 2024-09-21T04:15:25.502 INFO:tasks.workunit.client.1.smithi148.stdout:9/876: dread d5/d1e/d37/fca [0,4194304] 0 2024-09-21T04:15:25.503 INFO:tasks.workunit.client.0.smithi110.stdout:6/804: rename d4/d12/d17/d81/d90/dc3/ld7 to d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/l104 0 2024-09-21T04:15:25.503 INFO:tasks.workunit.client.0.smithi110.stdout:6/805: chown d4/d12/d17/d2e/d40/c37 132 1 2024-09-21T04:15:25.503 INFO:tasks.workunit.client.1.smithi148.stdout:7/836: dread d5/d8/fea [0,4194304] 0 2024-09-21T04:15:25.504 INFO:tasks.workunit.client.1.smithi148.stdout:9/877: symlink d5/d1f/d81/dcf/d106/d80/d104/l137 0 2024-09-21T04:15:25.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/837: mkdir d5/d14/d25/d40/dab/dad/db8/d125 0 2024-09-21T04:15:25.507 INFO:tasks.workunit.client.0.smithi110.stdout:6/806: mkdir d4/d12/d17/d81/dbf/d105 0 2024-09-21T04:15:25.507 INFO:tasks.workunit.client.0.smithi110.stdout:2/607: dwrite d2/d1a/d1f/d65/fa0 [0,4194304] 0 2024-09-21T04:15:25.508 INFO:tasks.workunit.client.1.smithi148.stdout:2/987: dwrite d0/d2b/da3/d8f/fa4 [0,4194304] 0 2024-09-21T04:15:25.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/832: dwrite d3/d1f/d4d/d82/d23/d28/d46/db3/fea [0,4194304] 0 2024-09-21T04:15:25.508 INFO:tasks.workunit.client.0.smithi110.stdout:2/608: read d2/d1a/d1f/d2a/f61 [292040,16027] 0 2024-09-21T04:15:25.508 INFO:tasks.workunit.client.0.smithi110.stdout:2/609: chown d2/d1a/d1f/d5f/d9d/f78 1327788721 1 2024-09-21T04:15:25.509 INFO:tasks.workunit.client.1.smithi148.stdout:7/838: mknod d5/d14/d25/d40/d7b/c126 0 2024-09-21T04:15:25.510 INFO:tasks.workunit.client.0.smithi110.stdout:4/940: truncate d17/d1e/fbf 1084474 0 2024-09-21T04:15:25.510 INFO:tasks.workunit.client.0.smithi110.stdout:4/941: fdatasync d17/d1e/d77/d10c/d57/db9/f11f 0 2024-09-21T04:15:25.510 INFO:tasks.workunit.client.0.smithi110.stdout:4/942: fsync d17/d1e/f48 0 2024-09-21T04:15:25.510 INFO:tasks.workunit.client.0.smithi110.stdout:4/943: fsync d17/d1e/d77/d10c/d20/dca/fd6 0 2024-09-21T04:15:25.511 INFO:tasks.workunit.client.0.smithi110.stdout:6/807: getdents d4/d12/d17/d81/db6 0 2024-09-21T04:15:25.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/878: rmdir d5/d1f/d81/d9e 39 2024-09-21T04:15:25.515 INFO:tasks.workunit.client.0.smithi110.stdout:2/610: creat d2/fbb x:0 0 0 2024-09-21T04:15:25.515 INFO:tasks.workunit.client.0.smithi110.stdout:2/611: chown d2/d50/d7b 3573 1 2024-09-21T04:15:25.519 INFO:tasks.workunit.client.0.smithi110.stdout:4/944: unlink d17/d1e/f129 0 2024-09-21T04:15:25.519 INFO:tasks.workunit.client.0.smithi110.stdout:4/945: fdatasync d17/d1e/d77/d10c/f91 0 2024-09-21T04:15:25.520 INFO:tasks.workunit.client.1.smithi148.stdout:2/988: creat d0/d9/d59/dda/f142 x:0 0 0 2024-09-21T04:15:25.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/833: mknod d3/d5/d6/d2b/d96/db8/c11c 0 2024-09-21T04:15:25.525 INFO:tasks.workunit.client.0.smithi110.stdout:4/946: creat d17/d1e/d77/d10c/d20/d43/d51/d52/d132/f13f x:0 0 0 2024-09-21T04:15:25.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/947: mknod d17/d1e/d77/d10c/d20/d43/d51/d7a/c140 0 2024-09-21T04:15:25.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/948: chown d17/d1e/d77/d10c/d20/d43/d51/d52/f59 23098755 1 2024-09-21T04:15:25.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/949: getdents d17/d1e/d64/d123 0 2024-09-21T04:15:25.530 INFO:tasks.workunit.client.1.smithi148.stdout:3/834: mknod d3/d1f/d4d/d82/d23/da1/c11d 0 2024-09-21T04:15:25.530 INFO:tasks.workunit.client.1.smithi148.stdout:3/835: write d3/d1f/d4d/d82/d74/fa0 [847794,86270] 0 2024-09-21T04:15:25.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/836: fdatasync d3/d1f/d4d/d82/d21/fd2 0 2024-09-21T04:15:25.534 INFO:tasks.workunit.client.1.smithi148.stdout:7/839: dwrite d5/fa7 [0,4194304] 0 2024-09-21T04:15:25.536 INFO:tasks.workunit.client.0.smithi110.stdout:8/778: creat d0/d30/d34/f11a x:0 0 0 2024-09-21T04:15:25.536 INFO:tasks.workunit.client.0.smithi110.stdout:8/779: truncate d0/d30/d105/f114 852643 0 2024-09-21T04:15:25.536 INFO:tasks.workunit.client.0.smithi110.stdout:8/780: readlink d0/d7/d13/d1c/d41/lca 0 2024-09-21T04:15:25.537 INFO:tasks.workunit.client.0.smithi110.stdout:8/781: truncate d0/d7/d13/d46/df8/fe5 32572 0 2024-09-21T04:15:25.537 INFO:tasks.workunit.client.0.smithi110.stdout:8/782: mkdir d0/d7/d23/d82/d85/d9b/d11b 0 2024-09-21T04:15:25.537 INFO:tasks.workunit.client.0.smithi110.stdout:8/783: chown d0/d7/d23/d82/ffa 25256016 1 2024-09-21T04:15:25.538 INFO:tasks.workunit.client.0.smithi110.stdout:5/819: mknod d1/d49/d4d/d1e/d4a/c120 0 2024-09-21T04:15:25.538 INFO:tasks.workunit.client.0.smithi110.stdout:5/820: chown d1/d49/d4d/d1e/d5a/f67 1588 1 2024-09-21T04:15:25.539 INFO:tasks.workunit.client.0.smithi110.stdout:8/784: dread d0/d7/d13/d1c/fa0 [0,4194304] 0 2024-09-21T04:15:25.539 INFO:tasks.workunit.client.0.smithi110.stdout:8/785: chown d0/d7/d13/d4b 229443289 1 2024-09-21T04:15:25.540 INFO:tasks.workunit.client.0.smithi110.stdout:8/786: symlink d0/d7/d13/d46/l11c 0 2024-09-21T04:15:25.541 INFO:tasks.workunit.client.0.smithi110.stdout:8/787: rename d0/d7/c4f to d0/d30/d54/c11d 0 2024-09-21T04:15:25.542 INFO:tasks.workunit.client.0.smithi110.stdout:8/788: mkdir d0/d11e 0 2024-09-21T04:15:25.544 INFO:tasks.workunit.client.0.smithi110.stdout:8/789: unlink d0/d7/d23/d82/ca4 0 2024-09-21T04:15:25.544 INFO:tasks.workunit.client.0.smithi110.stdout:8/790: read d0/d30/d54/d6b/fcc [155578,33208] 0 2024-09-21T04:15:25.547 INFO:tasks.workunit.client.0.smithi110.stdout:3/698: mknod d0/d12/d3c/d5a/d70/cf2 0 2024-09-21T04:15:25.548 INFO:tasks.workunit.client.0.smithi110.stdout:8/791: dread d0/f3d [0,4194304] 0 2024-09-21T04:15:25.549 INFO:tasks.workunit.client.0.smithi110.stdout:3/699: symlink d0/d22/d29/lf3 0 2024-09-21T04:15:25.549 INFO:tasks.workunit.client.1.smithi148.stdout:3/837: dread d3/d1f/d4d/d82/d21/fd2 [0,4194304] 0 2024-09-21T04:15:25.550 INFO:tasks.workunit.client.1.smithi148.stdout:7/840: write d5/d8/fe3 [1786926,75898] 0 2024-09-21T04:15:25.550 INFO:tasks.workunit.client.1.smithi148.stdout:7/841: chown d5/d8/d6f/l7c 49 1 2024-09-21T04:15:25.550 INFO:tasks.workunit.client.1.smithi148.stdout:7/842: chown d5/d8/d6f/d7d/d90/fb5 17085 1 2024-09-21T04:15:25.551 INFO:tasks.workunit.client.0.smithi110.stdout:3/700: rename d0/d12/d2a/f30 to d0/d12/d3c/ff4 0 2024-09-21T04:15:25.551 INFO:tasks.workunit.client.0.smithi110.stdout:3/701: fdatasync d0/fd3 0 2024-09-21T04:15:25.552 INFO:tasks.workunit.client.1.smithi148.stdout:3/838: truncate d3/d1f/d4d/d82/d52/fab 184621 0 2024-09-21T04:15:25.552 INFO:tasks.workunit.client.0.smithi110.stdout:6/808: dwrite d4/d78/daa/fb4 [0,4194304] 0 2024-09-21T04:15:25.553 INFO:tasks.workunit.client.0.smithi110.stdout:4/950: write d17/d1e/d77/d10c/d20/d43/d51/d7a/fa7 [6631766,99907] 0 2024-09-21T04:15:25.555 INFO:tasks.workunit.client.0.smithi110.stdout:3/702: unlink d0/d12/d16/d1a/d6f/lc3 0 2024-09-21T04:15:25.557 INFO:tasks.workunit.client.1.smithi148.stdout:2/989: dwrite d0/d2b/fbb [4194304,4194304] 0 2024-09-21T04:15:25.557 INFO:tasks.workunit.client.1.smithi148.stdout:7/843: creat d5/d14/d25/d79/d86/f127 x:0 0 0 2024-09-21T04:15:25.559 INFO:tasks.workunit.client.1.smithi148.stdout:9/879: dwrite d5/d2b/d32/f5b [0,4194304] 0 2024-09-21T04:15:25.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/839: mknod d3/d5/d6/d2b/da2/da3/c11e 0 2024-09-21T04:15:25.560 INFO:tasks.workunit.client.0.smithi110.stdout:2/612: dwrite d2/d1a/d1f/d5f/f95 [0,4194304] 0 2024-09-21T04:15:25.560 INFO:tasks.workunit.client.0.smithi110.stdout:2/613: chown d2/d1a/d1f/d29/f45 0 1 2024-09-21T04:15:25.560 INFO:tasks.workunit.client.0.smithi110.stdout:2/614: dread - d2/d1a/d1f/d5f/fba zero size 2024-09-21T04:15:25.562 INFO:tasks.workunit.client.1.smithi148.stdout:0/866: mkdir d5/d111 0 2024-09-21T04:15:25.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/703: unlink d0/d12/d2a/f61 0 2024-09-21T04:15:25.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/704: creat d0/d12/d16/d1a/d5c/ff5 x:0 0 0 2024-09-21T04:15:25.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/705: write d0/d12/d3c/d5a/d65/f75 [1120404,116303] 0 2024-09-21T04:15:25.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/706: fdatasync d0/d22/d29/d76/fa6 0 2024-09-21T04:15:25.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/990: symlink d0/d2b/da3/d8f/d9f/l143 0 2024-09-21T04:15:25.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/991: creat d0/d2b/f144 x:0 0 0 2024-09-21T04:15:25.567 INFO:tasks.workunit.client.0.smithi110.stdout:5/821: dwrite d1/d49/d4d/d1e/f81 [0,4194304] 0 2024-09-21T04:15:25.568 INFO:tasks.workunit.client.1.smithi148.stdout:9/880: creat d5/d60/de1/f138 x:0 0 0 2024-09-21T04:15:25.568 INFO:tasks.workunit.client.1.smithi148.stdout:9/881: truncate d5/d60/d28/d2f/dbf/f131 881823 0 2024-09-21T04:15:25.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/707: rename d0/d12/d16/f25 to d0/d22/dd2/ff6 0 2024-09-21T04:15:25.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/708: fsync d0/d22/fd6 0 2024-09-21T04:15:25.572 INFO:tasks.workunit.client.1.smithi148.stdout:7/844: dread d5/d14/d25/fa2 [0,4194304] 0 2024-09-21T04:15:25.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/867: mkdir d5/dcb/de6/d110/d7/dd0/d112 0 2024-09-21T04:15:25.573 INFO:tasks.workunit.client.0.smithi110.stdout:8/792: dwrite d0/d7/fd [4194304,4194304] 0 2024-09-21T04:15:25.574 INFO:tasks.workunit.client.0.smithi110.stdout:5/822: rmdir d1/d22/d6e 39 2024-09-21T04:15:25.574 INFO:tasks.workunit.client.0.smithi110.stdout:5/823: write d1/d49/d6f/fbd [246731,113688] 0 2024-09-21T04:15:25.575 INFO:tasks.workunit.client.1.smithi148.stdout:2/992: unlink d0/d9/d16/f60 0 2024-09-21T04:15:25.575 INFO:tasks.workunit.client.1.smithi148.stdout:2/993: dread - d0/d2b/da3/d8f/d9f/dc9/fe9 zero size 2024-09-21T04:15:25.576 INFO:tasks.workunit.client.0.smithi110.stdout:8/793: dread d0/d7/d13/d46/df8/faf [0,4194304] 0 2024-09-21T04:15:25.576 INFO:tasks.workunit.client.0.smithi110.stdout:8/794: write d0/d7/d13/d4b/f1f [3202861,87066] 0 2024-09-21T04:15:25.581 INFO:tasks.workunit.client.0.smithi110.stdout:8/795: mknod d0/d30/d54/d10c/c11f 0 2024-09-21T04:15:25.581 INFO:tasks.workunit.client.0.smithi110.stdout:8/796: readlink d0/d7/d89/l9e 0 2024-09-21T04:15:25.582 INFO:tasks.workunit.client.0.smithi110.stdout:2/615: dwrite d2/d1a/d1f/d5f/f1d [0,4194304] 0 2024-09-21T04:15:25.583 INFO:tasks.workunit.client.0.smithi110.stdout:2/616: read d2/d1a/d1f/d29/f45 [3561492,120315] 0 2024-09-21T04:15:25.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/882: mkdir d5/d60/d4c/d5e/d139 0 2024-09-21T04:15:25.587 INFO:tasks.workunit.client.0.smithi110.stdout:8/797: rename d0/d7/d13/d1c/f61 to d0/d7/d6f/f120 0 2024-09-21T04:15:25.589 INFO:tasks.workunit.client.1.smithi148.stdout:7/845: rename d5/d8/d6f/d7d/d90/c117 to d5/d14/d25/d79/d86/c128 0 2024-09-21T04:15:25.590 INFO:tasks.workunit.client.1.smithi148.stdout:0/868: symlink d5/dcb/de6/d110/d2a/d3f/d6d/da3/df4/l113 0 2024-09-21T04:15:25.590 INFO:tasks.workunit.client.0.smithi110.stdout:5/824: dwrite d1/d39/f21 [4194304,4194304] 0 2024-09-21T04:15:25.592 INFO:tasks.workunit.client.0.smithi110.stdout:8/798: symlink d0/d7/d13/d4b/ddc/l121 0 2024-09-21T04:15:25.592 INFO:tasks.workunit.client.0.smithi110.stdout:8/799: getdents d0/d7/d23/d6c/df2 0 2024-09-21T04:15:25.593 INFO:tasks.workunit.client.1.smithi148.stdout:2/994: unlink d0/f68 0 2024-09-21T04:15:25.593 INFO:tasks.workunit.client.1.smithi148.stdout:2/995: creat d0/f145 x:0 0 0 2024-09-21T04:15:25.596 INFO:tasks.workunit.client.1.smithi148.stdout:9/883: mknod d5/d1e/d37/d46/c13a 0 2024-09-21T04:15:25.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/800: creat d0/d30/d34/d3e/d5d/dd7/dda/d119/f122 x:0 0 0 2024-09-21T04:15:25.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/801: chown d0/d7/d13/d46/f106 3 1 2024-09-21T04:15:25.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/802: chown d0/d7/le 557688634 1 2024-09-21T04:15:25.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/803: dread - d0/d30/d9d/dad/f104 zero size 2024-09-21T04:15:25.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/804: dread - d0/d7/d13/f57 zero size 2024-09-21T04:15:25.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/846: unlink d5/d8/d6f/fec 0 2024-09-21T04:15:25.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/869: rename d5/dcb/de6/d110/d7/d11/d48/f6b to d5/dcb/de6/d110/d2a/d3f/d6d/f114 0 2024-09-21T04:15:25.601 INFO:tasks.workunit.client.0.smithi110.stdout:8/805: rename d0/d7/d89/l9e to d0/d7/d23/d52/l123 0 2024-09-21T04:15:25.601 INFO:tasks.workunit.client.0.smithi110.stdout:8/806: readlink d0/la2 0 2024-09-21T04:15:25.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/996: creat d0/d2b/da3/da1/d11d/f146 x:0 0 0 2024-09-21T04:15:25.604 INFO:tasks.workunit.client.0.smithi110.stdout:8/807: rename d0/d7/l48 to d0/d30/d34/d3e/d5d/l124 0 2024-09-21T04:15:25.604 INFO:tasks.workunit.client.0.smithi110.stdout:8/808: creat d0/d30/d34/d3e/d5d/f125 x:0 0 0 2024-09-21T04:15:25.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/809: creat d0/d7/d23/d82/d85/f126 x:0 0 0 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/810: dread - d0/d7/d6f/f93 zero size 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/811: truncate d0/d7/d13/d1c/fa0 800267 0 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/812: chown d0/d30/d34/d3e/d5d/dd7/fe0 1840 1 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/813: chown d0/d30/d54/d6b/da8 3567 1 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/814: fdatasync d0/d7/d23/d82/f83 0 2024-09-21T04:15:25.607 INFO:tasks.workunit.client.1.smithi148.stdout:0/870: mknod d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/c115 0 2024-09-21T04:15:25.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/997: symlink d0/d2b/da3/da1/l147 0 2024-09-21T04:15:25.609 INFO:tasks.workunit.client.0.smithi110.stdout:8/815: symlink d0/d30/l127 0 2024-09-21T04:15:25.610 INFO:tasks.workunit.client.1.smithi148.stdout:7/847: write d5/d14/d25/d40/dab/fd4 [258322,114136] 0 2024-09-21T04:15:25.610 INFO:tasks.workunit.client.1.smithi148.stdout:7/848: chown d5/d14/d25/d40/d8b/f113 240 1 2024-09-21T04:15:25.610 INFO:tasks.workunit.client.1.smithi148.stdout:2/998: creat d0/d9/d100/d140/f148 x:0 0 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/840: truncate d3/d1f/d4d/d82/d23/d28/d46/f107 3861014 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/841: fdatasync d3/d1f/d4d/d82/d23/d28/f108 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/842: creat d3/d5/d6/d12/d43/f11f x:0 0 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/843: creat d3/d1f/d4d/d82/d23/d28/d46/f120 x:0 0 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/844: write d3/d5/d5c/fdb [131304,123351] 0 2024-09-21T04:15:25.612 INFO:tasks.workunit.client.0.smithi110.stdout:8/816: creat d0/d30/d54/d6b/da8/f128 x:0 0 0 2024-09-21T04:15:25.613 INFO:tasks.workunit.client.0.smithi110.stdout:8/817: chown d0/d30/d54/d6b/da8/f128 49442 1 2024-09-21T04:15:25.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/825: dwrite d1/d22/ffa [0,4194304] 0 2024-09-21T04:15:25.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/826: truncate d1/d49/d4d/d1e/f3b 1775425 0 2024-09-21T04:15:25.614 INFO:tasks.workunit.client.0.smithi110.stdout:2/617: dwrite d2/d1a/d1f/d5f/f2b [0,4194304] 0 2024-09-21T04:15:25.615 INFO:tasks.workunit.client.0.smithi110.stdout:8/818: unlink d0/d7/d13/d4b/c71 0 2024-09-21T04:15:25.616 INFO:tasks.workunit.client.1.smithi148.stdout:0/871: dread d5/dcb/de6/d110/d7/d11/f1d [4194304,4194304] 0 2024-09-21T04:15:25.619 INFO:tasks.workunit.client.1.smithi148.stdout:3/845: creat d3/d5/d6/d2b/d96/db8/d10b/f121 x:0 0 0 2024-09-21T04:15:25.619 INFO:tasks.workunit.client.1.smithi148.stdout:3/846: chown d3/d1f/d4d/d82/l1b 5393332 1 2024-09-21T04:15:25.621 INFO:tasks.workunit.client.1.smithi148.stdout:0/872: rename d5/c67 to d5/dcb/de6/d110/d7/dd0/c116 0 2024-09-21T04:15:25.623 INFO:tasks.workunit.client.0.smithi110.stdout:5/827: mkdir d1/d49/d121 0 2024-09-21T04:15:25.626 INFO:tasks.workunit.client.1.smithi148.stdout:0/873: truncate d5/dcb/de6/d110/d2a/dcd/df8/da5/fbf 1304826 0 2024-09-21T04:15:25.626 INFO:tasks.workunit.client.0.smithi110.stdout:2/618: link d2/d1a/d1f/d29/f45 d2/d1a/d1f/d5f/d9d/d86/fbc 0 2024-09-21T04:15:25.628 INFO:tasks.workunit.client.0.smithi110.stdout:8/819: unlink d0/d7/d89/ff3 0 2024-09-21T04:15:25.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/820: write d0/d7/d6f/d86/fa6 [213888,60615] 0 2024-09-21T04:15:25.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/821: creat d0/d30/d34/d3e/d5d/dd7/dda/d119/f129 x:0 0 0 2024-09-21T04:15:25.629 INFO:tasks.workunit.client.1.smithi148.stdout:0/874: truncate f4 1393728 0 2024-09-21T04:15:25.630 INFO:tasks.workunit.client.0.smithi110.stdout:8/822: dread d0/d30/d34/d3e/d32/f73 [0,4194304] 0 2024-09-21T04:15:25.630 INFO:tasks.workunit.client.0.smithi110.stdout:8/823: creat d0/d30/d9d/dad/f12a x:0 0 0 2024-09-21T04:15:25.631 INFO:tasks.workunit.client.0.smithi110.stdout:5/828: rename d1/d49/c53 to d1/d7/d58/d72/dcb/c122 0 2024-09-21T04:15:25.631 INFO:tasks.workunit.client.0.smithi110.stdout:5/829: write d1/dc/fea [18808,8034] 0 2024-09-21T04:15:25.632 INFO:tasks.workunit.client.0.smithi110.stdout:5/830: read d1/d49/d4d/d1e/d4a/ff3 [984306,68851] 0 2024-09-21T04:15:25.633 INFO:tasks.workunit.client.0.smithi110.stdout:2/619: mknod d2/d1a/d1f/d29/cbd 0 2024-09-21T04:15:25.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/999: dwrite d0/d2b/f7d [0,4194304] 0 2024-09-21T04:15:25.637 INFO:tasks.workunit.client.0.smithi110.stdout:8/824: symlink d0/d7/l12b 0 2024-09-21T04:15:25.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/831: mkdir d1/d49/d121/d123 0 2024-09-21T04:15:25.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/832: fdatasync d1/d49/d4d/d1e/d4a/faf 0 2024-09-21T04:15:25.638 INFO:tasks.workunit.client.1.smithi148.stdout:3/847: dwrite d3/d1f/d4d/d82/f3d [0,4194304] 0 2024-09-21T04:15:25.639 INFO:tasks.workunit.client.0.smithi110.stdout:2/620: mknod d2/d1a/cbe 0 2024-09-21T04:15:25.641 INFO:tasks.workunit.client.0.smithi110.stdout:5/833: creat d1/d49/d4d/d1e/d4a/dfb/f124 x:0 0 0 2024-09-21T04:15:25.641 INFO:tasks.workunit.client.0.smithi110.stdout:5/834: readlink d1/d49/d4d/d1e/l34 0 2024-09-21T04:15:25.641 INFO:tasks.workunit.client.0.smithi110.stdout:5/835: stat d1/f4c 0 2024-09-21T04:15:25.643 INFO:tasks.workunit.client.0.smithi110.stdout:5/836: link d1/d22/lba d1/d22/d6e/l125 0 2024-09-21T04:15:25.643 INFO:tasks.workunit.client.0.smithi110.stdout:5/837: chown d1/d49/df1 1 1 2024-09-21T04:15:25.647 INFO:tasks.workunit.client.1.smithi148.stdout:5/848: sync 2024-09-21T04:15:25.650 INFO:tasks.workunit.client.0.smithi110.stdout:8/825: write d0/d7/d13/d4b/f28 [2192084,78627] 0 2024-09-21T04:15:25.650 INFO:tasks.workunit.client.0.smithi110.stdout:8/826: write d0/d7/d13/d4b/f28 [3295686,107542] 0 2024-09-21T04:15:25.651 INFO:tasks.workunit.client.0.smithi110.stdout:8/827: truncate d0/d7/d13/d1c/fa0 1822856 0 2024-09-21T04:15:25.651 INFO:tasks.workunit.client.0.smithi110.stdout:2/621: dread d2/d1a/d60/f8a [0,4194304] 0 2024-09-21T04:15:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/849: symlink d3/d34/d78/dbf/l122 0 2024-09-21T04:15:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/850: write d3/d14/d1a/f9a [5216061,32183] 0 2024-09-21T04:15:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/851: write d3/d4a/fe0 [686595,25121] 0 2024-09-21T04:15:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:0/875: dwrite d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/fe3 [0,4194304] 0 2024-09-21T04:15:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:0/876: fsync d5/dcb/de6/d110/d2a/dcd/df8/da5/f86 0 2024-09-21T04:15:25.653 INFO:tasks.workunit.client.0.smithi110.stdout:2/622: link d2/f44 d2/d1a/d1f/d5f/d9d/fbf 0 2024-09-21T04:15:25.653 INFO:tasks.workunit.client.0.smithi110.stdout:2/623: fsync d2/d1a/d1f/d2a/f61 0 2024-09-21T04:15:25.653 INFO:tasks.workunit.client.0.smithi110.stdout:2/624: chown d2/d1a/c5e 535822 1 2024-09-21T04:15:25.653 INFO:tasks.workunit.client.0.smithi110.stdout:2/625: chown d2/d1a/d1f/d5f/f1d 471756738 1 2024-09-21T04:15:25.654 INFO:tasks.workunit.client.1.smithi148.stdout:3/848: dwrite d3/d1f/d4d/d82/d23/d28/d46/f120 [0,4194304] 0 2024-09-21T04:15:25.655 INFO:tasks.workunit.client.1.smithi148.stdout:5/852: unlink d3/d14/d1f/d26/d2e/d56/da7/fab 0 2024-09-21T04:15:25.657 INFO:tasks.workunit.client.0.smithi110.stdout:2/626: dread d2/d1a/d60/f8a [0,4194304] 0 2024-09-21T04:15:25.659 INFO:tasks.workunit.client.0.smithi110.stdout:8/828: write d0/d7/f5c [3450043,62508] 0 2024-09-21T04:15:25.659 INFO:tasks.workunit.client.0.smithi110.stdout:8/829: fsync d0/d7/d23/d6c/f90 0 2024-09-21T04:15:25.662 INFO:tasks.workunit.client.0.smithi110.stdout:8/830: creat d0/d7/d13/d46/df8/f12c x:0 0 0 2024-09-21T04:15:25.663 INFO:tasks.workunit.client.1.smithi148.stdout:0/877: write d5/dcb/de6/d110/d2a/d2b/d99/db7/fc7 [3652794,98416] 0 2024-09-21T04:15:25.664 INFO:tasks.workunit.client.0.smithi110.stdout:8/831: mkdir d0/d7/d13/d4b/d108/d12d 0 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:5/838: dwrite d1/d49/d4d/d1e/d5a/f7a [0,4194304] 0 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:5/839: chown d1/d7/d18/d3f/d117 54321266 1 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:5/840: stat d1/d39/f21 0 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:5/841: chown d1/d7/d18/d8f/lcd 1361186 1 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:2/627: dread d2/d1a/d1f/d65/f68 [0,4194304] 0 2024-09-21T04:15:25.665 INFO:tasks.workunit.client.0.smithi110.stdout:2/628: chown d2/d1a/d1f/d5f/caa 645 1 2024-09-21T04:15:25.666 INFO:tasks.workunit.client.1.smithi148.stdout:0/878: mknod d5/dcb/de6/d110/d2a/dcd/df8/c117 0 2024-09-21T04:15:25.667 INFO:tasks.workunit.client.1.smithi148.stdout:0/879: fsync d5/dcb/de6/d110/d2a/d2b/f5b 0 2024-09-21T04:15:25.667 INFO:tasks.workunit.client.1.smithi148.stdout:5/853: write d3/d34/d3c/d5d/f93 [6800313,54759] 0 2024-09-21T04:15:25.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/854: fdatasync d3/d4a/fe0 0 2024-09-21T04:15:25.669 INFO:tasks.workunit.client.1.smithi148.stdout:5/855: write d3/d34/f64 [1155701,73657] 0 2024-09-21T04:15:25.672 INFO:tasks.workunit.client.1.smithi148.stdout:5/856: creat d3/d14/d1f/f123 x:0 0 0 2024-09-21T04:15:25.674 INFO:tasks.workunit.client.0.smithi110.stdout:2/629: rmdir d2/d50/d9c/db6 0 2024-09-21T04:15:25.674 INFO:tasks.workunit.client.1.smithi148.stdout:0/880: rmdir d5/d82/df0 39 2024-09-21T04:15:25.675 INFO:tasks.workunit.client.1.smithi148.stdout:0/881: write d5/dcb/de6/d110/d2a/d3f/d6d/da3/dcc/fdf [382796,91996] 0 2024-09-21T04:15:25.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/882: chown d5/dcb/de6/d110/d2a/dcd/df8/da5/f3a 833886335 1 2024-09-21T04:15:25.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/883: truncate d5/dcb/de6/d110/d2a/dcd/df8/da5/f3a 4395617 0 2024-09-21T04:15:25.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/849: dwrite d3/d1f/d4d/d82/d52/f4a [0,4194304] 0 2024-09-21T04:15:25.677 INFO:tasks.workunit.client.1.smithi148.stdout:5/857: rmdir d3/d14/d1f/d26/d59/d97 39 2024-09-21T04:15:25.678 INFO:tasks.workunit.client.1.smithi148.stdout:5/858: dread - d3/d24/ffd zero size 2024-09-21T04:15:25.680 INFO:tasks.workunit.client.1.smithi148.stdout:5/859: creat d3/d34/d78/f124 x:0 0 0 2024-09-21T04:15:25.682 INFO:tasks.workunit.client.1.smithi148.stdout:5/860: truncate d3/d4a/f52 3511232 0 2024-09-21T04:15:25.684 INFO:tasks.workunit.client.1.smithi148.stdout:3/850: dread d3/d1f/d4d/d82/d21/fbf [0,4194304] 0 2024-09-21T04:15:25.686 INFO:tasks.workunit.client.1.smithi148.stdout:3/851: write d3/d5/d6/d2b/da2/fc8 [73612,31373] 0 2024-09-21T04:15:25.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/852: creat d3/d1f/d4d/d82/d23/db5/d105/f122 x:0 0 0 2024-09-21T04:15:25.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/853: dread - d3/d1f/d4d/d82/d21/f4e zero size 2024-09-21T04:15:25.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/854: truncate d3/d5/d6/d12/d15/d73/f7c 2550434 0 2024-09-21T04:15:25.689 INFO:tasks.workunit.client.0.smithi110.stdout:5/842: dwrite d1/d22/d51/f70 [0,4194304] 0 2024-09-21T04:15:25.691 INFO:tasks.workunit.client.0.smithi110.stdout:5/843: creat d1/d22/d6e/f126 x:0 0 0 2024-09-21T04:15:25.693 INFO:tasks.workunit.client.1.smithi148.stdout:3/855: rename d3/d1f/d4d/d82/d74/fa0 to d3/d5/d6/d12/d43/d54/f123 0 2024-09-21T04:15:25.693 INFO:tasks.workunit.client.1.smithi148.stdout:3/856: rename d3/d5 to d3/d5/d6/d2b/d96/d124 22 2024-09-21T04:15:25.693 INFO:tasks.workunit.client.1.smithi148.stdout:3/857: dread - d3/d5/d6/d12/d43/fe2 zero size 2024-09-21T04:15:25.694 INFO:tasks.workunit.client.1.smithi148.stdout:3/858: rename d3/d1f/d4d/d82/d52/f4a to d3/d5/d6/d2b/d96/d10d/f125 0 2024-09-21T04:15:25.694 INFO:tasks.workunit.client.1.smithi148.stdout:3/859: chown d3/d1f/d4d/d82/d23/d28/d46/f9f 139 1 2024-09-21T04:15:25.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/884: dwrite d5/dcb/de6/d110/d2a/d2b/fd1 [0,4194304] 0 2024-09-21T04:15:25.709 INFO:tasks.workunit.client.0.smithi110.stdout:2/630: dwrite d2/d1a/d1f/d29/f4f [0,4194304] 0 2024-09-21T04:15:25.710 INFO:tasks.workunit.client.1.smithi148.stdout:3/860: dwrite d3/d5/d6/d12/d43/f11f [0,4194304] 0 2024-09-21T04:15:25.712 INFO:tasks.workunit.client.0.smithi110.stdout:2/631: mkdir d2/d1a/d1f/d5f/dc0 0 2024-09-21T04:15:25.716 INFO:tasks.workunit.client.0.smithi110.stdout:2/632: creat d2/d1a/d8b/da5/da9/fc1 x:0 0 0 2024-09-21T04:15:25.718 INFO:tasks.workunit.client.0.smithi110.stdout:2/633: symlink d2/d50/d56/lc2 0 2024-09-21T04:15:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:6/945: sync 2024-09-21T04:15:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:6/946: stat d0/d34/d97/d106/d23/d53 0 2024-09-21T04:15:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/787: sync 2024-09-21T04:15:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:3/861: dwrite d3/d5/d5c/f10e [0,4194304] 0 2024-09-21T04:15:25.729 INFO:tasks.workunit.client.1.smithi148.stdout:6/947: read d0/d34/d38/dc6/f109 [159964,121885] 0 2024-09-21T04:15:25.729 INFO:tasks.workunit.client.1.smithi148.stdout:6/948: truncate d0/d34/d38/dc6/f119 821363 0 2024-09-21T04:15:25.731 INFO:tasks.workunit.client.1.smithi148.stdout:1/860: sync 2024-09-21T04:15:25.734 INFO:tasks.workunit.client.1.smithi148.stdout:0/885: dwrite d5/dcb/de6/d110/d7/d11/f38 [0,4194304] 0 2024-09-21T04:15:25.734 INFO:tasks.workunit.client.1.smithi148.stdout:0/886: fdatasync d5/dcb/de6/d110/d7/dd0/fea 0 2024-09-21T04:15:25.734 INFO:tasks.workunit.client.1.smithi148.stdout:6/949: link d0/d34/d97/d106/d23/dc/d68/f86 d0/d34/d97/d106/d23/dc/d68/f12d 0 2024-09-21T04:15:25.740 INFO:tasks.workunit.client.0.smithi110.stdout:2/634: dwrite d2/d1a/d1f/d65/fa0 [0,4194304] 0 2024-09-21T04:15:25.740 INFO:tasks.workunit.client.1.smithi148.stdout:0/887: truncate d5/dcb/de6/d110/d2a/d3f/f8d 3927845 0 2024-09-21T04:15:25.742 INFO:tasks.workunit.client.1.smithi148.stdout:6/950: creat d0/d34/d97/d106/d23/d27/d80/da4/df1/d115/f12e x:0 0 0 2024-09-21T04:15:25.743 INFO:tasks.workunit.client.1.smithi148.stdout:3/862: rename d3/d5/d6/d2b/d96/db8/d10b to d3/d5/d5c/d126 0 2024-09-21T04:15:25.743 INFO:tasks.workunit.client.1.smithi148.stdout:3/863: readlink d3/d5/d5c/l8c 0 2024-09-21T04:15:25.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/861: write db/dd/f12 [139660,8408] 0 2024-09-21T04:15:25.745 INFO:tasks.workunit.client.1.smithi148.stdout:1/862: write db/d24/d2e/f7b [3543565,53086] 0 2024-09-21T04:15:25.746 INFO:tasks.workunit.client.1.smithi148.stdout:8/788: dwrite dc/d13/d76/d42/fd6 [0,4194304] 0 2024-09-21T04:15:25.746 INFO:tasks.workunit.client.1.smithi148.stdout:8/789: stat dc/d13/d76/d50/lf1 0 2024-09-21T04:15:25.746 INFO:tasks.workunit.client.0.smithi110.stdout:2/635: mkdir d2/d1a/d1f/d5f/d9d/d86/d9f/dc3 0 2024-09-21T04:15:25.747 INFO:tasks.workunit.client.0.smithi110.stdout:2/636: dread d2/d50/d56/f7e [0,4194304] 0 2024-09-21T04:15:25.748 INFO:tasks.workunit.client.0.smithi110.stdout:2/637: fdatasync d2/d1a/d1f/d29/f4f 0 2024-09-21T04:15:25.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/951: creat d0/d34/d97/d106/d23/d27/ddd/d107/f12f x:0 0 0 2024-09-21T04:15:25.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/952: write d0/d34/d97/d106/db8/fea [278332,73063] 0 2024-09-21T04:15:25.748 INFO:tasks.workunit.client.1.smithi148.stdout:6/953: chown d0/d34/da0/dd4 21321 1 2024-09-21T04:15:25.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/888: rename d5/dcb/de6/d110/d2a/d2b/f55 to d5/dcb/de6/d110/d7/dd0/d112/f118 0 2024-09-21T04:15:25.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/889: truncate d5/dcb/de6/d110/d7/d11/fc5 108128 0 2024-09-21T04:15:25.754 INFO:tasks.workunit.client.0.smithi110.stdout:8/832: rename d0/d30/d9d/dad to d0/d7/d23/d6c/d12e 0 2024-09-21T04:15:25.754 INFO:tasks.workunit.client.0.smithi110.stdout:8/833: fsync d0/d7/d13/d4b/f67 0 2024-09-21T04:15:25.754 INFO:tasks.workunit.client.0.smithi110.stdout:2/638: dread d2/d1a/d60/f8a [0,4194304] 0 2024-09-21T04:15:25.756 INFO:tasks.workunit.client.0.smithi110.stdout:5/844: rename d1/d49/l10a to d1/d22/l127 0 2024-09-21T04:15:25.760 INFO:tasks.workunit.client.0.smithi110.stdout:2/639: rename d2/d81/f9a to d2/d50/d56/fc4 0 2024-09-21T04:15:25.762 INFO:tasks.workunit.client.0.smithi110.stdout:5/845: symlink d1/d49/d4d/d1e/d4a/dae/l128 0 2024-09-21T04:15:25.763 INFO:tasks.workunit.client.0.smithi110.stdout:5/846: stat d1/d7/d18/d3f/f105 0 2024-09-21T04:15:25.763 INFO:tasks.workunit.client.0.smithi110.stdout:5/847: stat d1/d49/d4d/d1e/d4a/dae/l128 0 2024-09-21T04:15:25.764 INFO:tasks.workunit.client.1.smithi148.stdout:6/954: symlink d0/d34/d97/d106/d23/d27/d62/l130 0 2024-09-21T04:15:25.764 INFO:tasks.workunit.client.1.smithi148.stdout:6/955: write d0/d34/d97/d106/d23/d27/d80/da4/df1/d115/f12e [150296,76329] 0 2024-09-21T04:15:25.764 INFO:tasks.workunit.client.1.smithi148.stdout:6/956: chown d0/d34/d97/d106/d23/d27/d80/f94 62064196 1 2024-09-21T04:15:25.766 INFO:tasks.workunit.client.1.smithi148.stdout:0/890: symlink d5/dcb/de6/d110/d2a/dcd/df8/da5/l119 0 2024-09-21T04:15:25.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/790: dwrite dc/fa8 [0,4194304] 0 2024-09-21T04:15:25.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/791: chown dc/d13/d76/d50/df0 0 1 2024-09-21T04:15:25.767 INFO:tasks.workunit.client.1.smithi148.stdout:3/864: truncate d3/d1f/d4d/d82/d23/d28/d46/f120 144113 0 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.1.smithi148.stdout:1/863: dwrite db/d46/d4c/f86 [0,4194304] 0 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/640: symlink d2/lc5 0 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/641: chown d2/d1a/d1f/d29/d19/l84 22086280 1 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/642: fsync d2/d1a/d1f/d29/f4f 0 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/643: truncate d2/d1a/f88 4074297 0 2024-09-21T04:15:25.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/644: chown d2/d81/c85 1558118987 1 2024-09-21T04:15:25.769 INFO:tasks.workunit.client.1.smithi148.stdout:0/891: mknod d5/dcb/de6/d110/d7/d18/c11a 0 2024-09-21T04:15:25.776 INFO:tasks.workunit.client.0.smithi110.stdout:2/645: rmdir d2/d81/da8 0 2024-09-21T04:15:25.779 INFO:tasks.workunit.client.1.smithi148.stdout:6/957: dwrite d0/df9/d116/f4a [0,4194304] 0 2024-09-21T04:15:25.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/646: mknod d2/d81/cc6 0 2024-09-21T04:15:25.783 INFO:tasks.workunit.client.0.smithi110.stdout:5/848: rmdir d1/d22/db6/de1 0 2024-09-21T04:15:25.784 INFO:tasks.workunit.client.0.smithi110.stdout:5/849: chown d1/dc/ded 2667 1 2024-09-21T04:15:25.784 INFO:tasks.workunit.client.0.smithi110.stdout:5/850: fsync d1/d7/d58/d72/fc9 0 2024-09-21T04:15:25.784 INFO:tasks.workunit.client.0.smithi110.stdout:5/851: read - d1/d39/da2/fd8 zero size 2024-09-21T04:15:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/865: rename d3/d5/d6/d2b/d96/d10d to d3/d1f/d4d/d127 0 2024-09-21T04:15:25.784 INFO:tasks.workunit.client.1.smithi148.stdout:3/866: write d3/d1f/d4d/d82/d23/daf/fd3 [357857,103997] 0 2024-09-21T04:15:25.789 INFO:tasks.workunit.client.1.smithi148.stdout:1/864: mkdir db/d46/da0/d11e 0 2024-09-21T04:15:25.789 INFO:tasks.workunit.client.1.smithi148.stdout:1/865: fdatasync db/dd/fae 0 2024-09-21T04:15:25.790 INFO:tasks.workunit.client.1.smithi148.stdout:5/861: link d3/d4a/f52 d3/d34/d78/f125 0 2024-09-21T04:15:25.790 INFO:tasks.workunit.client.1.smithi148.stdout:5/862: fsync d3/d14/d1f/d72/fc5 0 2024-09-21T04:15:25.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/958: creat d0/d34/d97/d106/d23/d32/f131 x:0 0 0 2024-09-21T04:15:25.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/959: creat d0/d34/d38/f132 x:0 0 0 2024-09-21T04:15:25.796 INFO:tasks.workunit.client.0.smithi110.stdout:2/647: dread d2/d1a/d1f/d5f/d9d/f78 [0,4194304] 0 2024-09-21T04:15:25.799 INFO:tasks.workunit.client.1.smithi148.stdout:0/892: rename d5/d111 to d5/dcb/de6/d110/d7/d11/d11b 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/863: mknod d3/d14/d1f/db1/db8/c126 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:6/960: unlink d0/d8/d85/f9d 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:0/893: write d5/dcb/de6/d110/fdc [1293721,50494] 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:6/961: truncate d0/d34/da0/dd4/f10c 1044255 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:0/894: chown d5/dcb/de6/d110/d2a/d45/d70/l9e 24 1 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/864: fsync d3/d14/f2d 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/865: fsync d3/d14/d1f/d26/d2e/d3d/f89 0 2024-09-21T04:15:25.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/866: write d3/d34/d78/f114 [1002788,112619] 0 2024-09-21T04:15:25.801 INFO:tasks.workunit.client.1.smithi148.stdout:1/866: getdents db/d46/d4c/d82 0 2024-09-21T04:15:25.805 INFO:tasks.workunit.client.1.smithi148.stdout:6/962: getdents d0/d34/d97/d106/d23/d27/d80/db0 0 2024-09-21T04:15:25.805 INFO:tasks.workunit.client.1.smithi148.stdout:3/867: dwrite d3/d1f/d4d/d127/f125 [0,4194304] 0 2024-09-21T04:15:25.805 INFO:tasks.workunit.client.0.smithi110.stdout:2/648: dread d2/f3b [0,4194304] 0 2024-09-21T04:15:25.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/867: truncate d3/d14/d1f/d26/d2e/ffb 3867040 0 2024-09-21T04:15:25.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/868: creat d3/d14/d1f/d26/d2e/ddb/dde/f127 x:0 0 0 2024-09-21T04:15:25.807 INFO:tasks.workunit.client.1.smithi148.stdout:1/867: truncate db/dd/f13 464715 0 2024-09-21T04:15:25.807 INFO:tasks.workunit.client.0.smithi110.stdout:2/649: symlink d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/lc7 0 2024-09-21T04:15:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/895: link d5/dcb/de6/d110/d2a/d45/c92 d5/dcb/de6/d110/d7/dd0/c11c 0 2024-09-21T04:15:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/896: chown d5/dcb/de6/d110/d2a/d3f/d8b 218564 1 2024-09-21T04:15:25.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/897: dread - d5/dcb/de6/d110/d7/d22/f81 zero size 2024-09-21T04:15:25.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/963: rename d0/d34/da0/dd4/f10c to d0/d34/da0/f133 0 2024-09-21T04:15:25.811 INFO:tasks.workunit.client.0.smithi110.stdout:2/650: symlink d2/d50/d56/lc8 0 2024-09-21T04:15:25.812 INFO:tasks.workunit.client.1.smithi148.stdout:3/868: link d3/d5/d6/d2b/d96/d99/l115 d3/d5/dac/l128 0 2024-09-21T04:15:25.816 INFO:tasks.workunit.client.1.smithi148.stdout:5/869: link d3/d14/d1f/d26/d59/fb2 d3/d14/d1f/d26/d2e/ddb/deb/f128 0 2024-09-21T04:15:25.816 INFO:tasks.workunit.client.1.smithi148.stdout:5/870: readlink d3/d14/d1f/d26/d2e/l6e 0 2024-09-21T04:15:25.816 INFO:tasks.workunit.client.1.smithi148.stdout:1/868: creat db/d46/d8f/d106/f11f x:0 0 0 2024-09-21T04:15:25.817 INFO:tasks.workunit.client.1.smithi148.stdout:1/869: readlink db/l18 0 2024-09-21T04:15:25.817 INFO:tasks.workunit.client.1.smithi148.stdout:0/898: symlink d5/dcb/de6/d110/d7/d11/l11d 0 2024-09-21T04:15:25.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/869: symlink d3/d5/d6/d2b/da2/l129 0 2024-09-21T04:15:25.821 INFO:tasks.workunit.client.0.smithi110.stdout:2/651: dread d2/d1a/d1f/f5d [0,4194304] 0 2024-09-21T04:15:25.822 INFO:tasks.workunit.client.1.smithi148.stdout:5/871: link d3/d88/ldc d3/d14/d1f/d72/dc9/dd8/l129 0 2024-09-21T04:15:25.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/870: mkdir d3/d1f/d4d/d82/d21/d12a 0 2024-09-21T04:15:25.824 INFO:tasks.workunit.client.0.smithi110.stdout:2/652: creat d2/d1a/d90/fc9 x:0 0 0 2024-09-21T04:15:25.824 INFO:tasks.workunit.client.0.smithi110.stdout:2/653: chown d2/d50/d56/c33 931 1 2024-09-21T04:15:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:1/870: read db/d46/d4c/f5b [384979,87116] 0 2024-09-21T04:15:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:5/872: creat d3/d34/d3c/d5d/d75/f12a x:0 0 0 2024-09-21T04:15:25.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/871: creat db/dd/d21/db0/f120 x:0 0 0 2024-09-21T04:15:25.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/872: fsync db/d46/fb6 0 2024-09-21T04:15:25.826 INFO:tasks.workunit.client.1.smithi148.stdout:5/873: chown d3/d34/d3c/d5d/d75/fec 143 1 2024-09-21T04:15:25.827 INFO:tasks.workunit.client.0.smithi110.stdout:2/654: symlink d2/d1a/d1f/d65/lca 0 2024-09-21T04:15:25.828 INFO:tasks.workunit.client.1.smithi148.stdout:3/871: symlink d3/d5/d6/d12/l12b 0 2024-09-21T04:15:25.829 INFO:tasks.workunit.client.1.smithi148.stdout:1/873: rmdir db/d24/d2a/da4 39 2024-09-21T04:15:25.830 INFO:tasks.workunit.client.1.smithi148.stdout:1/874: creat db/d24/d2e/daf/f121 x:0 0 0 2024-09-21T04:15:25.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/964: dwrite d0/d8/d18/f60 [0,4194304] 0 2024-09-21T04:15:25.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/965: write d0/d34/d97/f30 [764178,88504] 0 2024-09-21T04:15:25.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/966: stat d0/d34/d38/dc6/fd3 0 2024-09-21T04:15:25.831 INFO:tasks.workunit.client.1.smithi148.stdout:6/967: readlink d0/d34/d97/d106/d23/d32/l6c 0 2024-09-21T04:15:25.831 INFO:tasks.workunit.client.0.smithi110.stdout:2/655: unlink d2/d81/l8e 0 2024-09-21T04:15:25.833 INFO:tasks.workunit.client.0.smithi110.stdout:2/656: creat d2/d81/fcb x:0 0 0 2024-09-21T04:15:25.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/899: dread d5/dcb/de6/d110/d7/d22/f24 [0,4194304] 0 2024-09-21T04:15:25.834 INFO:tasks.workunit.client.1.smithi148.stdout:0/900: fdatasync d5/dcb/de6/d110/d2a/dcd/df8/da5/f49 0 2024-09-21T04:15:25.835 INFO:tasks.workunit.client.0.smithi110.stdout:2/657: dread d2/d50/d9c/fab [0,4194304] 0 2024-09-21T04:15:25.838 INFO:tasks.workunit.client.0.smithi110.stdout:2/658: link d2/d1a/d1f/d29/c13 d2/d1a/d8b/da5/ccc 0 2024-09-21T04:15:25.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/659: readlink d2/d1a/d1f/d29/l30 0 2024-09-21T04:15:25.844 INFO:tasks.workunit.client.1.smithi148.stdout:6/968: dread d0/d34/d97/d106/d23/d32/d93/fa8 [0,4194304] 0 2024-09-21T04:15:25.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/792: write dc/d13/d76/d5d/f83 [1492523,119241] 0 2024-09-21T04:15:25.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/793: write dc/d13/d6a/fda [535462,102624] 0 2024-09-21T04:15:25.847 INFO:tasks.workunit.client.1.smithi148.stdout:8/794: stat dc/d13/d76/d42 0 2024-09-21T04:15:25.848 INFO:tasks.workunit.client.1.smithi148.stdout:5/874: dwrite d3/d34/d78/dbf/d6f/f10c [0,4194304] 0 2024-09-21T04:15:25.848 INFO:tasks.workunit.client.1.smithi148.stdout:5/875: chown d3/d14/d1f/d72/dc9/dd8/cf2 279919 1 2024-09-21T04:15:25.849 INFO:tasks.workunit.client.1.smithi148.stdout:6/969: truncate d0/d34/d97/d106/d23/d53/fc1 2127757 0 2024-09-21T04:15:25.849 INFO:tasks.workunit.client.1.smithi148.stdout:6/970: dread - d0/d59/f10a zero size 2024-09-21T04:15:25.850 INFO:tasks.workunit.client.1.smithi148.stdout:8/795: creat dc/d13/d39/ff2 x:0 0 0 2024-09-21T04:15:25.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/660: dread d2/d1a/d1f/d5f/d9d/f78 [0,4194304] 0 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/849: sync 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:4/937: sync 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:9/884: sync 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:9/885: chown d5/c11 6 1 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/850: fsync d5/d8/d6f/fc9 0 2024-09-21T04:15:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/851: chown d5/d8/d6f 663 1 2024-09-21T04:15:25.857 INFO:tasks.workunit.client.1.smithi148.stdout:8/796: mkdir dc/d13/d76/d50/df3 0 2024-09-21T04:15:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/886: link d5/ff d5/d60/f13b 0 2024-09-21T04:15:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/887: chown d5/d60/d28/d73/d108/f12f 53768 1 2024-09-21T04:15:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/888: truncate d5/ff 1064471 0 2024-09-21T04:15:25.864 INFO:tasks.workunit.client.1.smithi148.stdout:1/875: dwrite db/d24/d2e/f9c [0,4194304] 0 2024-09-21T04:15:25.866 INFO:tasks.workunit.client.1.smithi148.stdout:6/971: write d0/d34/d97/d4f/f8c [43285,9042] 0 2024-09-21T04:15:25.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/972: unlink d0/d34/d97/d106/d23/f19 0 2024-09-21T04:15:25.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/973: truncate d0/d8/dba/dbc/ffd 71902 0 2024-09-21T04:15:25.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/974: write d0/d34/d97/d106/d23/d27/d62/f67 [893269,10477] 0 2024-09-21T04:15:25.871 INFO:tasks.workunit.client.1.smithi148.stdout:6/975: mknod d0/d34/d97/d106/d23/d27/d80/d110/c134 0 2024-09-21T04:15:25.872 INFO:tasks.workunit.client.1.smithi148.stdout:9/889: read d5/d60/d4c/d5e/d134/f11d [340626,27147] 0 2024-09-21T04:15:25.872 INFO:tasks.workunit.client.1.smithi148.stdout:9/890: dread - d5/d1e/d37/d50/ff9 zero size 2024-09-21T04:15:25.873 INFO:tasks.workunit.client.1.smithi148.stdout:9/891: write d5/d60/d4c/f115 [290065,62698] 0 2024-09-21T04:15:25.873 INFO:tasks.workunit.client.1.smithi148.stdout:6/976: getdents d0/d34/d97/d106/d23/d27/d80/da4/df1 0 2024-09-21T04:15:25.875 INFO:tasks.workunit.client.1.smithi148.stdout:1/876: write db/dd/d21/f93 [3792598,59008] 0 2024-09-21T04:15:25.875 INFO:tasks.workunit.client.1.smithi148.stdout:1/877: creat db/dd/d21/db0/f122 x:0 0 0 2024-09-21T04:15:25.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/878: symlink db/d24/d2e/d59/da7/dc1/dd2/d10c/l123 0 2024-09-21T04:15:25.881 INFO:tasks.workunit.client.1.smithi148.stdout:6/977: write d0/d34/d97/d106/d23/d53/fc1 [3079470,75034] 0 2024-09-21T04:15:25.883 INFO:tasks.workunit.client.1.smithi148.stdout:6/978: creat d0/d8/d18/f135 x:0 0 0 2024-09-21T04:15:25.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/879: dread f6 [0,4194304] 0 2024-09-21T04:15:25.891 INFO:tasks.workunit.client.1.smithi148.stdout:1/880: unlink db/dd/l3f 0 2024-09-21T04:15:25.893 INFO:tasks.workunit.client.1.smithi148.stdout:1/881: rename db/d24/d2e/d59/fa2 to db/d24/d2e/daf/f124 0 2024-09-21T04:15:25.893 INFO:tasks.workunit.client.1.smithi148.stdout:1/882: chown db/dd/d21/d89/dc9/ce2 93124 1 2024-09-21T04:15:25.894 INFO:tasks.workunit.client.1.smithi148.stdout:1/883: mknod db/d46/d8f/d106/c125 0 2024-09-21T04:15:25.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/884: dread db/d24/d2e/daf/f124 [0,4194304] 0 2024-09-21T04:15:25.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/885: write db/d24/d2a/da4/f115 [75226,29740] 0 2024-09-21T04:15:25.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/886: fdatasync db/d24/d2e/f63 0 2024-09-21T04:15:25.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/887: read - db/d24/d2e/d59/da7/f103 zero size 2024-09-21T04:15:25.910 INFO:tasks.workunit.client.1.smithi148.stdout:1/888: symlink db/d46/d8f/d106/l126 0 2024-09-21T04:15:25.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/661: dwrite d2/d1a/d90/fc9 [0,4194304] 0 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:6/979: dwrite d0/d34/d97/d106/d23/d27/d80/da4/df1/d115/f12e [0,4194304] 0 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/938: dwrite d4/d66/fbe [0,4194304] 0 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/939: chown d4/d12/d16/d29/dbd/f68 20 1 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/940: fdatasync d4/f11 0 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/941: creat d4/d12/d83/d103/f127 x:0 0 0 2024-09-21T04:15:25.915 INFO:tasks.workunit.client.1.smithi148.stdout:4/942: read d4/d66/d7f/d2a/fbc [1269632,34362] 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:8/797: dwrite f5 [0,4194304] 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:7/852: dwrite d5/d8/f1e [0,4194304] 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:6/980: write d0/d34/d97/d106/d23/d27/d80/db0/fff [146292,7094] 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:6/981: getdents d0/d34/d97/d106/d23/dec 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/892: dwrite d5/d1f/fe6 [4194304,4194304] 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/893: fsync d5/d2b/d32/f5a 0 2024-09-21T04:15:25.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/894: write d5/d60/d28/f105 [256047,101908] 0 2024-09-21T04:15:25.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/876: write d3/d14/d1f/d26/d2e/d56/f15 [1025078,54525] 0 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/877: write d3/d34/d3c/d5d/d75/fe2 [768052,105323] 0 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/878: chown d3/d24/f2f 16963366 1 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:6/982: write d0/d34/d97/d106/d23/fdf [3290,29092] 0 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:6/983: chown d0/d34/d97/d4f/f8c 7 1 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:6/984: fdatasync d0/d34/d97/f52 0 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/879: creat d3/d34/d78/dbf/d6f/f12b x:0 0 0 2024-09-21T04:15:25.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/880: chown d3/d14/ld6 10 1 2024-09-21T04:15:25.920 INFO:tasks.workunit.client.1.smithi148.stdout:9/895: write d5/f1b [1493342,85063] 0 2024-09-21T04:15:25.922 INFO:tasks.workunit.client.0.smithi110.stdout:2/662: symlink d2/d1a/d1f/d29/d19/lcd 0 2024-09-21T04:15:25.924 INFO:tasks.workunit.client.0.smithi110.stdout:2/663: getdents d2/d1a/d8b/da5 0 2024-09-21T04:15:25.925 INFO:tasks.workunit.client.0.smithi110.stdout:2/664: mknod d2/d1a/d90/cce 0 2024-09-21T04:15:25.926 INFO:tasks.workunit.client.1.smithi148.stdout:1/889: dwrite db/dd/d21/f43 [0,4194304] 0 2024-09-21T04:15:25.926 INFO:tasks.workunit.client.1.smithi148.stdout:1/890: fdatasync db/d46/f66 0 2024-09-21T04:15:25.926 INFO:tasks.workunit.client.1.smithi148.stdout:7/853: creat d5/d14/d25/d40/dab/dad/f129 x:0 0 0 2024-09-21T04:15:25.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/854: chown d5/d14/d25/d79/d86/fd9 2 1 2024-09-21T04:15:25.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/855: creat d5/d14/d25/d40/d7b/f12a x:0 0 0 2024-09-21T04:15:25.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/856: dread d5/d8/d6f/d7d/d90/dee/f118 [0,4194304] 0 2024-09-21T04:15:25.928 INFO:tasks.workunit.client.1.smithi148.stdout:8/798: rename dc/d13/d76/dd8/le3 to dc/d13/d3c/d81/d9d/lf4 0 2024-09-21T04:15:25.928 INFO:tasks.workunit.client.1.smithi148.stdout:8/799: write dc/d13/d39/d6e/fd7 [979772,51435] 0 2024-09-21T04:15:25.928 INFO:tasks.workunit.client.1.smithi148.stdout:8/800: write dc/d43/d4c/d4f/fce [402933,80604] 0 2024-09-21T04:15:25.928 INFO:tasks.workunit.client.1.smithi148.stdout:8/801: creat dc/d13/d76/dd8/ff5 x:0 0 0 2024-09-21T04:15:25.929 INFO:tasks.workunit.client.0.smithi110.stdout:2/665: mknod d2/d1a/d8b/da5/da9/ccf 0 2024-09-21T04:15:25.929 INFO:tasks.workunit.client.0.smithi110.stdout:2/666: dread - d2/d81/fcb zero size 2024-09-21T04:15:25.932 INFO:tasks.workunit.client.1.smithi148.stdout:9/896: creat d5/d1e/d37/d50/d135/f13c x:0 0 0 2024-09-21T04:15:25.939 INFO:tasks.workunit.client.1.smithi148.stdout:1/891: getdents db/dd/d21 0 2024-09-21T04:15:25.939 INFO:tasks.workunit.client.1.smithi148.stdout:1/892: creat db/dd/d21/d89/f127 x:0 0 0 2024-09-21T04:15:25.939 INFO:tasks.workunit.client.1.smithi148.stdout:1/893: chown db/d24/d2a/dbd/fe8 1 1 2024-09-21T04:15:25.944 INFO:tasks.workunit.client.1.smithi148.stdout:7/857: creat d5/d14/d25/d40/db9/dff/f12b x:0 0 0 2024-09-21T04:15:25.946 INFO:tasks.workunit.client.0.smithi110.stdout:2/667: dwrite d2/d1a/da3/fb1 [0,4194304] 0 2024-09-21T04:15:25.946 INFO:tasks.workunit.client.1.smithi148.stdout:6/985: rename d0/d34/d38/dc6/fce to d0/d34/d97/d106/d23/f136 0 2024-09-21T04:15:25.950 INFO:tasks.workunit.client.1.smithi148.stdout:8/802: link dc/d13/d3c/d81/d85/da1/fca dc/d13/d76/d50/d80/ff6 0 2024-09-21T04:15:25.951 INFO:tasks.workunit.client.1.smithi148.stdout:9/897: getdents d5/d1e 0 2024-09-21T04:15:25.951 INFO:tasks.workunit.client.1.smithi148.stdout:9/898: truncate d5/f1b 2428682 0 2024-09-21T04:15:25.951 INFO:tasks.workunit.client.1.smithi148.stdout:9/899: write d5/d2b/dce/df6/f129 [276204,12065] 0 2024-09-21T04:15:25.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/668: symlink d2/d50/d9c/ld0 0 2024-09-21T04:15:25.953 INFO:tasks.workunit.client.0.smithi110.stdout:2/669: creat d2/d1a/d1f/fd1 x:0 0 0 2024-09-21T04:15:25.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/670: link d2/d1a/d1f/d2a/c82 d2/d1a/d1f/d29/d19/cd2 0 2024-09-21T04:15:25.959 INFO:tasks.workunit.client.0.smithi110.stdout:2/671: rename d2/d1a/c69 to d2/d1a/d1f/d5f/d9d/cd3 0 2024-09-21T04:15:25.960 INFO:tasks.workunit.client.1.smithi148.stdout:1/894: unlink db/d24/d2e/d71/d94/d9a/c62 0 2024-09-21T04:15:25.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/672: rename d2/d1a/d1f/d5f/d9d/d86/dad/lb8 to d2/d1a/d1f/d5f/dc0/ld4 0 2024-09-21T04:15:25.963 INFO:tasks.workunit.client.0.smithi110.stdout:2/673: rename d2/c99 to d2/d1a/d1f/d29/cd5 0 2024-09-21T04:15:25.965 INFO:tasks.workunit.client.0.smithi110.stdout:2/674: mknod d2/d1a/d1f/d5f/d9d/d86/d9b/cd6 0 2024-09-21T04:15:25.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/858: symlink d5/d8/d6f/l12c 0 2024-09-21T04:15:25.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/881: rename d3/d34/d3c/d5d/d75/f12a to d3/d14/d1f/d26/d2e/ddb/deb/f12c 0 2024-09-21T04:15:25.968 INFO:tasks.workunit.client.0.smithi110.stdout:2/675: mknod d2/d1a/d8b/da5/da9/cd7 0 2024-09-21T04:15:25.968 INFO:tasks.workunit.client.0.smithi110.stdout:2/676: fdatasync d2/d1a/d1f/d5f/f95 0 2024-09-21T04:15:25.969 INFO:tasks.workunit.client.1.smithi148.stdout:6/986: unlink d0/d34/d97/d106/d23/d27/dc2/de0/f12a 0 2024-09-21T04:15:25.971 INFO:tasks.workunit.client.1.smithi148.stdout:8/803: creat dc/d13/d76/d5d/ff7 x:0 0 0 2024-09-21T04:15:25.972 INFO:tasks.workunit.client.1.smithi148.stdout:8/804: truncate dc/d43/d4c/d4f/fce 897667 0 2024-09-21T04:15:25.972 INFO:tasks.workunit.client.1.smithi148.stdout:7/859: mknod d5/d8/d6f/d7d/d8a/c12d 0 2024-09-21T04:15:25.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/882: dread d3/d34/d78/dbf/d6f/f10c [0,4194304] 0 2024-09-21T04:15:25.974 INFO:tasks.workunit.client.1.smithi148.stdout:9/900: rename d5/d2b/d32/f5b to d5/d2b/dce/df6/f13d 0 2024-09-21T04:15:25.978 INFO:tasks.workunit.client.1.smithi148.stdout:6/987: creat d0/d8/dba/f137 x:0 0 0 2024-09-21T04:15:25.979 INFO:tasks.workunit.client.1.smithi148.stdout:6/988: chown d0/d34/d97/f3f 2891061 1 2024-09-21T04:15:25.980 INFO:tasks.workunit.client.1.smithi148.stdout:1/895: rename db/dd/f13 to db/dd/ddd/f128 0 2024-09-21T04:15:25.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/805: mkdir dc/d13/d39/df8 0 2024-09-21T04:15:25.982 INFO:tasks.workunit.client.0.smithi110.stdout:2/677: dwrite d2/d50/d56/f51 [0,4194304] 0 2024-09-21T04:15:25.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/896: dread db/d24/d2e/d71/d94/d9a/f95 [0,4194304] 0 2024-09-21T04:15:25.985 INFO:tasks.workunit.client.1.smithi148.stdout:8/806: creat dc/d13/ff9 x:0 0 0 2024-09-21T04:15:25.989 INFO:tasks.workunit.client.1.smithi148.stdout:8/807: creat dc/d13/d3c/d81/d85/da1/ffa x:0 0 0 2024-09-21T04:15:25.990 INFO:tasks.workunit.client.1.smithi148.stdout:7/860: rename d5/d14/d25/d40/db9/dff to d5/d14/d25/d79/d86/d12e 0 2024-09-21T04:15:25.991 INFO:tasks.workunit.client.1.smithi148.stdout:8/808: rmdir dc/d43/d4c/d4f/dc5 39 2024-09-21T04:15:25.993 INFO:tasks.workunit.client.1.smithi148.stdout:5/883: dwrite d3/d14/d1f/d26/d2e/d56/f11a [0,4194304] 0 2024-09-21T04:15:25.994 INFO:tasks.workunit.client.1.smithi148.stdout:1/897: rename db/d24/d2e/d71/dad to db/d24/d2e/d71/d94/dde/de1/d129 0 2024-09-21T04:15:25.994 INFO:tasks.workunit.client.1.smithi148.stdout:1/898: fsync f6 0 2024-09-21T04:15:25.994 INFO:tasks.workunit.client.1.smithi148.stdout:1/899: write db/d46/d76/fe3 [509780,123893] 0 2024-09-21T04:15:25.994 INFO:tasks.workunit.client.1.smithi148.stdout:1/900: write db/dd/fc3 [814930,4251] 0 2024-09-21T04:15:25.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/884: write d3/d14/d1f/d26/d2e/d56/f73 [128537,126229] 0 2024-09-21T04:15:25.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/885: fdatasync d3/d24/f2f 0 2024-09-21T04:15:25.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/886: dread - d3/d14/d1f/d26/d2e/ddb/dde/f127 zero size 2024-09-21T04:15:25.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/861: rename d5/d14/d25/fa2 to d5/d14/d25/d40/dab/dad/f12f 0 2024-09-21T04:15:25.999 INFO:tasks.workunit.client.1.smithi148.stdout:7/862: symlink d5/d8/d6f/d7d/d90/dee/l130 0 2024-09-21T04:15:26.000 INFO:tasks.workunit.client.0.smithi110.stdout:2/678: dwrite d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:26.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/863: rename d5/d8/d6f/d7d/d8a/c12d to d5/d8/d6f/d7d/d90/dee/c131 0 2024-09-21T04:15:26.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/864: chown d5/d14/d25/d40/dab/dad/fb4 5976 1 2024-09-21T04:15:26.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/865: readlink d5/d14/d25/d40/d7b/l97 0 2024-09-21T04:15:26.002 INFO:tasks.workunit.client.1.smithi148.stdout:7/866: fdatasync d5/d8/d6f/d7d/d90/f6e 0 2024-09-21T04:15:26.005 INFO:tasks.workunit.client.1.smithi148.stdout:7/867: mkdir d5/d8/d6f/dcd/d132 0 2024-09-21T04:15:26.005 INFO:tasks.workunit.client.1.smithi148.stdout:5/887: write d3/d24/fb0 [3980205,1110] 0 2024-09-21T04:15:26.006 INFO:tasks.workunit.client.1.smithi148.stdout:3/872: sync 2024-09-21T04:15:26.006 INFO:tasks.workunit.client.1.smithi148.stdout:0/901: sync 2024-09-21T04:15:26.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/902: fdatasync d5/fd8 0 2024-09-21T04:15:26.009 INFO:tasks.workunit.client.0.smithi110.stdout:2/679: dread d2/d1a/d1f/d5f/f6f [0,4194304] 0 2024-09-21T04:15:26.010 INFO:tasks.workunit.client.1.smithi148.stdout:3/873: creat d3/d1f/f12c x:0 0 0 2024-09-21T04:15:26.011 INFO:tasks.workunit.client.1.smithi148.stdout:0/903: dread d5/dcb/de6/d110/d2a/d2b/f5b [0,4194304] 0 2024-09-21T04:15:26.011 INFO:tasks.workunit.client.0.smithi110.stdout:2/680: mknod d2/d1a/d1f/d5f/d9d/d86/cd8 0 2024-09-21T04:15:26.011 INFO:tasks.workunit.client.0.smithi110.stdout:2/681: chown d2/d1a/d1f/d29/d19/cd2 13942 1 2024-09-21T04:15:26.012 INFO:tasks.workunit.client.1.smithi148.stdout:3/874: mknod d3/d5/d6/d12/d15/c12d 0 2024-09-21T04:15:26.013 INFO:tasks.workunit.client.1.smithi148.stdout:1/901: dwrite db/d24/d2e/d71/d94/dde/fea [0,4194304] 0 2024-09-21T04:15:26.015 INFO:tasks.workunit.client.1.smithi148.stdout:3/875: link d3/d1f/d4d/d82/d23/da1/cc7 d3/d5/c12e 0 2024-09-21T04:15:26.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/904: dread d5/dcb/de6/d110/d2a/d45/d70/fc6 [0,4194304] 0 2024-09-21T04:15:26.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/905: write d5/dcb/de6/d110/d7/d11/fca [80161,6039] 0 2024-09-21T04:15:26.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/902: link db/d46/d4c/cfe db/d24/d2e/d71/d94/dde/de1/c12a 0 2024-09-21T04:15:26.018 INFO:tasks.workunit.client.1.smithi148.stdout:1/903: truncate db/d24/d2e/f9c 4457322 0 2024-09-21T04:15:26.019 INFO:tasks.workunit.client.1.smithi148.stdout:3/876: rename d3/d5/d6/d12/d43/d54/f71 to d3/d1f/d4d/d82/d23/d28/d46/f12f 0 2024-09-21T04:15:26.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/906: creat d5/dcb/de6/d110/d7/dd0/f11e x:0 0 0 2024-09-21T04:15:26.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/907: dread - d5/dcb/de6/d110/d7/dd0/fea zero size 2024-09-21T04:15:26.020 INFO:tasks.workunit.client.1.smithi148.stdout:3/877: dread d3/d5/d6/d2b/da2/da3/fc5 [0,4194304] 0 2024-09-21T04:15:26.021 INFO:tasks.workunit.client.1.smithi148.stdout:1/904: creat db/d24/d2e/d59/f12b x:0 0 0 2024-09-21T04:15:26.022 INFO:tasks.workunit.client.1.smithi148.stdout:0/908: symlink d5/dcb/de6/d110/d2a/d2b/d99/l11f 0 2024-09-21T04:15:26.023 INFO:tasks.workunit.client.1.smithi148.stdout:1/905: symlink db/d46/d8f/l12c 0 2024-09-21T04:15:26.024 INFO:tasks.workunit.client.0.smithi110.stdout:2/682: dread d2/d1a/da3/fb1 [0,4194304] 0 2024-09-21T04:15:26.024 INFO:tasks.workunit.client.0.smithi110.stdout:2/683: stat d2/d1a/d1f/c28 0 2024-09-21T04:15:26.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/878: dwrite d3/fc0 [0,4194304] 0 2024-09-21T04:15:26.042 INFO:tasks.workunit.client.1.smithi148.stdout:1/906: dwrite db/d46/d8f/d106/f11f [0,4194304] 0 2024-09-21T04:15:26.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/879: write d3/d1f/d4d/d82/d23/d28/d46/f120 [418955,104876] 0 2024-09-21T04:15:26.045 INFO:tasks.workunit.client.1.smithi148.stdout:1/907: stat db/dd/c38 0 2024-09-21T04:15:26.046 INFO:tasks.workunit.client.1.smithi148.stdout:1/908: creat db/d24/d2e/d59/da7/dc1/dd2/d118/f12d x:0 0 0 2024-09-21T04:15:26.046 INFO:tasks.workunit.client.1.smithi148.stdout:1/909: mkdir db/d24/d2e/d71/d12e 0 2024-09-21T04:15:26.047 INFO:tasks.workunit.client.1.smithi148.stdout:1/910: chown db/dd/db9/dc2 9 1 2024-09-21T04:15:26.048 INFO:tasks.workunit.client.1.smithi148.stdout:1/911: creat db/d24/d2e/d71/d94/dde/de1/d129/f12f x:0 0 0 2024-09-21T04:15:26.048 INFO:tasks.workunit.client.1.smithi148.stdout:1/912: fsync db/d24/d2a/da4/f115 0 2024-09-21T04:15:26.071 INFO:tasks.workunit.client.1.smithi148.stdout:7/868: dwrite d5/d8/d6f/d7d/d90/d28/f9c [0,4194304] 0 2024-09-21T04:15:26.073 INFO:tasks.workunit.client.1.smithi148.stdout:7/869: mkdir d5/d8/d6f/d7d/d120/d133 0 2024-09-21T04:15:26.075 INFO:tasks.workunit.client.1.smithi148.stdout:7/870: unlink d5/d14/d25/d40/d8b/fe9 0 2024-09-21T04:15:26.076 INFO:tasks.workunit.client.1.smithi148.stdout:7/871: link d5/d14/c3f d5/d8/d6f/d7d/d90/d28/c134 0 2024-09-21T04:15:26.077 INFO:tasks.workunit.client.1.smithi148.stdout:7/872: creat d5/d8/d6f/d7d/d90/dee/f135 x:0 0 0 2024-09-21T04:15:26.077 INFO:tasks.workunit.client.1.smithi148.stdout:7/873: chown d5/d8/d6f/d7d/d90/dee/f135 4511 1 2024-09-21T04:15:26.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/913: dwrite db/d24/d2e/d71/d94/dde/de1/d129/f12f [0,4194304] 0 2024-09-21T04:15:26.089 INFO:tasks.workunit.client.1.smithi148.stdout:5/888: dwrite d3/d14/d1f/d26/d2e/d56/fe3 [0,4194304] 0 2024-09-21T04:15:26.090 INFO:tasks.workunit.client.1.smithi148.stdout:1/914: symlink db/d24/d2e/d71/d94/d9a/l130 0 2024-09-21T04:15:26.090 INFO:tasks.workunit.client.1.smithi148.stdout:1/915: write db/d24/d2e/d59/da7/dc1/dd2/d118/f12d [792483,36667] 0 2024-09-21T04:15:26.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/889: symlink d3/d4a/d11c/l12d 0 2024-09-21T04:15:26.092 INFO:tasks.workunit.client.1.smithi148.stdout:1/916: link db/d24/d2e/d71/d94/dde/de1/l111 db/d24/d2a/dbd/df5/l131 0 2024-09-21T04:15:26.092 INFO:tasks.workunit.client.1.smithi148.stdout:5/890: mknod d3/d14/d1f/dfc/c12e 0 2024-09-21T04:15:26.093 INFO:tasks.workunit.client.1.smithi148.stdout:7/874: dread d5/d8/d6f/fc9 [0,4194304] 0 2024-09-21T04:15:26.093 INFO:tasks.workunit.client.1.smithi148.stdout:7/875: dread - d5/d14/d25/d40/d7b/f119 zero size 2024-09-21T04:15:26.094 INFO:tasks.workunit.client.1.smithi148.stdout:1/917: unlink db/d24/d2e/d59/f7e 0 2024-09-21T04:15:26.094 INFO:tasks.workunit.client.1.smithi148.stdout:1/918: read db/d24/f2b [2626109,84523] 0 2024-09-21T04:15:26.094 INFO:tasks.workunit.client.1.smithi148.stdout:1/919: readlink db/d24/d2a/l81 0 2024-09-21T04:15:26.094 INFO:tasks.workunit.client.1.smithi148.stdout:7/876: rename d5/d8/d6f/d7d/cd8 to d5/d14/d25/d40/dab/dad/db8/d125/c136 0 2024-09-21T04:15:26.095 INFO:tasks.workunit.client.1.smithi148.stdout:1/920: stat db/d24/d2e/d71/d94/dde/de1/l111 0 2024-09-21T04:15:26.095 INFO:tasks.workunit.client.1.smithi148.stdout:3/880: dwrite d3/d1f/d4d/d82/d52/f59 [0,4194304] 0 2024-09-21T04:15:26.095 INFO:tasks.workunit.client.1.smithi148.stdout:0/909: dwrite d5/dcb/de6/d110/fdc [0,4194304] 0 2024-09-21T04:15:26.097 INFO:tasks.workunit.client.0.smithi110.stdout:2/684: dwrite d2/d1a/d1f/d5f/d9d/d86/d9b/fb4 [0,4194304] 0 2024-09-21T04:15:26.103 INFO:tasks.workunit.client.0.smithi110.stdout:2/685: mknod d2/d1a/d60/cd9 0 2024-09-21T04:15:26.111 INFO:tasks.workunit.client.1.smithi148.stdout:3/881: write d3/d1f/d4d/d82/d23/d28/d46/db3/fea [3483079,70245] 0 2024-09-21T04:15:26.111 INFO:tasks.workunit.client.1.smithi148.stdout:3/882: creat d3/d1f/d4d/d82/d23/db5/d105/f130 x:0 0 0 2024-09-21T04:15:26.111 INFO:tasks.workunit.client.1.smithi148.stdout:3/883: stat d3/d5/d6/d12/d43/d54 0 2024-09-21T04:15:26.111 INFO:tasks.workunit.client.1.smithi148.stdout:0/910: creat d5/dcb/de6/d110/d2a/d45/f120 x:0 0 0 2024-09-21T04:15:26.112 INFO:tasks.workunit.client.1.smithi148.stdout:7/877: getdents d5/d8/d6f/d7d/d90/dee 0 2024-09-21T04:15:26.115 INFO:tasks.workunit.client.1.smithi148.stdout:5/891: dwrite d3/d14/d1f/d26/d2e/d56/fe3 [4194304,4194304] 0 2024-09-21T04:15:26.117 INFO:tasks.workunit.client.1.smithi148.stdout:0/911: truncate d5/dcb/de6/d110/d2a/d2b/fd3 909640 0 2024-09-21T04:15:26.118 INFO:tasks.workunit.client.1.smithi148.stdout:7/878: creat d5/d8/d64/f137 x:0 0 0 2024-09-21T04:15:26.120 INFO:tasks.workunit.client.1.smithi148.stdout:5/892: truncate d3/d34/d78/fbe 2591900 0 2024-09-21T04:15:26.120 INFO:tasks.workunit.client.1.smithi148.stdout:1/921: dwrite db/d46/d4c/da3/fff [0,4194304] 0 2024-09-21T04:15:26.120 INFO:tasks.workunit.client.1.smithi148.stdout:1/922: dread - db/dd/d21/d89/f127 zero size 2024-09-21T04:15:26.120 INFO:tasks.workunit.client.1.smithi148.stdout:1/923: chown db/l49 5218 1 2024-09-21T04:15:26.123 INFO:tasks.workunit.client.1.smithi148.stdout:3/884: write d3/d1f/d4d/d82/d23/d28/d46/db3/fb7 [1303628,120395] 0 2024-09-21T04:15:26.123 INFO:tasks.workunit.client.1.smithi148.stdout:3/885: dread - d3/d1f/d4d/ff5 zero size 2024-09-21T04:15:26.125 INFO:tasks.workunit.client.1.smithi148.stdout:0/912: dread d5/dcb/de6/d110/d2a/d2b/d99/db7/fc7 [0,4194304] 0 2024-09-21T04:15:26.125 INFO:tasks.workunit.client.1.smithi148.stdout:0/913: write d5/dcb/de6/d110/d7/d22/f24 [3703295,96712] 0 2024-09-21T04:15:26.126 INFO:tasks.workunit.client.1.smithi148.stdout:7/879: rename d5/d8/d6f/cd6 to d5/d14/d25/d79/d86/d12e/c138 0 2024-09-21T04:15:26.126 INFO:tasks.workunit.client.1.smithi148.stdout:7/880: write d5/d8/d6f/d7d/d8a/fa1 [423853,103557] 0 2024-09-21T04:15:26.126 INFO:tasks.workunit.client.1.smithi148.stdout:7/881: read - d5/daa/f11c zero size 2024-09-21T04:15:26.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/914: mknod d5/dcb/de6/d110/d2a/d3f/d6d/da3/df4/c121 0 2024-09-21T04:15:26.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/915: creat d5/dcb/de6/d110/d7/dd0/f122 x:0 0 0 2024-09-21T04:15:26.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/943: sync 2024-09-21T04:15:26.130 INFO:tasks.workunit.client.1.smithi148.stdout:0/916: dread - d5/dcb/de6/d110/d2a/d45/fe5 zero size 2024-09-21T04:15:26.133 INFO:tasks.workunit.client.1.smithi148.stdout:0/917: rename d5/dcb/de6/d110/d2a/d45/f120 to d5/dcb/de6/d110/d2a/d2b/d99/f123 0 2024-09-21T04:15:26.134 INFO:tasks.workunit.client.1.smithi148.stdout:0/918: fsync d5/dcb/de6/d110/f5a 0 2024-09-21T04:15:26.137 INFO:tasks.workunit.client.1.smithi148.stdout:0/919: mknod d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/de8/df7/c124 0 2024-09-21T04:15:26.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/901: sync 2024-09-21T04:15:26.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/809: sync 2024-09-21T04:15:26.139 INFO:tasks.workunit.client.1.smithi148.stdout:6/989: sync 2024-09-21T04:15:26.139 INFO:tasks.workunit.client.1.smithi148.stdout:6/990: chown d0/d8/dba 1514637 1 2024-09-21T04:15:26.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/920: getdents d5/dcb/de6/d110/d2a/d3f/d6d/da3 0 2024-09-21T04:15:26.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/921: write d5/dcb/de6/d110/d2a/d45/d70/f7f [693817,44136] 0 2024-09-21T04:15:26.140 INFO:tasks.workunit.client.1.smithi148.stdout:9/902: mkdir d5/d60/de1/d13e 0 2024-09-21T04:15:26.140 INFO:tasks.workunit.client.1.smithi148.stdout:9/903: stat d5/c34 0 2024-09-21T04:15:26.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/810: link dc/d13/d76/c67 dc/d13/d28/d62/cfb 0 2024-09-21T04:15:26.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/922: rmdir d5/dcb/de6/d110/d2a/d3f/d8b 39 2024-09-21T04:15:26.143 INFO:tasks.workunit.client.1.smithi148.stdout:8/811: creat dc/d13/d39/d6e/ffc x:0 0 0 2024-09-21T04:15:26.143 INFO:tasks.workunit.client.1.smithi148.stdout:6/991: dread d0/d34/d38/dc6/f109 [0,4194304] 0 2024-09-21T04:15:26.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/923: creat d5/dcb/de6/d110/d2a/f125 x:0 0 0 2024-09-21T04:15:26.145 INFO:tasks.workunit.client.1.smithi148.stdout:6/992: rename d0/d34/d38/d57/d58 to d0/d34/d97/d106/d23/d27/d80/d110/d127/d138 0 2024-09-21T04:15:26.146 INFO:tasks.workunit.client.1.smithi148.stdout:6/993: fdatasync d0/d34/d97/d4f/f8c 0 2024-09-21T04:15:26.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/924: rename d5/dcb/de6/d110/d7/d18 to d5/d82/df9/d126 0 2024-09-21T04:15:26.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/893: dwrite d3/d14/d1f/d26/d2e/d3d/f67 [0,4194304] 0 2024-09-21T04:15:26.152 INFO:tasks.workunit.client.1.smithi148.stdout:5/894: creat d3/d14/d1a/f12f x:0 0 0 2024-09-21T04:15:26.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/882: dwrite d5/d8/f1b [0,4194304] 0 2024-09-21T04:15:26.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/925: dread d5/dcb/de6/d110/d2a/dcd/df8/da5/f27 [0,4194304] 0 2024-09-21T04:15:26.152 INFO:tasks.workunit.client.1.smithi148.stdout:1/924: dwrite db/d24/f8b [0,4194304] 0 2024-09-21T04:15:26.155 INFO:tasks.workunit.client.1.smithi148.stdout:7/883: symlink d5/d14/d25/d79/d86/d12e/l139 0 2024-09-21T04:15:26.155 INFO:tasks.workunit.client.1.smithi148.stdout:7/884: dread - d5/d8/d6f/d7d/d90/d28/f6d zero size 2024-09-21T04:15:26.156 INFO:tasks.workunit.client.1.smithi148.stdout:5/895: dread d3/d34/d78/fa1 [0,4194304] 0 2024-09-21T04:15:26.157 INFO:tasks.workunit.client.1.smithi148.stdout:5/896: truncate d3/d14/d1f/d26/d2e/d3d/fa2 1024768 0 2024-09-21T04:15:26.157 INFO:tasks.workunit.client.1.smithi148.stdout:0/926: creat d5/dcb/de6/d110/f127 x:0 0 0 2024-09-21T04:15:26.158 INFO:tasks.workunit.client.1.smithi148.stdout:1/925: creat db/d24/d2e/d59/da7/dc1/f132 x:0 0 0 2024-09-21T04:15:26.158 INFO:tasks.workunit.client.1.smithi148.stdout:1/926: chown db/d24/d2e/d71/d94/ff3 1619097837 1 2024-09-21T04:15:26.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/897: symlink d3/d14/d1f/d26/d2e/ddb/l130 0 2024-09-21T04:15:26.161 INFO:tasks.workunit.client.1.smithi148.stdout:8/812: dwrite dc/d13/d28/de4/fea [0,4194304] 0 2024-09-21T04:15:26.163 INFO:tasks.workunit.client.1.smithi148.stdout:7/885: rename d5/d8/d6f/d7d/f7e to d5/d14/d25/d40/f13a 0 2024-09-21T04:15:26.166 INFO:tasks.workunit.client.1.smithi148.stdout:7/886: creat d5/d8/d6f/d7d/df5/f13b x:0 0 0 2024-09-21T04:15:26.166 INFO:tasks.workunit.client.1.smithi148.stdout:7/887: chown d5/d8/f10c 16978 1 2024-09-21T04:15:26.168 INFO:tasks.workunit.client.1.smithi148.stdout:7/888: mkdir d5/d8/d6f/d7d/d90/d13c 0 2024-09-21T04:15:26.169 INFO:tasks.workunit.client.1.smithi148.stdout:7/889: dread d5/daa/fc5 [0,4194304] 0 2024-09-21T04:15:26.170 INFO:tasks.workunit.client.1.smithi148.stdout:1/927: rmdir db/d24/d2e/d59/da7/dc1 39 2024-09-21T04:15:26.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/890: rename d5/d8/f91 to d5/d8/d6f/d7d/d120/f13d 0 2024-09-21T04:15:26.175 INFO:tasks.workunit.client.1.smithi148.stdout:7/891: dread d5/d8/f9 [0,4194304] 0 2024-09-21T04:15:26.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/892: readlink d5/d14/lae 0 2024-09-21T04:15:26.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/893: readlink d5/d14/lae 0 2024-09-21T04:15:26.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/894: stat d5/d14/d25/d79/fed 0 2024-09-21T04:15:26.177 INFO:tasks.workunit.client.1.smithi148.stdout:7/895: link d5/d14/d25/d40/dab/dad/fe5 d5/d8/d6f/d7d/d120/d133/f13e 0 2024-09-21T04:15:26.179 INFO:tasks.workunit.client.1.smithi148.stdout:1/928: dread db/d46/d4c/f5b [0,4194304] 0 2024-09-21T04:15:26.180 INFO:tasks.workunit.client.1.smithi148.stdout:1/929: creat db/d24/d2e/d71/d12e/f133 x:0 0 0 2024-09-21T04:15:26.181 INFO:tasks.workunit.client.1.smithi148.stdout:0/927: write d5/dcb/de6/d110/d2a/dcd/df8/da5/f3a [4126540,11333] 0 2024-09-21T04:15:26.182 INFO:tasks.workunit.client.1.smithi148.stdout:0/928: chown d5/dcb/de6/d110/d7/d22/cb1 97408140 1 2024-09-21T04:15:26.185 INFO:tasks.workunit.client.1.smithi148.stdout:7/896: read d5/d14/d25/d40/d7b/f2b [1559833,35246] 0 2024-09-21T04:15:26.186 INFO:tasks.workunit.client.1.smithi148.stdout:7/897: chown d5/l60 3 1 2024-09-21T04:15:26.190 INFO:tasks.workunit.client.1.smithi148.stdout:7/898: dread d5/d14/d25/d40/dab/dad/f12f [0,4194304] 0 2024-09-21T04:15:26.191 INFO:tasks.workunit.client.1.smithi148.stdout:8/813: dwrite dc/d43/f54 [0,4194304] 0 2024-09-21T04:15:26.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/899: creat d5/d8/d6f/d7d/f13f x:0 0 0 2024-09-21T04:15:26.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/900: write d5/d14/d25/d40/db9/f10e [867707,110819] 0 2024-09-21T04:15:26.191 INFO:tasks.workunit.client.1.smithi148.stdout:8/814: creat dc/d3e/ffd x:0 0 0 2024-09-21T04:15:26.192 INFO:tasks.workunit.client.1.smithi148.stdout:7/901: mkdir d5/d14/d25/d140 0 2024-09-21T04:15:26.193 INFO:tasks.workunit.client.1.smithi148.stdout:7/902: mkdir d5/d8/d6f/d7d/d120/d133/d141 0 2024-09-21T04:15:26.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/929: dread d5/dcb/de6/d110/d7/f4f [0,4194304] 0 2024-09-21T04:15:26.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/930: chown d5/dcb/de6/d110/d2a/dcd/df8/db4 8 1 2024-09-21T04:15:26.201 INFO:tasks.workunit.client.1.smithi148.stdout:0/931: write f4 [1889648,76343] 0 2024-09-21T04:15:26.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/815: dread dc/d13/d39/d6e/dac/faf [0,4194304] 0 2024-09-21T04:15:26.203 INFO:tasks.workunit.client.1.smithi148.stdout:7/903: dread d5/d8/d6f/d7d/f7f [0,4194304] 0 2024-09-21T04:15:26.203 INFO:tasks.workunit.client.1.smithi148.stdout:7/904: creat d5/d8/d6f/dcd/d132/f142 x:0 0 0 2024-09-21T04:15:26.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/905: rmdir d5/daa 39 2024-09-21T04:15:26.223 INFO:tasks.workunit.client.1.smithi148.stdout:8/816: dwrite dc/d13/d39/d6e/dac/faf [4194304,4194304] 0 2024-09-21T04:15:26.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/932: dwrite d5/dcb/de6/d110/d2a/d3f/d6d/da3/fda [4194304,4194304] 0 2024-09-21T04:15:26.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/933: chown d5/dcb/de6/d110/d7/dd0/d112/f118 249 1 2024-09-21T04:15:26.224 INFO:tasks.workunit.client.1.smithi148.stdout:8/817: creat dc/d13/d76/d50/df0/ffe x:0 0 0 2024-09-21T04:15:26.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/934: link d5/dcb/de6/d110/d7/d11/fcf d5/dcb/de6/d110/d7/f128 0 2024-09-21T04:15:26.238 INFO:tasks.workunit.client.1.smithi148.stdout:7/906: dwrite d5/d8/f10c [0,4194304] 0 2024-09-21T04:15:26.240 INFO:tasks.workunit.client.1.smithi148.stdout:7/907: creat d5/d8/d6f/f143 x:0 0 0 2024-09-21T04:15:26.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/908: creat d5/d14/d25/d79/f144 x:0 0 0 2024-09-21T04:15:26.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/909: mkdir d5/d8/d64/d145 0 2024-09-21T04:15:26.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/910: rmdir d5/d8/d64/d145 0 2024-09-21T04:15:26.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/911: fdatasync d5/d14/d25/d40/d7b/f12a 0 2024-09-21T04:15:26.243 INFO:tasks.workunit.client.1.smithi148.stdout:7/912: rmdir d5/d14/d25/d40/d7b 39 2024-09-21T04:15:26.244 INFO:tasks.workunit.client.1.smithi148.stdout:0/935: dwrite d5/dcb/de6/d110/d7/d11/f1d [0,4194304] 0 2024-09-21T04:15:26.244 INFO:tasks.workunit.client.1.smithi148.stdout:8/818: dwrite dc/d3e/fd2 [0,4194304] 0 2024-09-21T04:15:26.245 INFO:tasks.workunit.client.1.smithi148.stdout:0/936: mknod d5/dcb/de6/d110/d7/dd0/d112/c129 0 2024-09-21T04:15:26.245 INFO:tasks.workunit.client.1.smithi148.stdout:0/937: stat d5/dcb/de6/d110/f62 0 2024-09-21T04:15:26.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/819: dread dc/d3e/db4/fe2 [0,4194304] 0 2024-09-21T04:15:26.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/820: dread - dc/d13/d39/fc6 zero size 2024-09-21T04:15:26.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/821: chown dc/f46 127562 1 2024-09-21T04:15:26.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/822: chown dc/d43/d4c/d4f/l59 805513 1 2024-09-21T04:15:26.249 INFO:tasks.workunit.client.1.smithi148.stdout:8/823: dread dc/f41 [0,4194304] 0 2024-09-21T04:15:26.250 INFO:tasks.workunit.client.1.smithi148.stdout:8/824: creat dc/d13/d76/d50/fff x:0 0 0 2024-09-21T04:15:26.251 INFO:tasks.workunit.client.1.smithi148.stdout:8/825: symlink dc/d13/d76/l100 0 2024-09-21T04:15:26.251 INFO:tasks.workunit.client.1.smithi148.stdout:8/826: fdatasync dc/d13/d76/d50/d80/ff6 0 2024-09-21T04:15:26.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/827: symlink dc/d13/d3c/d81/l101 0 2024-09-21T04:15:26.253 INFO:tasks.workunit.client.1.smithi148.stdout:0/938: dread d5/d82/df9/d126/f32 [0,4194304] 0 2024-09-21T04:15:26.255 INFO:tasks.workunit.client.1.smithi148.stdout:0/939: symlink d5/dcb/de6/d110/l12a 0 2024-09-21T04:15:26.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/940: chown d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/fe3 883 1 2024-09-21T04:15:26.256 INFO:tasks.workunit.client.1.smithi148.stdout:7/913: dread d5/d8/d6f/d7d/d120/f13d [0,4194304] 0 2024-09-21T04:15:26.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/941: link d5/dcb/de6/d110/d2a/d3f/d8b/fc2 d5/dcb/de6/d110/d7/f12b 0 2024-09-21T04:15:26.257 INFO:tasks.workunit.client.1.smithi148.stdout:0/942: dread - d5/dcb/de6/d110/d7/dd0/f122 zero size 2024-09-21T04:15:26.258 INFO:tasks.workunit.client.1.smithi148.stdout:7/914: creat d5/d14/d25/d40/db9/d108/f146 x:0 0 0 2024-09-21T04:15:26.259 INFO:tasks.workunit.client.1.smithi148.stdout:0/943: symlink d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/l12c 0 2024-09-21T04:15:26.259 INFO:tasks.workunit.client.1.smithi148.stdout:0/944: fdatasync d5/dcb/de6/d110/d7/d22/f8a 0 2024-09-21T04:15:26.259 INFO:tasks.workunit.client.1.smithi148.stdout:0/945: write d5/dcb/de6/d110/f62 [3239115,86631] 0 2024-09-21T04:15:26.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/915: link d5/d8/d6f/f9d d5/daa/f147 0 2024-09-21T04:15:26.269 INFO:tasks.workunit.client.1.smithi148.stdout:0/946: dread d5/dcb/de6/d110/d7/d11/f1d [0,4194304] 0 2024-09-21T04:15:26.271 INFO:tasks.workunit.client.1.smithi148.stdout:0/947: creat d5/dcb/de6/d110/d2a/d3f/d6d/da3/df4/f12d x:0 0 0 2024-09-21T04:15:26.271 INFO:tasks.workunit.client.1.smithi148.stdout:0/948: dread - d5/dcb/de6/d110/d7/dd0/f11e zero size 2024-09-21T04:15:26.276 INFO:tasks.workunit.client.1.smithi148.stdout:7/916: dwrite d5/d8/d6f/d7d/d90/dee/f104 [0,4194304] 0 2024-09-21T04:15:26.277 INFO:tasks.workunit.client.1.smithi148.stdout:7/917: mknod d5/d8/d6f/d7d/d120/d133/d141/c148 0 2024-09-21T04:15:26.279 INFO:tasks.workunit.client.1.smithi148.stdout:7/918: mkdir d5/d8/d6f/d7d/df5/d149 0 2024-09-21T04:15:26.279 INFO:tasks.workunit.client.1.smithi148.stdout:7/919: fdatasync d5/d8/d6f/d7d/d90/f5d 0 2024-09-21T04:15:26.281 INFO:tasks.workunit.client.1.smithi148.stdout:7/920: link d5/d14/d25/d40/d7b/l38 d5/d8/l14a 0 2024-09-21T04:15:26.282 INFO:tasks.workunit.client.1.smithi148.stdout:7/921: unlink d5/d14/f78 0 2024-09-21T04:15:26.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/922: unlink d5/d8/d6f/d7d/d90/d9b/c102 0 2024-09-21T04:15:26.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/923: dread - d5/daa/fd0 zero size 2024-09-21T04:15:26.285 INFO:tasks.workunit.client.1.smithi148.stdout:7/924: dread d5/d8/d6f/d7d/d90/f5d [0,4194304] 0 2024-09-21T04:15:26.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/949: dwrite d5/dcb/de6/d110/f62 [0,4194304] 0 2024-09-21T04:15:26.295 INFO:tasks.workunit.client.1.smithi148.stdout:3/886: sync 2024-09-21T04:15:26.298 INFO:tasks.workunit.client.1.smithi148.stdout:3/887: dread d3/f55 [0,4194304] 0 2024-09-21T04:15:26.299 INFO:tasks.workunit.client.1.smithi148.stdout:3/888: mkdir d3/d5/d6/d12/d43/d54/d9b/d131 0 2024-09-21T04:15:26.301 INFO:tasks.workunit.client.1.smithi148.stdout:7/925: dwrite d5/daa/fd0 [0,4194304] 0 2024-09-21T04:15:26.302 INFO:tasks.workunit.client.1.smithi148.stdout:7/926: symlink d5/d14/d25/d40/dab/dad/d111/l14b 0 2024-09-21T04:15:26.303 INFO:tasks.workunit.client.1.smithi148.stdout:0/950: dread d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/fe3 [0,4194304] 0 2024-09-21T04:15:26.321 INFO:tasks.workunit.client.1.smithi148.stdout:7/927: dwrite d5/d14/d25/d40/d8b/f113 [0,4194304] 0 2024-09-21T04:15:26.326 INFO:tasks.workunit.client.1.smithi148.stdout:3/889: dwrite d3/d1f/d4d/d82/d23/db5/d105/f8f [0,4194304] 0 2024-09-21T04:15:26.326 INFO:tasks.workunit.client.1.smithi148.stdout:3/890: truncate d3/d5/d6/f17 8990373 0 2024-09-21T04:15:26.327 INFO:tasks.workunit.client.1.smithi148.stdout:0/951: dwrite d5/dcb/de6/d110/d2a/d45/f6e [0,4194304] 0 2024-09-21T04:15:26.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/952: chown d5/dcb/de6/d110/d7/f12b 24 1 2024-09-21T04:15:26.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/953: stat d5/dcb/de6/d110/d2a/d45/d70/lfb 0 2024-09-21T04:15:26.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/954: dread - d5/dcb/de6/d110/f127 zero size 2024-09-21T04:15:26.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/955: dread d5/dcb/de6/d110/d7/d11/f17 [4194304,4194304] 0 2024-09-21T04:15:26.334 INFO:tasks.workunit.client.1.smithi148.stdout:0/956: mkdir d5/dcb/de6/d110/d2a/dcd/df8/daf/d12e 0 2024-09-21T04:15:26.334 INFO:tasks.workunit.client.1.smithi148.stdout:0/957: truncate d5/dcb/de6/d110/d2a/dcd/df8/da5/f86 1058259 0 2024-09-21T04:15:26.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/891: dread d3/d5/f41 [0,4194304] 0 2024-09-21T04:15:26.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/892: dread - d3/d5/fbe zero size 2024-09-21T04:15:26.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/893: readlink d3/d1f/d4d/l68 0 2024-09-21T04:15:26.336 INFO:tasks.workunit.client.1.smithi148.stdout:0/958: rename d5/dcb/de6/d110/l8f to d5/d82/l12f 0 2024-09-21T04:15:26.336 INFO:tasks.workunit.client.1.smithi148.stdout:0/959: symlink d5/dcb/l130 0 2024-09-21T04:15:26.338 INFO:tasks.workunit.client.1.smithi148.stdout:7/928: dwrite d5/d8/d6f/d7d/f98 [4194304,4194304] 0 2024-09-21T04:15:26.338 INFO:tasks.workunit.client.1.smithi148.stdout:7/929: stat d5/d14/d25/d140 0 2024-09-21T04:15:26.338 INFO:tasks.workunit.client.1.smithi148.stdout:7/930: read - d5/d8/d6f/d7d/d90/f11d zero size 2024-09-21T04:15:26.344 INFO:tasks.workunit.client.1.smithi148.stdout:0/960: write d5/d82/f87 [1100676,96736] 0 2024-09-21T04:15:26.344 INFO:tasks.workunit.client.1.smithi148.stdout:0/961: truncate d5/dcb/de6/d110/d2a/dcd/df8/db4/fe1 1766468 0 2024-09-21T04:15:26.346 INFO:tasks.workunit.client.1.smithi148.stdout:0/962: mknod d5/dcb/de6/d110/d7/dd0/c131 0 2024-09-21T04:15:26.348 INFO:tasks.workunit.client.1.smithi148.stdout:0/963: link d5/dcb/de6/d110/d7/cd d5/dcb/de6/d110/d2a/dcd/df8/db4/c132 0 2024-09-21T04:15:26.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/964: rename d5/dcb/de6/d110/d2a/d45/fe5 to d5/dcb/de6/d110/d2a/d3f/d6d/da3/dcc/f133 0 2024-09-21T04:15:26.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/965: write d5/dcb/de6/d110/d2a/d3f/d6d/da3/dcc/f133 [664027,7468] 0 2024-09-21T04:15:26.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/966: write d5/dcb/de6/d110/d2a/d2b/d99/db7/fc7 [1717467,91624] 0 2024-09-21T04:15:26.352 INFO:tasks.workunit.client.1.smithi148.stdout:0/967: rename d5/dcb/de6/d110/d2a/d3f/d8b/dab/fc9 to d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/f134 0 2024-09-21T04:15:26.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/931: dwrite d5/d8/d6f/d7d/d90/d28/fda [0,4194304] 0 2024-09-21T04:15:26.366 INFO:tasks.workunit.client.1.smithi148.stdout:7/932: dread d5/d14/d25/d40/db9/fbd [0,4194304] 0 2024-09-21T04:15:26.366 INFO:tasks.workunit.client.1.smithi148.stdout:0/968: dwrite d5/dcb/de6/d110/d2a/f125 [0,4194304] 0 2024-09-21T04:15:26.370 INFO:tasks.workunit.client.1.smithi148.stdout:0/969: truncate d5/dcb/de6/d110/d2a/dcd/df8/db4/fb5 742895 0 2024-09-21T04:15:26.371 INFO:tasks.workunit.client.1.smithi148.stdout:7/933: rename d5/d8/c44 to d5/d14/d25/d40/dab/dad/db8/d125/c14c 0 2024-09-21T04:15:26.372 INFO:tasks.workunit.client.1.smithi148.stdout:7/934: mkdir d5/d8/d6f/d7d/df5/d149/d14d 0 2024-09-21T04:15:26.373 INFO:tasks.workunit.client.1.smithi148.stdout:7/935: mkdir d5/d8/d6f/dcd/d132/d14e 0 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:0/970: dread d5/dcb/de6/d110/d7/d22/fa2 [0,4194304] 0 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:6/994: sync 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:9/904: sync 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:4/944: sync 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:5/898: sync 2024-09-21T04:15:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/930: sync 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/828: sync 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/995: dread - d0/d34/f100 zero size 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:5/899: dread - d3/d34/d78/f107 zero size 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/996: chown d0/d34/d97/d106/d23/fdf 2952 1 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/997: write d0/d34/d97/d106/d23/f14 [1712774,72181] 0 2024-09-21T04:15:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/998: creat d0/d8/dba/dbc/f139 x:0 0 0 2024-09-21T04:15:26.387 INFO:tasks.workunit.client.1.smithi148.stdout:9/905: dread d5/d1f/d81/dcf/d106/d80/d104/f99 [0,4194304] 0 2024-09-21T04:15:26.396 INFO:tasks.workunit.client.1.smithi148.stdout:9/906: dread d5/d2b/d4e/fa3 [0,4194304] 0 2024-09-21T04:15:26.399 INFO:tasks.workunit.client.1.smithi148.stdout:4/945: dwrite d4/d87/d2d/f117 [0,4194304] 0 2024-09-21T04:15:26.399 INFO:tasks.workunit.client.1.smithi148.stdout:4/946: fdatasync d4/d66/d7f/d9a/dce/ffe 0 2024-09-21T04:15:26.399 INFO:tasks.workunit.client.1.smithi148.stdout:4/947: fsync d4/d12/fed 0 2024-09-21T04:15:26.399 INFO:tasks.workunit.client.1.smithi148.stdout:4/948: fdatasync d4/d87/f32 0 2024-09-21T04:15:26.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/931: dwrite db/d24/d2e/d71/d94/ff3 [0,4194304] 0 2024-09-21T04:15:26.400 INFO:tasks.workunit.client.1.smithi148.stdout:1/932: creat db/d24/d2e/d71/d12e/f134 x:0 0 0 2024-09-21T04:15:26.407 INFO:tasks.workunit.client.1.smithi148.stdout:1/933: dread db/d24/d2e/f9c [0,4194304] 0 2024-09-21T04:15:26.407 INFO:tasks.workunit.client.1.smithi148.stdout:1/934: fsync db/d24/f26 0 2024-09-21T04:15:26.412 INFO:tasks.workunit.client.1.smithi148.stdout:9/907: dwrite d5/ff [0,4194304] 0 2024-09-21T04:15:26.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/971: symlink d5/dcb/de6/d110/d7/dd0/l135 0 2024-09-21T04:15:26.425 INFO:tasks.workunit.client.1.smithi148.stdout:0/972: readlink d5/dcb/de6/d110/d7/dd0/lef 0 2024-09-21T04:15:26.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/900: creat d3/d34/d3c/d5d/d75/d121/f131 x:0 0 0 2024-09-21T04:15:26.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/901: write d3/d4a/fae [4283428,123067] 0 2024-09-21T04:15:26.428 INFO:tasks.workunit.client.1.smithi148.stdout:8/829: mknod dc/d13/d3c/d81/d98/de5/c102 0 2024-09-21T04:15:26.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/999: symlink d0/d34/d97/d106/d23/d32/d93/d96/l13a 0 2024-09-21T04:15:26.434 INFO:tasks.workunit.client.1.smithi148.stdout:4/949: symlink d4/d12/d16/d29/d61/d6c/df1/l128 0 2024-09-21T04:15:26.440 INFO:tasks.workunit.client.1.smithi148.stdout:1/935: rmdir db/d24/d2e/d59/da7/dc1 39 2024-09-21T04:15:26.440 INFO:tasks.workunit.client.1.smithi148.stdout:1/936: readlink db/l49 0 2024-09-21T04:15:26.441 INFO:tasks.workunit.client.1.smithi148.stdout:1/937: read db/f75 [820406,86873] 0 2024-09-21T04:15:26.442 INFO:tasks.workunit.client.1.smithi148.stdout:9/908: unlink d5/d60/d4c/l9d 0 2024-09-21T04:15:26.442 INFO:tasks.workunit.client.1.smithi148.stdout:9/909: write d5/d2b/dce/df6/f129 [57361,9867] 0 2024-09-21T04:15:26.448 INFO:tasks.workunit.client.1.smithi148.stdout:7/936: mkdir d5/d14/d25/d14f 0 2024-09-21T04:15:26.449 INFO:tasks.workunit.client.1.smithi148.stdout:0/973: truncate d5/dcb/de6/d110/d7/d22/fa2 1038915 0 2024-09-21T04:15:26.449 INFO:tasks.workunit.client.1.smithi148.stdout:0/974: readlink d5/dcb/de6/d110/d2a/d3f/d6d/l72 0 2024-09-21T04:15:26.453 INFO:tasks.workunit.client.1.smithi148.stdout:9/910: dread d5/d2b/dbe/f9a [0,4194304] 0 2024-09-21T04:15:26.453 INFO:tasks.workunit.client.1.smithi148.stdout:9/911: dread - d5/d1e/faa zero size 2024-09-21T04:15:26.454 INFO:tasks.workunit.client.1.smithi148.stdout:9/912: truncate d5/d39/f98 149558 0 2024-09-21T04:15:26.454 INFO:tasks.workunit.client.1.smithi148.stdout:5/902: link d3/d34/d78/f107 d3/d14/d1f/d26/d59/d97/f132 0 2024-09-21T04:15:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/830: link dc/d43/d4c/d4f/l59 dc/d13/d28/de4/l103 0 2024-09-21T04:15:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/831: chown dc/d13/d3c/d81/d85/da1/fb8 763075825 1 2024-09-21T04:15:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/832: write dc/d13/d76/d50/df0/ffe [48533,120443] 0 2024-09-21T04:15:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/833: fsync dc/d13/d39/fc6 0 2024-09-21T04:15:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:8/834: read dc/d13/d6a/f94 [650547,105640] 0 2024-09-21T04:15:26.458 INFO:tasks.workunit.client.1.smithi148.stdout:4/950: link d4/d12/d16/d29/d61/d6c/f88 d4/d12/d83/d103/f129 0 2024-09-21T04:15:26.459 INFO:tasks.workunit.client.1.smithi148.stdout:1/938: mkdir db/dd/d21/d9b/d135 0 2024-09-21T04:15:26.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/937: dwrite d5/d8/d6f/d7d/d90/fce [0,4194304] 0 2024-09-21T04:15:26.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/938: truncate d5/d8/d6f/d7d/d8a/fa1 1540255 0 2024-09-21T04:15:26.464 INFO:tasks.workunit.client.1.smithi148.stdout:0/975: symlink d5/dcb/de6/d110/d2a/dcd/df8/l136 0 2024-09-21T04:15:26.465 INFO:tasks.workunit.client.1.smithi148.stdout:0/976: dread d5/d82/fc4 [0,4194304] 0 2024-09-21T04:15:26.465 INFO:tasks.workunit.client.1.smithi148.stdout:0/977: chown d5/dcb/de6/d110/d2a/d3f/d6d/l72 123 1 2024-09-21T04:15:26.465 INFO:tasks.workunit.client.1.smithi148.stdout:0/978: chown d5/dcb/de6/d110/d7/c1c 161 1 2024-09-21T04:15:26.465 INFO:tasks.workunit.client.1.smithi148.stdout:0/979: creat d5/d82/f137 x:0 0 0 2024-09-21T04:15:26.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/913: mkdir d5/d13f 0 2024-09-21T04:15:26.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/894: sync 2024-09-21T04:15:26.467 INFO:tasks.workunit.client.1.smithi148.stdout:3/895: readlink d3/d5/d6/d2b/d96/l9e 0 2024-09-21T04:15:26.467 INFO:tasks.workunit.client.1.smithi148.stdout:3/896: rename d3 to d3/d1f/d4d/d82/d23/db5/d132 22 2024-09-21T04:15:26.467 INFO:tasks.workunit.client.1.smithi148.stdout:3/897: rename d3 to d3/d5/d6/d12/d43/d133 22 2024-09-21T04:15:26.467 INFO:tasks.workunit.client.1.smithi148.stdout:3/898: getdents d3/d5/d6/d109 0 2024-09-21T04:15:26.470 INFO:tasks.workunit.client.1.smithi148.stdout:8/835: mkdir dc/d13/d3c/d81/d85/d104 0 2024-09-21T04:15:26.470 INFO:tasks.workunit.client.1.smithi148.stdout:8/836: creat dc/d43/d4c/f105 x:0 0 0 2024-09-21T04:15:26.472 INFO:tasks.workunit.client.1.smithi148.stdout:1/939: unlink db/c109 0 2024-09-21T04:15:26.475 INFO:tasks.workunit.client.1.smithi148.stdout:0/980: read d5/dcb/de6/d110/d2a/dcd/df8/daf/f90 [1095979,128404] 0 2024-09-21T04:15:26.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/903: getdents d3/d14/d1f/d26/d2e/ddb 0 2024-09-21T04:15:26.479 INFO:tasks.workunit.client.1.smithi148.stdout:3/899: mknod d3/d1f/d4d/d82/d23/daf/c134 0 2024-09-21T04:15:26.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/837: write dc/d13/f9b [1566636,25289] 0 2024-09-21T04:15:26.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/940: symlink db/d24/d2e/d59/da7/l136 0 2024-09-21T04:15:26.481 INFO:tasks.workunit.client.1.smithi148.stdout:4/951: dwrite d4/d87/f32 [0,4194304] 0 2024-09-21T04:15:26.482 INFO:tasks.workunit.client.1.smithi148.stdout:5/904: mkdir d3/d4a/d11c/d133 0 2024-09-21T04:15:26.484 INFO:tasks.workunit.client.1.smithi148.stdout:1/941: dread db/f64 [0,4194304] 0 2024-09-21T04:15:26.484 INFO:tasks.workunit.client.1.smithi148.stdout:8/838: mknod dc/d13/d28/d62/c106 0 2024-09-21T04:15:26.485 INFO:tasks.workunit.client.1.smithi148.stdout:8/839: write dc/f68 [1962416,83346] 0 2024-09-21T04:15:26.485 INFO:tasks.workunit.client.1.smithi148.stdout:8/840: chown dc/d13/d76/d50/d80/ff6 168559 1 2024-09-21T04:15:26.485 INFO:tasks.workunit.client.1.smithi148.stdout:8/841: write dc/d13/d3c/d81/d9d/f95 [725537,116463] 0 2024-09-21T04:15:26.485 INFO:tasks.workunit.client.1.smithi148.stdout:4/952: creat d4/d3a/f12a x:0 0 0 2024-09-21T04:15:26.486 INFO:tasks.workunit.client.1.smithi148.stdout:7/939: dwrite d5/daa/fd0 [4194304,4194304] 0 2024-09-21T04:15:26.487 INFO:tasks.workunit.client.1.smithi148.stdout:8/842: mkdir dc/d13/d39/d107 0 2024-09-21T04:15:26.487 INFO:tasks.workunit.client.1.smithi148.stdout:8/843: fsync dc/fa8 0 2024-09-21T04:15:26.488 INFO:tasks.workunit.client.1.smithi148.stdout:3/900: write d3/d1f/d4d/d82/d23/d28/f42 [2161592,52461] 0 2024-09-21T04:15:26.490 INFO:tasks.workunit.client.1.smithi148.stdout:7/940: unlink d5/d8/d6f/d7d/d90/d9b/cbf 0 2024-09-21T04:15:26.492 INFO:tasks.workunit.client.1.smithi148.stdout:8/844: mkdir dc/d13/d39/d6e/d108 0 2024-09-21T04:15:26.492 INFO:tasks.workunit.client.1.smithi148.stdout:3/901: dread d3/d1f/d4d/d82/f11 [0,4194304] 0 2024-09-21T04:15:26.493 INFO:tasks.workunit.client.0.smithi110.stdout:9/721: sync 2024-09-21T04:15:26.493 INFO:tasks.workunit.client.0.smithi110.stdout:9/722: creat d7/d96/d99/d3c/d1f/d43/db5/ff7 x:0 0 0 2024-09-21T04:15:26.493 INFO:tasks.workunit.client.0.smithi110.stdout:9/723: dread - d7/d96/d99/d3c/ff4 zero size 2024-09-21T04:15:26.493 INFO:tasks.workunit.client.0.smithi110.stdout:9/724: fdatasync d7/d96/d99/d3c/d4f/fa1 0 2024-09-21T04:15:26.494 INFO:tasks.workunit.client.1.smithi148.stdout:9/914: dwrite d5/f42 [0,4194304] 0 2024-09-21T04:15:26.494 INFO:tasks.workunit.client.1.smithi148.stdout:9/915: write d5/d39/f6f [1662825,118418] 0 2024-09-21T04:15:26.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/845: symlink dc/d43/l109 0 2024-09-21T04:15:26.495 INFO:tasks.workunit.client.1.smithi148.stdout:7/941: mkdir d5/d8/d150 0 2024-09-21T04:15:26.495 INFO:tasks.workunit.client.1.smithi148.stdout:7/942: dread - d5/d14/ff2 zero size 2024-09-21T04:15:26.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/942: dread db/d24/f70 [4194304,4194304] 0 2024-09-21T04:15:26.496 INFO:tasks.workunit.client.1.smithi148.stdout:1/943: stat db/d24/f25 0 2024-09-21T04:15:26.498 INFO:tasks.workunit.client.1.smithi148.stdout:3/902: unlink d3/d1f/d4d/d82/d23/db5/fd0 0 2024-09-21T04:15:26.501 INFO:tasks.workunit.client.1.smithi148.stdout:9/916: getdents d5/d60/d69 0 2024-09-21T04:15:26.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/944: link db/d46/d76/fe3 db/f137 0 2024-09-21T04:15:26.507 INFO:tasks.workunit.client.1.smithi148.stdout:5/905: dwrite d3/d14/d1f/d26/d2e/ddb/ffe [0,4194304] 0 2024-09-21T04:15:26.507 INFO:tasks.workunit.client.1.smithi148.stdout:5/906: stat d3/d14/d1f/d26/d2e/d3d/f60 0 2024-09-21T04:15:26.510 INFO:tasks.workunit.client.1.smithi148.stdout:5/907: creat d3/d14/d1f/d26/d2e/d3d/da0/f134 x:0 0 0 2024-09-21T04:15:26.510 INFO:tasks.workunit.client.1.smithi148.stdout:5/908: stat d3/d34/d78/dbf/f96 0 2024-09-21T04:15:26.511 INFO:tasks.workunit.client.1.smithi148.stdout:5/909: mknod d3/d14/d1f/d26/d2e/d56/c135 0 2024-09-21T04:15:26.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/917: dread d5/d2b/dce/df6/f13d [0,4194304] 0 2024-09-21T04:15:26.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/918: fdatasync d5/d60/f13b 0 2024-09-21T04:15:26.516 INFO:tasks.workunit.client.1.smithi148.stdout:9/919: creat d5/d1e/f140 x:0 0 0 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/903: dread d3/d5/d5c/fc4 [0,4194304] 0 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:3/904: chown d3/d5/d6/d12/d15/c102 7969391 1 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:1/945: dread db/d46/f8d [0,4194304] 0 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/920: link d5/d60/d28/d2f/l5f d5/d60/de1/l141 0 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:1/946: write db/d24/d2e/d59/da7/fb4 [427415,67808] 0 2024-09-21T04:15:26.517 INFO:tasks.workunit.client.1.smithi148.stdout:9/921: chown d5/d60/d28/f6a 375495 1 2024-09-21T04:15:26.518 INFO:tasks.workunit.client.1.smithi148.stdout:9/922: write d5/d60/d4c/d5e/fb4 [376076,79377] 0 2024-09-21T04:15:26.518 INFO:tasks.workunit.client.1.smithi148.stdout:3/905: mknod d3/d5/d6/d12/d43/d54/d9b/d131/c135 0 2024-09-21T04:15:26.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/923: creat d5/d60/d28/d2f/dbf/f142 x:0 0 0 2024-09-21T04:15:26.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/924: truncate d5/f117 809400 0 2024-09-21T04:15:26.519 INFO:tasks.workunit.client.1.smithi148.stdout:9/925: stat d5/d2b/dce 0 2024-09-21T04:15:26.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/906: unlink d3/d1f/d4d/d82/d23/da1/cc7 0 2024-09-21T04:15:26.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/926: symlink d5/d2b/d4e/db3/ded/l143 0 2024-09-21T04:15:26.522 INFO:tasks.workunit.client.1.smithi148.stdout:9/927: creat d5/d2b/d4e/db3/ded/f144 x:0 0 0 2024-09-21T04:15:26.522 INFO:tasks.workunit.client.1.smithi148.stdout:3/907: creat d3/d5/db4/dfb/f136 x:0 0 0 2024-09-21T04:15:26.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/908: fdatasync d3/d1f/d4d/f70 0 2024-09-21T04:15:26.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/909: write d3/d1f/d4d/d82/d23/d28/d46/f12f [4454105,122643] 0 2024-09-21T04:15:26.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/928: rmdir d5/d60/d28/d2f/d136 0 2024-09-21T04:15:26.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/929: chown d5/d1e/d37/d46/f6d 0 1 2024-09-21T04:15:26.524 INFO:tasks.workunit.client.1.smithi148.stdout:9/930: write d5/d60/d28/d73/f102 [27176,21609] 0 2024-09-21T04:15:26.524 INFO:tasks.workunit.client.1.smithi148.stdout:3/910: mkdir d3/d5/d6/d2b/d137 0 2024-09-21T04:15:26.525 INFO:tasks.workunit.client.1.smithi148.stdout:9/931: getdents d5/d60/d28/d2f/dbf/dc2 0 2024-09-21T04:15:26.525 INFO:tasks.workunit.client.1.smithi148.stdout:9/932: chown d5/d1e/l2a 60648 1 2024-09-21T04:15:26.526 INFO:tasks.workunit.client.1.smithi148.stdout:3/911: mknod d3/d5/d6/d2b/da2/da3/de4/d117/c138 0 2024-09-21T04:15:26.526 INFO:tasks.workunit.client.1.smithi148.stdout:9/933: mknod d5/d1f/d81/c145 0 2024-09-21T04:15:26.527 INFO:tasks.workunit.client.1.smithi148.stdout:1/947: write db/d24/d2a/dbd/fe8 [1648091,80333] 0 2024-09-21T04:15:26.527 INFO:tasks.workunit.client.1.smithi148.stdout:3/912: symlink d3/d5/d6/d12/d15/d73/l139 0 2024-09-21T04:15:26.527 INFO:tasks.workunit.client.1.smithi148.stdout:9/934: symlink d5/d2b/l146 0 2024-09-21T04:15:26.528 INFO:tasks.workunit.client.1.smithi148.stdout:1/948: mknod db/dd/d21/c138 0 2024-09-21T04:15:26.528 INFO:tasks.workunit.client.1.smithi148.stdout:9/935: mknod d5/d2b/dce/c147 0 2024-09-21T04:15:26.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/936: truncate d5/d1e/f65 2358272 0 2024-09-21T04:15:26.530 INFO:tasks.workunit.client.1.smithi148.stdout:1/949: dread db/d24/d2e/d71/d94/d9a/f7d [0,4194304] 0 2024-09-21T04:15:26.531 INFO:tasks.workunit.client.1.smithi148.stdout:9/937: creat d5/d60/d28/f148 x:0 0 0 2024-09-21T04:15:26.531 INFO:tasks.workunit.client.1.smithi148.stdout:9/938: chown d5/d1e/d37/d46/c13a 11 1 2024-09-21T04:15:26.531 INFO:tasks.workunit.client.1.smithi148.stdout:9/939: chown d5/d2b/d4e/db3/ded/c107 1019461 1 2024-09-21T04:15:26.532 INFO:tasks.workunit.client.1.smithi148.stdout:1/950: creat db/d24/d2a/da4/f139 x:0 0 0 2024-09-21T04:15:26.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/846: dwrite dc/d13/d3c/d81/d85/fcc [0,4194304] 0 2024-09-21T04:15:26.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/725: dwrite d7/d96/d99/d3c/d1f/d43/fad [0,4194304] 0 2024-09-21T04:15:26.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/726: fdatasync f2 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:4/953: dwrite d4/d12/d16/d29/d61/fe2 [4194304,4194304] 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:5/910: dwrite d3/f1b [0,4194304] 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:1/951: rmdir db/d24/d2e/d71/d94/dde/de1 39 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:1/952: getdents db/d46/da0/d11e 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:4/954: write d4/d12/f63 [1079000,90962] 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:5/911: write d3/d34/f11d [475030,112870] 0 2024-09-21T04:15:26.535 INFO:tasks.workunit.client.1.smithi148.stdout:4/955: chown d4/d66/d7f/d2a/d118 10761735 1 2024-09-21T04:15:26.537 INFO:tasks.workunit.client.1.smithi148.stdout:5/912: mkdir d3/d34/d78/dbf/d136 0 2024-09-21T04:15:26.538 INFO:tasks.workunit.client.1.smithi148.stdout:5/913: fsync d3/d14/d1f/d72/dc9/f119 0 2024-09-21T04:15:26.540 INFO:tasks.workunit.client.1.smithi148.stdout:4/956: symlink d4/d12/d78/de4/d10c/l12b 0 2024-09-21T04:15:26.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/953: dread db/d24/d2a/f45 [0,4194304] 0 2024-09-21T04:15:26.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/954: fdatasync db/d24/f70 0 2024-09-21T04:15:26.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/955: stat db/d24/d2e/d59/l5d 0 2024-09-21T04:15:26.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/956: chown db/d24/d2e/d71/d94/dde/de1 13700 1 2024-09-21T04:15:26.545 INFO:tasks.workunit.client.1.smithi148.stdout:8/847: dread dc/d13/f35 [0,4194304] 0 2024-09-21T04:15:26.548 INFO:tasks.workunit.client.1.smithi148.stdout:1/957: link db/dd/l68 db/d24/d2a/d112/l13a 0 2024-09-21T04:15:26.548 INFO:tasks.workunit.client.0.smithi110.stdout:9/727: read d7/d96/d99/d3c/d4f/d9f/d6f/d78/fb9 [316838,111083] 0 2024-09-21T04:15:26.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/957: write d4/d12/d16/d29/d61/d7e/fa0 [2038460,78418] 0 2024-09-21T04:15:26.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/940: dwrite d5/d1e/f29 [0,4194304] 0 2024-09-21T04:15:26.550 INFO:tasks.workunit.client.1.smithi148.stdout:1/958: unlink db/d46/d4c/f86 0 2024-09-21T04:15:26.550 INFO:tasks.workunit.client.1.smithi148.stdout:8/848: getdents dc/d13/d76/dd8 0 2024-09-21T04:15:26.550 INFO:tasks.workunit.client.1.smithi148.stdout:8/849: readlink dc/d13/d76/d5d/la3 0 2024-09-21T04:15:26.550 INFO:tasks.workunit.client.1.smithi148.stdout:8/850: truncate dc/d13/d76/d50/df0/ffe 876913 0 2024-09-21T04:15:26.551 INFO:tasks.workunit.client.1.smithi148.stdout:8/851: fsync dc/fa8 0 2024-09-21T04:15:26.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/958: mkdir d4/d12/d83/dcc/d101/d12c 0 2024-09-21T04:15:26.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/959: creat d4/d12/d16/df0/dec/f12d x:0 0 0 2024-09-21T04:15:26.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/960: write d4/d12/d16/d29/d61/d7e/fa5 [2709149,39503] 0 2024-09-21T04:15:26.555 INFO:tasks.workunit.client.1.smithi148.stdout:9/941: mknod d5/d1f/d81/dcf/d106/df8/c149 0 2024-09-21T04:15:26.555 INFO:tasks.workunit.client.1.smithi148.stdout:9/942: chown d5/d1e/c33 153 1 2024-09-21T04:15:26.558 INFO:tasks.workunit.client.1.smithi148.stdout:1/959: unlink db/dd/d21/db0/le9 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/960: write db/dd/d21/db0/f120 [103248,828] 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/961: creat db/d24/d2a/da4/f13b x:0 0 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/962: read db/d24/d2e/daf/fb1 [776347,66439] 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:1/963: creat db/d24/d2e/d71/d94/dde/de1/d129/f13c x:0 0 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/913: dwrite d3/d5/db4/dfb/f136 [0,4194304] 0 2024-09-21T04:15:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/914: write d3/d5/d6/d12/d43/d54/fc6 [875068,45002] 0 2024-09-21T04:15:26.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/943: creat d5/d1f/d81/d9e/da2/f14a x:0 0 0 2024-09-21T04:15:26.564 INFO:tasks.workunit.client.1.smithi148.stdout:8/852: dread dc/f14 [0,4194304] 0 2024-09-21T04:15:26.565 INFO:tasks.workunit.client.1.smithi148.stdout:5/914: dwrite d3/d14/d1f/f80 [0,4194304] 0 2024-09-21T04:15:26.566 INFO:tasks.workunit.client.1.smithi148.stdout:9/944: rename d5/d60/d4c/d5e/d134/f11c to d5/d2b/d4e/f14b 0 2024-09-21T04:15:26.569 INFO:tasks.workunit.client.1.smithi148.stdout:8/853: unlink dc/d13/d76/f2e 0 2024-09-21T04:15:26.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/945: truncate d5/d1e/f49 3994417 0 2024-09-21T04:15:26.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/946: truncate d5/d1e/d37/d46/f72 3822419 0 2024-09-21T04:15:26.574 INFO:tasks.workunit.client.1.smithi148.stdout:9/947: creat d5/d2b/d4e/db3/ded/f14c x:0 0 0 2024-09-21T04:15:26.574 INFO:tasks.workunit.client.0.smithi110.stdout:9/728: dwrite d7/d96/d99/d3c/d1f/d43/f60 [4194304,4194304] 0 2024-09-21T04:15:26.578 INFO:tasks.workunit.client.1.smithi148.stdout:9/948: read d5/d60/f3c [862209,68510] 0 2024-09-21T04:15:26.580 INFO:tasks.workunit.client.0.smithi110.stdout:9/729: mknod d7/d96/d99/d3c/d1f/d43/db5/dbf/cf8 0 2024-09-21T04:15:26.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/961: dwrite d4/d87/f89 [0,4194304] 0 2024-09-21T04:15:26.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/949: mkdir d5/d60/d4c/d14d 0 2024-09-21T04:15:26.583 INFO:tasks.workunit.client.1.smithi148.stdout:9/950: chown d5/de3 685508 1 2024-09-21T04:15:26.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/730: creat d7/d96/d99/ff9 x:0 0 0 2024-09-21T04:15:26.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/731: chown d7/l4a 14 1 2024-09-21T04:15:26.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/732: write d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a [3907657,54072] 0 2024-09-21T04:15:26.586 INFO:tasks.workunit.client.1.smithi148.stdout:4/962: rename d4/d87/dc1 to d4/d66/d7f/d9a/dc2/d12e 0 2024-09-21T04:15:26.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/951: rename d5/d2b/d4e/db3/ded/ff7 to d5/d60/d28/d73/d108/f14e 0 2024-09-21T04:15:26.591 INFO:tasks.workunit.client.1.smithi148.stdout:9/952: link d5/d60/d28/d2f/dbf/lda d5/d60/d69/l14f 0 2024-09-21T04:15:26.592 INFO:tasks.workunit.client.1.smithi148.stdout:9/953: chown d5/d1e/d37/l7c 34720355 1 2024-09-21T04:15:26.592 INFO:tasks.workunit.client.1.smithi148.stdout:9/954: write d5/f1b [995372,79425] 0 2024-09-21T04:15:26.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/964: dwrite db/d24/d2e/d59/f12b [0,4194304] 0 2024-09-21T04:15:26.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/965: chown db/dd/ddd/ce5 3286 1 2024-09-21T04:15:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:2/686: sync 2024-09-21T04:15:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:3/709: sync 2024-09-21T04:15:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/809: sync 2024-09-21T04:15:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:7/749: sync 2024-09-21T04:15:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:4/951: sync 2024-09-21T04:15:26.601 INFO:tasks.workunit.client.0.smithi110.stdout:5/852: sync 2024-09-21T04:15:26.601 INFO:tasks.workunit.client.0.smithi110.stdout:5/853: chown d1/d7/f4e 158983500 1 2024-09-21T04:15:26.603 INFO:tasks.workunit.client.0.smithi110.stdout:3/710: mknod d0/d12/d16/d1a/cf7 0 2024-09-21T04:15:26.603 INFO:tasks.workunit.client.0.smithi110.stdout:3/711: truncate d0/d12/fd5 483612 0 2024-09-21T04:15:26.604 INFO:tasks.workunit.client.0.smithi110.stdout:6/810: mkdir d4/d12/d17/d81/d106 0 2024-09-21T04:15:26.606 INFO:tasks.workunit.client.1.smithi148.stdout:4/963: rename d4/d12/d83 to d4/d66/d7f/d9a/dce/d12f 0 2024-09-21T04:15:26.607 INFO:tasks.workunit.client.0.smithi110.stdout:7/750: write dd/d14/f2a [7044743,126358] 0 2024-09-21T04:15:26.609 INFO:tasks.workunit.client.0.smithi110.stdout:4/952: creat d17/d1e/d7b/f141 x:0 0 0 2024-09-21T04:15:26.611 INFO:tasks.workunit.client.0.smithi110.stdout:5/854: mkdir d1/d49/d4d/d1e/d4a/dd4/d129 0 2024-09-21T04:15:26.612 INFO:tasks.workunit.client.0.smithi110.stdout:2/687: rename d2/f3b to d2/d1a/d1f/d29/d19/fda 0 2024-09-21T04:15:26.613 INFO:tasks.workunit.client.0.smithi110.stdout:3/712: symlink d0/d12/d16/d1a/dc6/lf8 0 2024-09-21T04:15:26.615 INFO:tasks.workunit.client.0.smithi110.stdout:4/953: symlink d17/d1e/d77/d10c/d20/d43/d51/d7a/l142 0 2024-09-21T04:15:26.616 INFO:tasks.workunit.client.0.smithi110.stdout:2/688: creat d2/d1a/d1f/d29/d19/fdb x:0 0 0 2024-09-21T04:15:26.619 INFO:tasks.workunit.client.0.smithi110.stdout:3/713: truncate d0/d12/d16/d1a/d95/fc9 68579 0 2024-09-21T04:15:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:4/964: write d4/fb [1540699,64370] 0 2024-09-21T04:15:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/915: dwrite d3/d24/ffd [0,4194304] 0 2024-09-21T04:15:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/854: dwrite dc/d13/d76/dd8/ff5 [0,4194304] 0 2024-09-21T04:15:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:3/915: dwrite d3/d5/d6/ff3 [0,4194304] 0 2024-09-21T04:15:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:3/916: fsync d3/d1f/d4d/d82/d23/db5/d105/f122 0 2024-09-21T04:15:26.621 INFO:tasks.workunit.client.0.smithi110.stdout:7/751: dread dd/d1c/dad/db2/f3c [0,4194304] 0 2024-09-21T04:15:26.622 INFO:tasks.workunit.client.0.smithi110.stdout:5/855: write d1/d22/ffa [499071,9309] 0 2024-09-21T04:15:26.624 INFO:tasks.workunit.client.0.smithi110.stdout:4/954: rename d17/d1e/d77/d10c/d20/d43/d51/d52/lb7 to d17/d1e/d21/l143 0 2024-09-21T04:15:26.624 INFO:tasks.workunit.client.0.smithi110.stdout:4/955: chown d17/d1e/d77/ca9 1951089754 1 2024-09-21T04:15:26.627 INFO:tasks.workunit.client.0.smithi110.stdout:5/856: unlink d1/fce 0 2024-09-21T04:15:26.627 INFO:tasks.workunit.client.0.smithi110.stdout:3/714: dread d0/d22/d29/fc5 [0,4194304] 0 2024-09-21T04:15:26.632 INFO:tasks.workunit.client.0.smithi110.stdout:2/689: dread d2/d1a/d1f/d65/f80 [0,4194304] 0 2024-09-21T04:15:26.632 INFO:tasks.workunit.client.0.smithi110.stdout:5/857: rename d1/d7/l8c to d1/d22/db6/l12a 0 2024-09-21T04:15:26.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/834: sync 2024-09-21T04:15:26.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/835: creat d0/d30/d34/f12f x:0 0 0 2024-09-21T04:15:26.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/836: chown d0/d30/d54/c10a 1289753 1 2024-09-21T04:15:26.635 INFO:tasks.workunit.client.0.smithi110.stdout:2/690: link d2/d1a/d1f/d5f/l55 d2/d50/d9c/ldc 0 2024-09-21T04:15:26.637 INFO:tasks.workunit.client.1.smithi148.stdout:5/916: symlink d3/d34/l137 0 2024-09-21T04:15:26.639 INFO:tasks.workunit.client.1.smithi148.stdout:8/855: symlink dc/d13/d3c/d81/d85/d104/l10a 0 2024-09-21T04:15:26.639 INFO:tasks.workunit.client.1.smithi148.stdout:8/856: chown dc/d43 6937143 1 2024-09-21T04:15:26.639 INFO:tasks.workunit.client.1.smithi148.stdout:8/857: chown dc/d13/d3c/d81/db0/lc9 13342 1 2024-09-21T04:15:26.639 INFO:tasks.workunit.client.1.smithi148.stdout:8/858: creat dc/f10b x:0 0 0 2024-09-21T04:15:26.640 INFO:tasks.workunit.client.0.smithi110.stdout:2/691: rename d2/d50/d56/lc2 to d2/d1a/d1f/d5f/d9d/ldd 0 2024-09-21T04:15:26.641 INFO:tasks.workunit.client.0.smithi110.stdout:2/692: mknod d2/d1a/da3/cde 0 2024-09-21T04:15:26.644 INFO:tasks.workunit.client.0.smithi110.stdout:2/693: link d2/d1a/d1f/d29/f7d d2/d1a/d8b/da5/fdf 0 2024-09-21T04:15:26.646 INFO:tasks.workunit.client.0.smithi110.stdout:9/733: dwrite d7/d96/d99/d3c/fdf [0,4194304] 0 2024-09-21T04:15:26.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/955: dwrite d5/d60/f3c [0,4194304] 0 2024-09-21T04:15:26.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/956: write d5/d2b/d4e/fa3 [4839880,92688] 0 2024-09-21T04:15:26.646 INFO:tasks.workunit.client.1.smithi148.stdout:9/957: write d5/d60/d28/d73/fdf [1812864,111137] 0 2024-09-21T04:15:26.647 INFO:tasks.workunit.client.1.smithi148.stdout:9/958: dread - d5/d2b/d4e/f14b zero size 2024-09-21T04:15:26.647 INFO:tasks.workunit.client.0.smithi110.stdout:9/734: chown d7/d96/d99/d3c/d14/d31/c98 0 1 2024-09-21T04:15:26.647 INFO:tasks.workunit.client.1.smithi148.stdout:8/859: rename dc/d43/c56 to dc/d13/d76/d50/df3/c10c 0 2024-09-21T04:15:26.648 INFO:tasks.workunit.client.0.smithi110.stdout:2/694: creat d2/d1a/d1f/d2a/fe0 x:0 0 0 2024-09-21T04:15:26.650 INFO:tasks.workunit.client.1.smithi148.stdout:5/917: dread d3/d34/d78/dbf/d6f/fc8 [0,4194304] 0 2024-09-21T04:15:26.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/918: fdatasync d3/d14/f2d 0 2024-09-21T04:15:26.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/919: fdatasync d3/d14/d1f/d26/d2e/ddb/ffe 0 2024-09-21T04:15:26.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/920: dread - d3/d14/d1f/f123 zero size 2024-09-21T04:15:26.656 INFO:tasks.workunit.client.0.smithi110.stdout:9/735: mknod d7/d96/d99/d3c/d11/cfa 0 2024-09-21T04:15:26.656 INFO:tasks.workunit.client.0.smithi110.stdout:2/695: write d2/f44 [3676437,60741] 0 2024-09-21T04:15:26.657 INFO:tasks.workunit.client.1.smithi148.stdout:9/959: creat d5/d2b/d32/d103/d123/f150 x:0 0 0 2024-09-21T04:15:26.657 INFO:tasks.workunit.client.1.smithi148.stdout:9/960: creat d5/d60/d28/d2f/dbf/f151 x:0 0 0 2024-09-21T04:15:26.658 INFO:tasks.workunit.client.1.smithi148.stdout:8/860: creat dc/d13/d3c/d81/f10d x:0 0 0 2024-09-21T04:15:26.658 INFO:tasks.workunit.client.1.smithi148.stdout:8/861: getdents dc/d13/d76/d50/dd0 0 2024-09-21T04:15:26.658 INFO:tasks.workunit.client.0.smithi110.stdout:9/736: mkdir d7/d10/dfb 0 2024-09-21T04:15:26.659 INFO:tasks.workunit.client.0.smithi110.stdout:9/737: write d7/d96/d99/d3c/d35/d71/fc5 [537528,22411] 0 2024-09-21T04:15:26.659 INFO:tasks.workunit.client.1.smithi148.stdout:5/921: creat d3/d14/d1f/d26/d2e/d3d/f138 x:0 0 0 2024-09-21T04:15:26.660 INFO:tasks.workunit.client.0.smithi110.stdout:9/738: link d7/d96/d99/d3c/d35/f76 d7/d96/d99/d3c/d35/d71/ffc 0 2024-09-21T04:15:26.661 INFO:tasks.workunit.client.1.smithi148.stdout:9/961: creat d5/d2b/d4e/de4/f152 x:0 0 0 2024-09-21T04:15:26.661 INFO:tasks.workunit.client.0.smithi110.stdout:9/739: write d7/d96/d99/d3c/d35/f73 [153541,38744] 0 2024-09-21T04:15:26.662 INFO:tasks.workunit.client.1.smithi148.stdout:8/862: creat dc/d43/d4c/d4f/dc5/f10e x:0 0 0 2024-09-21T04:15:26.662 INFO:tasks.workunit.client.1.smithi148.stdout:8/863: creat dc/d13/d76/d5d/d78/f10f x:0 0 0 2024-09-21T04:15:26.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/864: rename dc/d43/c64 to dc/d13/d76/d50/db1/c110 0 2024-09-21T04:15:26.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/865: write dc/d13/d39/d6e/ffc [613262,25624] 0 2024-09-21T04:15:26.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/866: chown dc/d43 13265 1 2024-09-21T04:15:26.674 INFO:tasks.workunit.client.0.smithi110.stdout:8/837: read d0/d7/d13/d4b/f7d [5478101,86032] 0 2024-09-21T04:15:26.675 INFO:tasks.workunit.client.0.smithi110.stdout:2/696: write d2/d50/d56/f7a [3484779,79442] 0 2024-09-21T04:15:26.675 INFO:tasks.workunit.client.1.smithi148.stdout:8/867: write f9 [2179941,30055] 0 2024-09-21T04:15:26.679 INFO:tasks.workunit.client.0.smithi110.stdout:2/697: mknod d2/d1a/da3/db9/ce1 0 2024-09-21T04:15:26.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/868: rename dc/d13/d76/d42/f71 to dc/f111 0 2024-09-21T04:15:26.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/869: chown dc/d43/d4c/d4f/dc5/f10e 57 1 2024-09-21T04:15:26.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/870: fdatasync dc/d13/d39/fc6 0 2024-09-21T04:15:26.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/871: chown dc/d13/d3c/d81/f10d 409 1 2024-09-21T04:15:26.687 INFO:tasks.workunit.client.0.smithi110.stdout:9/740: write d7/d26/f90 [1790334,31913] 0 2024-09-21T04:15:26.690 INFO:tasks.workunit.client.0.smithi110.stdout:6/811: dwrite d4/d12/d17/d33/d3e/f53 [4194304,4194304] 0 2024-09-21T04:15:26.690 INFO:tasks.workunit.client.0.smithi110.stdout:6/812: write d4/d86/d8f/fc7 [708810,116010] 0 2024-09-21T04:15:26.690 INFO:tasks.workunit.client.0.smithi110.stdout:9/741: creat d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/ffd x:0 0 0 2024-09-21T04:15:26.691 INFO:tasks.workunit.client.0.smithi110.stdout:6/813: symlink d4/d86/l107 0 2024-09-21T04:15:26.693 INFO:tasks.workunit.client.0.smithi110.stdout:9/742: rename d7/d96/d99/d3c/f94 to d7/d96/d99/d3c/d1f/d43/db5/ffe 0 2024-09-21T04:15:26.693 INFO:tasks.workunit.client.0.smithi110.stdout:9/743: stat d7/d96/d99/d3c/d14 0 2024-09-21T04:15:26.694 INFO:tasks.workunit.client.0.smithi110.stdout:9/744: write d7/d96/d99/d3c/d1f/d72/d59/fbc [403779,67670] 0 2024-09-21T04:15:26.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/966: dwrite db/d24/d2e/d59/da7/dc1/dd2/d118/f12d [0,4194304] 0 2024-09-21T04:15:26.695 INFO:tasks.workunit.client.0.smithi110.stdout:6/814: rename d4/d12/d17/d33/d3e/f53 to d4/d12/d17/d81/f108 0 2024-09-21T04:15:26.695 INFO:tasks.workunit.client.1.smithi148.stdout:4/965: dwrite d4/d12/d16/d29/dbd/f68 [0,4194304] 0 2024-09-21T04:15:26.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/715: dwrite d0/d22/f72 [4194304,4194304] 0 2024-09-21T04:15:26.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/752: dwrite dd/d1c/dad/db2/f3b [0,4194304] 0 2024-09-21T04:15:26.697 INFO:tasks.workunit.client.0.smithi110.stdout:5/858: dwrite d1/d7/d18/f41 [4194304,4194304] 0 2024-09-21T04:15:26.697 INFO:tasks.workunit.client.0.smithi110.stdout:4/956: dwrite d17/d1e/d64/f136 [0,4194304] 0 2024-09-21T04:15:26.702 INFO:tasks.workunit.client.1.smithi148.stdout:5/922: dwrite d3/d14/d1f/ff0 [0,4194304] 0 2024-09-21T04:15:26.702 INFO:tasks.workunit.client.1.smithi148.stdout:5/923: readlink d3/d34/d3c/d5d/d75/l82 0 2024-09-21T04:15:26.703 INFO:tasks.workunit.client.0.smithi110.stdout:9/745: dread d7/d96/d99/d3c/d11/fc6 [0,4194304] 0 2024-09-21T04:15:26.706 INFO:tasks.workunit.client.1.smithi148.stdout:3/917: dwrite d3/d5/d6/f11b [0,4194304] 0 2024-09-21T04:15:26.707 INFO:tasks.workunit.client.0.smithi110.stdout:9/746: write d7/d96/d99/d3c/d11/fc6 [1064378,125597] 0 2024-09-21T04:15:26.709 INFO:tasks.workunit.client.0.smithi110.stdout:6/815: dread d4/d86/d8f/fc7 [0,4194304] 0 2024-09-21T04:15:26.709 INFO:tasks.workunit.client.0.smithi110.stdout:6/816: write d4/d78/daa/f84 [4761900,81669] 0 2024-09-21T04:15:26.709 INFO:tasks.workunit.client.0.smithi110.stdout:6/817: write d4/d12/d17/d33/d76/fc1 [1018124,105246] 0 2024-09-21T04:15:26.710 INFO:tasks.workunit.client.0.smithi110.stdout:5/859: rename d1/d49/d121 to d1/d22/db6/dc6/d12b 0 2024-09-21T04:15:26.713 INFO:tasks.workunit.client.0.smithi110.stdout:7/753: truncate dd/d1c/dad/db2/d56/f86 614225 0 2024-09-21T04:15:26.713 INFO:tasks.workunit.client.0.smithi110.stdout:7/754: dread - dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/fbb zero size 2024-09-21T04:15:26.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/957: write d17/d1e/d77/d10c/d57/db9/fd5 [496607,12145] 0 2024-09-21T04:15:26.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/958: write d17/d1e/f79 [607088,103906] 0 2024-09-21T04:15:26.722 INFO:tasks.workunit.client.1.smithi148.stdout:1/967: creat db/dd/d21/d9b/f13d x:0 0 0 2024-09-21T04:15:26.722 INFO:tasks.workunit.client.1.smithi148.stdout:1/968: chown db/dd 189365745 1 2024-09-21T04:15:26.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/747: symlink d7/d10/d88/d8f/da0/lff 0 2024-09-21T04:15:26.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/748: dread - d7/d96/d99/d3c/d14/fc2 zero size 2024-09-21T04:15:26.726 INFO:tasks.workunit.client.0.smithi110.stdout:8/838: dwrite d0/d7/d23/d6c/d12e/f104 [0,4194304] 0 2024-09-21T04:15:26.728 INFO:tasks.workunit.client.0.smithi110.stdout:6/818: creat d4/d12/d17/d2e/d40/dad/f109 x:0 0 0 2024-09-21T04:15:26.728 INFO:tasks.workunit.client.0.smithi110.stdout:8/839: dread d0/d7/d23/d82/f83 [0,4194304] 0 2024-09-21T04:15:26.731 INFO:tasks.workunit.client.0.smithi110.stdout:5/860: mkdir d1/d22/db6/dc6/d12b/d12c 0 2024-09-21T04:15:26.733 INFO:tasks.workunit.client.1.smithi148.stdout:3/918: creat d3/d5/d6/d2b/da2/da3/de4/d117/f13a x:0 0 0 2024-09-21T04:15:26.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/755: creat dd/d1c/dad/db2/d4a/da5/ff6 x:0 0 0 2024-09-21T04:15:26.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/756: write dd/d48/d9f/fef [552766,104186] 0 2024-09-21T04:15:26.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/757: dread - dd/d1c/dad/db2/d4a/f7f zero size 2024-09-21T04:15:26.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/962: dwrite d5/d60/d4c/d5e/fb4 [0,4194304] 0 2024-09-21T04:15:26.736 INFO:tasks.workunit.client.0.smithi110.stdout:2/698: dwrite d2/d50/d56/fc4 [0,4194304] 0 2024-09-21T04:15:26.736 INFO:tasks.workunit.client.1.smithi148.stdout:4/966: dwrite d4/d12/d16/d29/f5a [0,4194304] 0 2024-09-21T04:15:26.737 INFO:tasks.workunit.client.1.smithi148.stdout:1/969: rename db/d24/f26 to db/d24/d2a/dbd/df5/f13e 0 2024-09-21T04:15:26.738 INFO:tasks.workunit.client.0.smithi110.stdout:9/749: dwrite d7/d96/d99/d3c/d4f/d9f/d6f/fe8 [0,4194304] 0 2024-09-21T04:15:26.738 INFO:tasks.workunit.client.0.smithi110.stdout:4/959: symlink d17/d1e/d77/d10c/d20/d43/d51/d52/d132/l144 0 2024-09-21T04:15:26.738 INFO:tasks.workunit.client.0.smithi110.stdout:3/716: dwrite d0/d12/d2a/f31 [0,4194304] 0 2024-09-21T04:15:26.743 INFO:tasks.workunit.client.1.smithi148.stdout:5/924: dwrite d3/d14/d1f/d26/d2e/ddb/deb/f12c [0,4194304] 0 2024-09-21T04:15:26.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/963: rmdir d5/d1f/d81/dcf/d106/d80/d104 39 2024-09-21T04:15:26.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/964: readlink d5/d1f/d81/dcf/d106/laf 0 2024-09-21T04:15:26.746 INFO:tasks.workunit.client.0.smithi110.stdout:6/819: creat d4/d12/d16/d66/d7a/d9e/f10a x:0 0 0 2024-09-21T04:15:26.746 INFO:tasks.workunit.client.0.smithi110.stdout:6/820: dread - d4/d12/d17/d81/d90/dc3/fbd zero size 2024-09-21T04:15:26.746 INFO:tasks.workunit.client.0.smithi110.stdout:6/821: chown d4/d12/f1d 0 1 2024-09-21T04:15:26.749 INFO:tasks.workunit.client.0.smithi110.stdout:8/840: creat d0/d7/d13/d4b/ddc/f130 x:0 0 0 2024-09-21T04:15:26.751 INFO:tasks.workunit.client.0.smithi110.stdout:5/861: mknod d1/d7/d58/d9e/c12d 0 2024-09-21T04:15:26.753 INFO:tasks.workunit.client.0.smithi110.stdout:7/758: creat dd/d4d/dee/dbc/ff7 x:0 0 0 2024-09-21T04:15:26.753 INFO:tasks.workunit.client.0.smithi110.stdout:7/759: fsync dd/d1c/dad/db2/d4a/d52/f60 0 2024-09-21T04:15:26.753 INFO:tasks.workunit.client.1.smithi148.stdout:5/925: dread d3/d34/d78/dbf/fd1 [0,4194304] 0 2024-09-21T04:15:26.762 INFO:tasks.workunit.client.1.smithi148.stdout:3/919: dwrite d3/d5/d6/d2b/d96/f9d [0,4194304] 0 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.0.smithi110.stdout:2/699: getdents d2/d1a/d1f/d5f/d9d/d86/d9f 0 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.0.smithi110.stdout:2/700: chown d2/d1a/d8b/da5/da9/fc1 393479961 1 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.0.smithi110.stdout:2/701: read - d2/d81/fcb zero size 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.1.smithi148.stdout:5/926: creat d3/d14/d1f/db1/db8/dba/f139 x:0 0 0 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.1.smithi148.stdout:5/927: write d3/d14/d1f/d26/d2e/d56/f73 [1156198,74585] 0 2024-09-21T04:15:26.766 INFO:tasks.workunit.client.1.smithi148.stdout:5/928: fdatasync d3/d24/dad/fe6 0 2024-09-21T04:15:26.769 INFO:tasks.workunit.client.0.smithi110.stdout:9/750: truncate d7/d96/d99/d3c/d1f/f41 1286162 0 2024-09-21T04:15:26.769 INFO:tasks.workunit.client.0.smithi110.stdout:9/751: write d7/d96/d99/d3c/d1f/d72/f4d [1196022,12797] 0 2024-09-21T04:15:26.769 INFO:tasks.workunit.client.0.smithi110.stdout:9/752: fdatasync f2 0 2024-09-21T04:15:26.770 INFO:tasks.workunit.client.1.smithi148.stdout:4/967: dwrite d4/d87/d2d/dd0/fe3 [0,4194304] 0 2024-09-21T04:15:26.771 INFO:tasks.workunit.client.0.smithi110.stdout:3/717: truncate d0/d12/f9d 284889 0 2024-09-21T04:15:26.774 INFO:tasks.workunit.client.0.smithi110.stdout:4/960: dwrite d17/d1e/f48 [4194304,4194304] 0 2024-09-21T04:15:26.774 INFO:tasks.workunit.client.0.smithi110.stdout:8/841: symlink d0/d30/d54/l131 0 2024-09-21T04:15:26.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/929: creat d3/d34/d3c/f13a x:0 0 0 2024-09-21T04:15:26.776 INFO:tasks.workunit.client.0.smithi110.stdout:2/702: read d2/d1a/d1f/d65/f80 [148518,5637] 0 2024-09-21T04:15:26.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/970: dwrite db/dd/f23 [0,4194304] 0 2024-09-21T04:15:26.776 INFO:tasks.workunit.client.1.smithi148.stdout:1/971: fsync db/dd/d21/db0/f122 0 2024-09-21T04:15:26.777 INFO:tasks.workunit.client.0.smithi110.stdout:5/862: unlink d1/d22/f44 0 2024-09-21T04:15:26.778 INFO:tasks.workunit.client.0.smithi110.stdout:5/863: fsync d1/d39/fa4 0 2024-09-21T04:15:26.778 INFO:tasks.workunit.client.0.smithi110.stdout:5/864: creat d1/d7/d18/d3f/d117/f12e x:0 0 0 2024-09-21T04:15:26.778 INFO:tasks.workunit.client.0.smithi110.stdout:5/865: chown d1/d49/d4d/d1e/d4a/dae 30 1 2024-09-21T04:15:26.778 INFO:tasks.workunit.client.0.smithi110.stdout:5/866: write d1/d22/d6e/f126 [657742,87760] 0 2024-09-21T04:15:26.784 INFO:tasks.workunit.client.1.smithi148.stdout:1/972: symlink db/d46/d8f/d106/l13f 0 2024-09-21T04:15:26.787 INFO:tasks.workunit.client.0.smithi110.stdout:9/753: rename d7/d96/d99/d3c/d1f/d72/l44 to d7/d10/dfb/l100 0 2024-09-21T04:15:26.790 INFO:tasks.workunit.client.0.smithi110.stdout:5/867: dread d1/d39/f45 [0,4194304] 0 2024-09-21T04:15:26.791 INFO:tasks.workunit.client.1.smithi148.stdout:1/973: symlink db/d24/d2e/d71/d94/de4/d104/l140 0 2024-09-21T04:15:26.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/974: symlink db/d24/d2a/d112/l141 0 2024-09-21T04:15:26.796 INFO:tasks.workunit.client.0.smithi110.stdout:2/703: dwrite d2/fbb [0,4194304] 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/968: dwrite d4/d87/d2d/f117 [0,4194304] 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/969: readlink d4/d12/d16/d29/dbd/ldb 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/970: write d4/f6 [12914219,44284] 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/971: creat d4/d12/da7/f130 x:0 0 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/972: creat d4/d12/d75/db4/f131 x:0 0 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/973: readlink d4/d66/d7f/d116/l121 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/974: fdatasync d4/d66/d7f/d2a/f36 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/975: write d4/f51 [2819722,125013] 0 2024-09-21T04:15:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/976: fsync d4/d87/d91/fcf 0 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/975: mknod db/d24/d2a/da4/c142 0 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/976: fdatasync db/d24/d2a/da4/f139 0 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/977: dread - db/d24/fe7 zero size 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:1/978: stat db/d24/d2e/d59/f11d 0 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:3/920: dwrite d3/d5/db4/ff0 [0,4194304] 0 2024-09-21T04:15:26.799 INFO:tasks.workunit.client.1.smithi148.stdout:3/921: readlink d3/d5/d6/d12/l7a 0 2024-09-21T04:15:26.800 INFO:tasks.workunit.client.1.smithi148.stdout:0/981: sync 2024-09-21T04:15:26.801 INFO:tasks.workunit.client.1.smithi148.stdout:7/943: sync 2024-09-21T04:15:26.801 INFO:tasks.workunit.client.1.smithi148.stdout:7/944: write d5/d8/d6f/d7d/d120/d133/f13e [5073228,107017] 0 2024-09-21T04:15:26.801 INFO:tasks.workunit.client.0.smithi110.stdout:3/718: creat d0/d12/d3c/dad/ff9 x:0 0 0 2024-09-21T04:15:26.809 INFO:tasks.workunit.client.0.smithi110.stdout:4/961: symlink d17/d1e/d77/d10c/d20/l145 0 2024-09-21T04:15:26.812 INFO:tasks.workunit.client.0.smithi110.stdout:5/868: dwrite d1/d39/f50 [4194304,4194304] 0 2024-09-21T04:15:26.813 INFO:tasks.workunit.client.0.smithi110.stdout:8/842: mknod d0/d7/d6f/d86/c132 0 2024-09-21T04:15:26.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/979: rmdir db/d46/d4c/d82 39 2024-09-21T04:15:26.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/980: creat db/d46/d8f/f143 x:0 0 0 2024-09-21T04:15:26.820 INFO:tasks.workunit.client.1.smithi148.stdout:7/945: symlink d5/d14/d25/d40/db9/d108/l151 0 2024-09-21T04:15:26.820 INFO:tasks.workunit.client.1.smithi148.stdout:7/946: readlink d5/d8/l14a 0 2024-09-21T04:15:26.820 INFO:tasks.workunit.client.1.smithi148.stdout:7/947: readlink d5/d14/d25/d40/dab/dad/lcf 0 2024-09-21T04:15:26.823 INFO:tasks.workunit.client.0.smithi110.stdout:9/754: mkdir d7/d96/d99/d3c/d4f/d101 0 2024-09-21T04:15:26.826 INFO:tasks.workunit.client.0.smithi110.stdout:7/760: dwrite dd/d14/f73 [0,4194304] 0 2024-09-21T04:15:26.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/981: truncate db/d24/d2e/f9c 4206271 0 2024-09-21T04:15:26.829 INFO:tasks.workunit.client.1.smithi148.stdout:4/977: dwrite d4/f8 [4194304,4194304] 0 2024-09-21T04:15:26.830 INFO:tasks.workunit.client.1.smithi148.stdout:7/948: creat d5/d14/d25/d40/dab/f152 x:0 0 0 2024-09-21T04:15:26.831 INFO:tasks.workunit.client.0.smithi110.stdout:2/704: mknod d2/d1a/d1f/d5f/d9d/d86/d9b/ce2 0 2024-09-21T04:15:26.831 INFO:tasks.workunit.client.0.smithi110.stdout:2/705: chown d2/d1a/d1f/d65 252400 1 2024-09-21T04:15:26.831 INFO:tasks.workunit.client.0.smithi110.stdout:2/706: chown d2/d50/d56/c57 1 1 2024-09-21T04:15:26.833 INFO:tasks.workunit.client.0.smithi110.stdout:3/719: mkdir d0/d12/d16/d1a/d95/dfa 0 2024-09-21T04:15:26.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/720: truncate d0/d12/d16/d1a/d5c/faa 391518 0 2024-09-21T04:15:26.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/843: mkdir d0/d30/d54/d6b/d133 0 2024-09-21T04:15:26.839 INFO:tasks.workunit.client.0.smithi110.stdout:4/962: dwrite d17/d1e/d77/d10c/d20/dca/df3/ff2 [0,4194304] 0 2024-09-21T04:15:26.841 INFO:tasks.workunit.client.0.smithi110.stdout:9/755: rename d7/d96/d99/d3c/d4f/d9f/ce1 to d7/d10/d88/c102 0 2024-09-21T04:15:26.843 INFO:tasks.workunit.client.0.smithi110.stdout:2/707: symlink d2/d1a/d60/le3 0 2024-09-21T04:15:26.845 INFO:tasks.workunit.client.0.smithi110.stdout:3/721: truncate d0/d12/fb3 1069499 0 2024-09-21T04:15:26.845 INFO:tasks.workunit.client.0.smithi110.stdout:3/722: stat d0/d12/d3c/f7f 0 2024-09-21T04:15:26.846 INFO:tasks.workunit.client.0.smithi110.stdout:5/869: dwrite d1/d7/d18/f28 [4194304,4194304] 0 2024-09-21T04:15:26.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/922: dwrite d3/d1f/d4d/d127/f125 [4194304,4194304] 0 2024-09-21T04:15:26.847 INFO:tasks.workunit.client.1.smithi148.stdout:7/949: write d5/d8/d6f/d7d/f7f [3328380,47695] 0 2024-09-21T04:15:26.848 INFO:tasks.workunit.client.0.smithi110.stdout:4/963: unlink d17/d1e/d77/d10c/d20/d43/ff4 0 2024-09-21T04:15:26.849 INFO:tasks.workunit.client.0.smithi110.stdout:9/756: mknod d7/d10/d88/daa/c103 0 2024-09-21T04:15:26.851 INFO:tasks.workunit.client.1.smithi148.stdout:3/923: rename d3/d1f/d4d/d82/d23/db5/d105/f122 to d3/d1f/d4d/d82/f13b 0 2024-09-21T04:15:26.852 INFO:tasks.workunit.client.0.smithi110.stdout:7/761: dwrite dd/d4d/dee/dbc/fc2 [0,4194304] 0 2024-09-21T04:15:26.852 INFO:tasks.workunit.client.0.smithi110.stdout:7/762: readlink dd/d1c/dad/db2/d4a/l4e 0 2024-09-21T04:15:26.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/723: symlink d0/d12/d16/d1a/lfb 0 2024-09-21T04:15:26.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/724: creat d0/d51/d7b/ffc x:0 0 0 2024-09-21T04:15:26.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/924: mknod d3/d5/d6/d12/d43/d54/c13c 0 2024-09-21T04:15:26.857 INFO:tasks.workunit.client.0.smithi110.stdout:4/964: creat d17/d1e/d77/d10c/d20/d43/d51/d52/d98/d12d/f146 x:0 0 0 2024-09-21T04:15:26.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/925: mkdir d3/d5/d6/d12/d15/d13d 0 2024-09-21T04:15:26.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/926: write d3/d5/d6/ff3 [4592847,8610] 0 2024-09-21T04:15:26.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/927: chown d3/d5/d6/d12/d15/c102 59827280 1 2024-09-21T04:15:26.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/725: write d0/d12/d3c/d5a/d70/d81/fa8 [1183810,118599] 0 2024-09-21T04:15:26.860 INFO:tasks.workunit.client.0.smithi110.stdout:9/757: creat d7/d10/d88/f104 x:0 0 0 2024-09-21T04:15:26.863 INFO:tasks.workunit.client.1.smithi148.stdout:3/928: rmdir d3/d1f/d4d/d82/d23/d28/d46/db3 39 2024-09-21T04:15:26.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/708: dread d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:26.865 INFO:tasks.workunit.client.0.smithi110.stdout:7/763: creat dd/d1c/dad/db2/d56/d66/da6/d7a/ff8 x:0 0 0 2024-09-21T04:15:26.865 INFO:tasks.workunit.client.0.smithi110.stdout:7/764: readlink dd/d1c/dad/db2/d4a/d52/l8d 0 2024-09-21T04:15:26.865 INFO:tasks.workunit.client.0.smithi110.stdout:7/765: creat dd/d4d/dc1/ff9 x:0 0 0 2024-09-21T04:15:26.867 INFO:tasks.workunit.client.1.smithi148.stdout:3/929: rename d3/d5/db4/c11a to d3/d5/d6/d109/c13e 0 2024-09-21T04:15:26.867 INFO:tasks.workunit.client.1.smithi148.stdout:3/930: chown d3/d5/d6/d12/d43/d54/f6a 980309 1 2024-09-21T04:15:26.867 INFO:tasks.workunit.client.1.smithi148.stdout:3/931: creat d3/d5/d6/d2b/da2/da3/de4/d117/f13f x:0 0 0 2024-09-21T04:15:26.870 INFO:tasks.workunit.client.0.smithi110.stdout:5/870: symlink d1/d7/l12f 0 2024-09-21T04:15:26.870 INFO:tasks.workunit.client.0.smithi110.stdout:5/871: dread - d1/d49/d4d/fc8 zero size 2024-09-21T04:15:26.872 INFO:tasks.workunit.client.0.smithi110.stdout:4/965: creat d17/d1e/d13d/f147 x:0 0 0 2024-09-21T04:15:26.872 INFO:tasks.workunit.client.0.smithi110.stdout:4/966: dread - d17/d1e/d7b/daf/f139 zero size 2024-09-21T04:15:26.873 INFO:tasks.workunit.client.0.smithi110.stdout:3/726: dread d0/d12/d3c/ff4 [0,4194304] 0 2024-09-21T04:15:26.873 INFO:tasks.workunit.client.0.smithi110.stdout:3/727: chown d0/d12/d3c/f48 7024 1 2024-09-21T04:15:26.874 INFO:tasks.workunit.client.1.smithi148.stdout:3/932: unlink d3/d5/d6/d2b/da2/de0/cf8 0 2024-09-21T04:15:26.876 INFO:tasks.workunit.client.0.smithi110.stdout:9/758: rename d7/d26/f5b to d7/d96/d99/d3c/d14/d31/f105 0 2024-09-21T04:15:26.878 INFO:tasks.workunit.client.1.smithi148.stdout:3/933: rename d3/d5/d6/d12/d15/da7/fd7 to d3/d1f/d4d/d127/f140 0 2024-09-21T04:15:26.878 INFO:tasks.workunit.client.1.smithi148.stdout:3/934: readlink d3/d5/d6/d12/d43/l62 0 2024-09-21T04:15:26.878 INFO:tasks.workunit.client.1.smithi148.stdout:3/935: creat d3/d5/d6/d2b/d96/f141 x:0 0 0 2024-09-21T04:15:26.880 INFO:tasks.workunit.client.0.smithi110.stdout:7/766: creat dd/d14/ffa x:0 0 0 2024-09-21T04:15:26.881 INFO:tasks.workunit.client.0.smithi110.stdout:4/967: mkdir d17/d1e/d13d/d148 0 2024-09-21T04:15:26.881 INFO:tasks.workunit.client.0.smithi110.stdout:4/968: chown d17/d1e/d77/cb6 190811 1 2024-09-21T04:15:26.882 INFO:tasks.workunit.client.0.smithi110.stdout:3/728: getdents d0/d12/d16/d1a/dc6 0 2024-09-21T04:15:26.882 INFO:tasks.workunit.client.0.smithi110.stdout:3/729: write d0/d22/d29/f8b [1824679,106408] 0 2024-09-21T04:15:26.884 INFO:tasks.workunit.client.0.smithi110.stdout:7/767: rename dd/c10 to dd/d1c/dad/cfb 0 2024-09-21T04:15:26.885 INFO:tasks.workunit.client.0.smithi110.stdout:3/730: mknod d0/d12/d2a/db4/cfd 0 2024-09-21T04:15:26.885 INFO:tasks.workunit.client.0.smithi110.stdout:3/731: chown d0/d12/d16 263377 1 2024-09-21T04:15:26.886 INFO:tasks.workunit.client.0.smithi110.stdout:7/768: symlink dd/d1c/dad/db2/d56/d66/da6/d7a/lfc 0 2024-09-21T04:15:26.887 INFO:tasks.workunit.client.1.smithi148.stdout:3/936: write d3/d5/db4/ff0 [3090689,100235] 0 2024-09-21T04:15:26.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/982: dwrite db/dd/d21/d89/fd1 [0,4194304] 0 2024-09-21T04:15:26.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/983: chown db/d24/d2e/d71/dfc/l116 10223208 1 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/769: rmdir dd/d14/d17/dd1/df5 0 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/770: chown dd/d1c/dad/db2/d4a/da5 4341573 1 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/771: dread - dd/d1c/dad/db2/d56/fc4 zero size 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/772: dread - dd/d14/d17/fb9 zero size 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/773: creat dd/d1c/dad/db2/d4a/da5/ffd x:0 0 0 2024-09-21T04:15:26.889 INFO:tasks.workunit.client.0.smithi110.stdout:3/732: getdents d0/d12/d16/d1a/d6f 0 2024-09-21T04:15:26.890 INFO:tasks.workunit.client.1.smithi148.stdout:4/978: dwrite d4/d87/d2d/dd0/fd6 [4194304,4194304] 0 2024-09-21T04:15:26.890 INFO:tasks.workunit.client.1.smithi148.stdout:7/950: dwrite d5/d8/d6f/d7d/d90/dee/f104 [4194304,4194304] 0 2024-09-21T04:15:26.892 INFO:tasks.workunit.client.0.smithi110.stdout:8/844: dwrite d0/d7/d13/d46/f4c [0,4194304] 0 2024-09-21T04:15:26.892 INFO:tasks.workunit.client.0.smithi110.stdout:2/709: dwrite d2/d1a/d1f/d65/f75 [4194304,4194304] 0 2024-09-21T04:15:26.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/759: write d7/d10/f3f [2045862,8375] 0 2024-09-21T04:15:26.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/760: read f2 [1004425,1582] 0 2024-09-21T04:15:26.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/761: fsync d7/d96/d99/d3c/d35/f76 0 2024-09-21T04:15:26.894 INFO:tasks.workunit.client.1.smithi148.stdout:3/937: symlink d3/d5/d6/d2b/d137/l142 0 2024-09-21T04:15:26.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/969: dread d17/d3e/f9c [0,4194304] 0 2024-09-21T04:15:26.901 INFO:tasks.workunit.client.1.smithi148.stdout:4/979: mkdir d4/d66/d7f/d9a/dce/d12f/dcc/d101/d12c/d132 0 2024-09-21T04:15:26.901 INFO:tasks.workunit.client.1.smithi148.stdout:4/980: dread - d4/d12/d75/d85/fac zero size 2024-09-21T04:15:26.908 INFO:tasks.workunit.client.0.smithi110.stdout:8/845: write d0/d7/d13/f2d [831084,55226] 0 2024-09-21T04:15:26.908 INFO:tasks.workunit.client.1.smithi148.stdout:1/984: dread db/d24/d2e/d59/da7/dc1/dd2/d118/f12d [0,4194304] 0 2024-09-21T04:15:26.912 INFO:tasks.workunit.client.1.smithi148.stdout:7/951: dwrite d5/d8/d6f/d7d/d90/d28/f9c [4194304,4194304] 0 2024-09-21T04:15:26.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/710: getdents d2/d50/d9c 0 2024-09-21T04:15:26.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/711: truncate d2/d1a/d1f/d29/d19/fdb 443334 0 2024-09-21T04:15:26.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:26 smithi110 ceph-mon[27723]: pgmap v10: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 282 MiB/s wr, 420 op/s 2024-09-21T04:15:26.918 INFO:tasks.workunit.client.0.smithi110.stdout:8/846: getdents d0/d30/d34/d3e/d60/dec 0 2024-09-21T04:15:26.920 INFO:tasks.workunit.client.1.smithi148.stdout:3/938: mknod d3/d1f/d4d/d82/d23/d28/d46/db3/c143 0 2024-09-21T04:15:26.920 INFO:tasks.workunit.client.1.smithi148.stdout:3/939: dread - d3/d1f/d4d/d82/d21/f4e zero size 2024-09-21T04:15:26.921 INFO:tasks.workunit.client.0.smithi110.stdout:4/970: rmdir d17/d1e/d64 39 2024-09-21T04:15:26.921 INFO:tasks.workunit.client.0.smithi110.stdout:4/971: dread - d17/d3e/f115 zero size 2024-09-21T04:15:26.922 INFO:tasks.workunit.client.1.smithi148.stdout:1/985: symlink db/d46/d4c/l144 0 2024-09-21T04:15:26.922 INFO:tasks.workunit.client.1.smithi148.stdout:1/986: truncate db/d46/fb6 747210 0 2024-09-21T04:15:26.924 INFO:tasks.workunit.client.0.smithi110.stdout:2/712: getdents d2/d1a/d1f/d29 0 2024-09-21T04:15:26.924 INFO:tasks.workunit.client.0.smithi110.stdout:2/713: creat d2/d1a/d1f/fe4 x:0 0 0 2024-09-21T04:15:26.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/952: mkdir d5/d14/d25/d40/dab/dad/db8/d153 0 2024-09-21T04:15:26.927 INFO:tasks.workunit.client.1.smithi148.stdout:7/953: readlink d5/d8/d6f/d7d/d90/l59 0 2024-09-21T04:15:26.930 INFO:tasks.workunit.client.1.smithi148.stdout:1/987: rename db/dd/fae to db/d46/d76/f145 0 2024-09-21T04:15:26.930 INFO:tasks.workunit.client.1.smithi148.stdout:1/988: truncate db/dd/d21/d9b/f13d 682581 0 2024-09-21T04:15:26.932 INFO:tasks.workunit.client.1.smithi148.stdout:7/954: rename d5/d8/d6f/d7d/df5/l124 to d5/d14/d25/d40/dab/l154 0 2024-09-21T04:15:26.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/955: creat d5/daa/f155 x:0 0 0 2024-09-21T04:15:26.936 INFO:tasks.workunit.client.1.smithi148.stdout:7/956: mkdir d5/d14/d25/d40/d7b/d156 0 2024-09-21T04:15:26.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/940: dread d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:26.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/941: readlink d3/d1f/d4d/d82/d74/lb9 0 2024-09-21T04:15:26.938 INFO:tasks.workunit.client.1.smithi148.stdout:7/957: symlink d5/d8/d6f/d7d/d90/dee/l157 0 2024-09-21T04:15:26.938 INFO:tasks.workunit.client.1.smithi148.stdout:7/958: write d5/d8/d6f/d7d/d90/dee/f118 [696300,39495] 0 2024-09-21T04:15:26.938 INFO:tasks.workunit.client.1.smithi148.stdout:7/959: chown d5/d14/d25/d140 684 1 2024-09-21T04:15:26.939 INFO:tasks.workunit.client.1.smithi148.stdout:3/942: chown d3/d5/c12e 62460 1 2024-09-21T04:15:26.941 INFO:tasks.workunit.client.1.smithi148.stdout:7/960: creat d5/d8/d6f/d7d/df5/d149/f158 x:0 0 0 2024-09-21T04:15:26.941 INFO:tasks.workunit.client.0.smithi110.stdout:8/847: dread d0/d30/d34/d3e/d32/f7c [0,4194304] 0 2024-09-21T04:15:26.942 INFO:tasks.workunit.client.1.smithi148.stdout:3/943: write d3/d5/d5c/fee [118937,115992] 0 2024-09-21T04:15:26.944 INFO:tasks.workunit.client.1.smithi148.stdout:7/961: symlink d5/daa/dc6/l159 0 2024-09-21T04:15:26.946 INFO:tasks.workunit.client.0.smithi110.stdout:8/848: creat d0/d7/d13/d4b/f134 x:0 0 0 2024-09-21T04:15:26.948 INFO:tasks.workunit.client.1.smithi148.stdout:7/962: symlink d5/d8/l15a 0 2024-09-21T04:15:26.949 INFO:tasks.workunit.client.1.smithi148.stdout:7/963: write d5/daa/f11c [491489,100078] 0 2024-09-21T04:15:26.949 INFO:tasks.workunit.client.1.smithi148.stdout:7/964: chown d5/d8/d6f/f143 289814 1 2024-09-21T04:15:26.949 INFO:tasks.workunit.client.1.smithi148.stdout:7/965: write d5/d14/d25/d79/d86/fa3 [2093226,62037] 0 2024-09-21T04:15:26.953 INFO:tasks.workunit.client.1.smithi148.stdout:7/966: mknod d5/d14/d25/d40/c15b 0 2024-09-21T04:15:26.953 INFO:tasks.workunit.client.1.smithi148.stdout:7/967: truncate d5/d8/d6f/f143 91157 0 2024-09-21T04:15:26.955 INFO:tasks.workunit.client.1.smithi148.stdout:7/968: creat d5/d8/d6f/d7d/d90/d28/f15c x:0 0 0 2024-09-21T04:15:26.955 INFO:tasks.workunit.client.1.smithi148.stdout:7/969: stat d5/d8/d6f/d7d/df5/d149/f158 0 2024-09-21T04:15:26.955 INFO:tasks.workunit.client.1.smithi148.stdout:7/970: read - d5/d8/d6f/ff3 zero size 2024-09-21T04:15:26.956 INFO:tasks.workunit.client.1.smithi148.stdout:7/971: symlink d5/d14/d25/d40/db9/l15d 0 2024-09-21T04:15:26.963 INFO:tasks.workunit.client.0.smithi110.stdout:5/872: dwrite d1/d22/d6e/f126 [0,4194304] 0 2024-09-21T04:15:26.967 INFO:tasks.workunit.client.0.smithi110.stdout:4/972: dwrite d17/d1e/d13d/f87 [0,4194304] 0 2024-09-21T04:15:26.967 INFO:tasks.workunit.client.0.smithi110.stdout:4/973: chown d17/d1e/d21/l4f 106670 1 2024-09-21T04:15:26.967 INFO:tasks.workunit.client.0.smithi110.stdout:3/733: dwrite d0/d12/d3c/dad/ff9 [0,4194304] 0 2024-09-21T04:15:26.967 INFO:tasks.workunit.client.0.smithi110.stdout:9/762: dwrite d7/d96/d99/d3c/d35/d71/ffc [0,4194304] 0 2024-09-21T04:15:26.967 INFO:tasks.workunit.client.0.smithi110.stdout:9/763: write d7/d10/fde [864547,107987] 0 2024-09-21T04:15:26.970 INFO:tasks.workunit.client.0.smithi110.stdout:5/873: read d1/d49/d4d/d1e/d4a/dd4/dd5/f118 [1660546,21419] 0 2024-09-21T04:15:26.982 INFO:tasks.workunit.client.0.smithi110.stdout:2/714: dwrite d2/d1a/d1f/f5d [0,4194304] 0 2024-09-21T04:15:26.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/989: dwrite db/d24/d2e/d71/d94/dde/de1/d129/f12f [0,4194304] 0 2024-09-21T04:15:26.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/990: chown db/d24/d2e/d71/le0 17110939 1 2024-09-21T04:15:26.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/991: write db/d24/d2a/da4/f139 [483703,73091] 0 2024-09-21T04:15:26.983 INFO:tasks.workunit.client.0.smithi110.stdout:4/974: mkdir d17/d1e/d21/de1/d149 0 2024-09-21T04:15:26.983 INFO:tasks.workunit.client.0.smithi110.stdout:4/975: fsync d17/d1e/fbf 0 2024-09-21T04:15:26.985 INFO:tasks.workunit.client.1.smithi148.stdout:1/992: creat db/d24/d2e/d59/da7/dc1/dd2/d118/f146 x:0 0 0 2024-09-21T04:15:26.992 INFO:tasks.workunit.client.1.smithi148.stdout:7/972: dwrite d5/d14/d25/d79/d86/f127 [0,4194304] 0 2024-09-21T04:15:26.993 INFO:tasks.workunit.client.0.smithi110.stdout:8/849: dwrite d0/d30/d34/d3e/f88 [0,4194304] 0 2024-09-21T04:15:26.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/944: dwrite d3/d1f/d4d/d127/f125 [4194304,4194304] 0 2024-09-21T04:15:26.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/945: write d3/d1f/d4d/d82/d23/d28/d46/db3/fea [1618540,123715] 0 2024-09-21T04:15:26.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/946: stat d3/d1f/d4d/d82/d23/d28/d46/f5b 0 2024-09-21T04:15:26.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/947: write d3/d1f/d4d/ff9 [992112,9497] 0 2024-09-21T04:15:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/993: write db/dd/f23 [2662578,17126] 0 2024-09-21T04:15:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/994: dread - db/d24/d2e/d59/da7/f103 zero size 2024-09-21T04:15:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/995: write db/d24/d2a/da4/fec [558684,63933] 0 2024-09-21T04:15:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/996: dread - db/d46/d8f/f11c zero size 2024-09-21T04:15:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/997: read db/dd/d21/d89/fab [2473087,36964] 0 2024-09-21T04:15:26.999 INFO:tasks.workunit.client.0.smithi110.stdout:9/764: dwrite d7/d96/d99/d3c/d35/d71/fc5 [0,4194304] 0 2024-09-21T04:15:26.999 INFO:tasks.workunit.client.0.smithi110.stdout:2/715: dwrite d2/d1a/d1f/d29/d19/fda [0,4194304] 0 2024-09-21T04:15:27.001 INFO:tasks.workunit.client.0.smithi110.stdout:2/716: dread d2/d1a/d1f/d2a/f61 [0,4194304] 0 2024-09-21T04:15:27.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/973: dwrite d5/daa/f11c [0,4194304] 0 2024-09-21T04:15:27.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/974: write d5/d14/d25/d79/fe2 [1100658,58210] 0 2024-09-21T04:15:27.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/975: getdents d5/d14/d25/d140 0 2024-09-21T04:15:27.020 INFO:tasks.workunit.client.1.smithi148.stdout:3/948: dwrite d3/fc0 [0,4194304] 0 2024-09-21T04:15:27.024 INFO:tasks.workunit.client.0.smithi110.stdout:2/717: dwrite d2/d50/d56/f7e [0,4194304] 0 2024-09-21T04:15:27.024 INFO:tasks.workunit.client.0.smithi110.stdout:2/718: chown d2/d1a/d1f/d5f/d9d/d86/d9b/l98 9011886 1 2024-09-21T04:15:27.025 INFO:tasks.workunit.client.0.smithi110.stdout:3/734: mknod d0/d12/d16/d1a/d6f/dd9/cfe 0 2024-09-21T04:15:27.026 INFO:tasks.workunit.client.0.smithi110.stdout:5/874: link d1/d49/d6f/fc4 d1/dc/f130 0 2024-09-21T04:15:27.034 INFO:tasks.workunit.client.0.smithi110.stdout:2/719: write d2/d1a/d1f/d29/d19/fda [1464604,35802] 0 2024-09-21T04:15:27.035 INFO:tasks.workunit.client.0.smithi110.stdout:4/976: rmdir d17/d1e/d77/d10c/d20/d43/d51/d7a/d96 39 2024-09-21T04:15:27.036 INFO:tasks.workunit.client.0.smithi110.stdout:8/850: truncate d0/d7/fd 716359 0 2024-09-21T04:15:27.037 INFO:tasks.workunit.client.1.smithi148.stdout:8/872: sync 2024-09-21T04:15:27.044 INFO:tasks.workunit.client.0.smithi110.stdout:8/851: dread d0/d30/d54/f98 [0,4194304] 0 2024-09-21T04:15:27.044 INFO:tasks.workunit.client.0.smithi110.stdout:5/875: dwrite d1/d49/d6f/fa8 [0,4194304] 0 2024-09-21T04:15:27.044 INFO:tasks.workunit.client.0.smithi110.stdout:5/876: truncate d1/d39/da2/fd8 140785 0 2024-09-21T04:15:27.045 INFO:tasks.workunit.client.0.smithi110.stdout:9/765: symlink d7/d10/d88/d8f/da0/l106 0 2024-09-21T04:15:27.048 INFO:tasks.workunit.client.0.smithi110.stdout:2/720: dread d2/d1a/d1f/d29/f45 [4194304,4194304] 0 2024-09-21T04:15:27.052 INFO:tasks.workunit.client.0.smithi110.stdout:3/735: symlink d0/d12/d16/d1a/d6f/dd9/lff 0 2024-09-21T04:15:27.060 INFO:tasks.workunit.client.1.smithi148.stdout:1/998: symlink db/d24/l147 0 2024-09-21T04:15:27.061 INFO:tasks.workunit.client.1.smithi148.stdout:1/999: fsync db/d24/d2a/dbd/fe8 0 2024-09-21T04:15:27.065 INFO:tasks.workunit.client.1.smithi148.stdout:8/873: dwrite dc/f14 [4194304,4194304] 0 2024-09-21T04:15:27.065 INFO:tasks.workunit.client.1.smithi148.stdout:8/874: creat dc/d13/d76/d5d/d78/f112 x:0 0 0 2024-09-21T04:15:27.065 INFO:tasks.workunit.client.1.smithi148.stdout:7/976: mknod d5/d14/d25/d40/d7b/d156/c15e 0 2024-09-21T04:15:27.067 INFO:tasks.workunit.client.0.smithi110.stdout:9/766: dwrite d7/d96/d99/d3c/d11/f81 [0,4194304] 0 2024-09-21T04:15:27.071 INFO:tasks.workunit.client.1.smithi148.stdout:3/949: symlink d3/d5/d6/d109/l144 0 2024-09-21T04:15:27.071 INFO:tasks.workunit.client.1.smithi148.stdout:3/950: fdatasync d3/d1f/d4d/d82/d23/fec 0 2024-09-21T04:15:27.071 INFO:tasks.workunit.client.1.smithi148.stdout:3/951: chown d3/d5/d6/d2b/da2/fc8 40586 1 2024-09-21T04:15:27.074 INFO:tasks.workunit.client.1.smithi148.stdout:8/875: rename dc/d13/d3c/f72 to dc/d13/d39/d6e/dac/f113 0 2024-09-21T04:15:27.075 INFO:tasks.workunit.client.1.smithi148.stdout:5/930: sync 2024-09-21T04:15:27.075 INFO:tasks.workunit.client.1.smithi148.stdout:9/965: sync 2024-09-21T04:15:27.075 INFO:tasks.workunit.client.1.smithi148.stdout:5/931: chown d3/d14/d1f/d26/d2e/d3d/ld5 23 1 2024-09-21T04:15:27.076 INFO:tasks.workunit.client.1.smithi148.stdout:7/977: creat d5/d8/d150/f15f x:0 0 0 2024-09-21T04:15:27.079 INFO:tasks.workunit.client.1.smithi148.stdout:8/876: getdents dc/d13/d28/d62 0 2024-09-21T04:15:27.081 INFO:tasks.workunit.client.1.smithi148.stdout:9/966: creat d5/d1e/d37/d46/f153 x:0 0 0 2024-09-21T04:15:27.081 INFO:tasks.workunit.client.1.smithi148.stdout:9/967: write f1 [3553043,87417] 0 2024-09-21T04:15:27.081 INFO:tasks.workunit.client.1.smithi148.stdout:9/968: write d5/d2b/d32/fa4 [596991,75152] 0 2024-09-21T04:15:27.083 INFO:tasks.workunit.client.1.smithi148.stdout:7/978: rename d5/daa/f11c to d5/d8/d6f/dcd/d132/d14e/f160 0 2024-09-21T04:15:27.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/877: rename dc/d13/d76/l3a to dc/d13/d76/l114 0 2024-09-21T04:15:27.086 INFO:tasks.workunit.client.0.smithi110.stdout:4/977: mkdir d17/d1e/d77/d10c/d20/dca/df3/d14a 0 2024-09-21T04:15:27.088 INFO:tasks.workunit.client.0.smithi110.stdout:8/852: creat d0/d7/d13/f135 x:0 0 0 2024-09-21T04:15:27.088 INFO:tasks.workunit.client.0.smithi110.stdout:8/853: read - d0/d7/d13/f10d zero size 2024-09-21T04:15:27.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/969: symlink d5/d60/d28/d2f/dbf/dc2/l154 0 2024-09-21T04:15:27.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/970: stat d5/de3/de7/cb5 0 2024-09-21T04:15:27.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/971: stat d5/d2b/dbe/f52 0 2024-09-21T04:15:27.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/972: fdatasync d5/d1e/f65 0 2024-09-21T04:15:27.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/973: fsync d5/fa5 0 2024-09-21T04:15:27.090 INFO:tasks.workunit.client.0.smithi110.stdout:8/854: dread d0/d30/d34/d3e/f88 [4194304,4194304] 0 2024-09-21T04:15:27.093 INFO:tasks.workunit.client.1.smithi148.stdout:8/878: unlink dc/d13/d76/d5d/fa6 0 2024-09-21T04:15:27.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/877: mkdir d1/dc/ded/d109/d131 0 2024-09-21T04:15:27.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/878: write d1/d22/ffa [5042908,67088] 0 2024-09-21T04:15:27.096 INFO:tasks.workunit.client.1.smithi148.stdout:9/974: symlink d5/d2b/d32/d103/d123/l155 0 2024-09-21T04:15:27.096 INFO:tasks.workunit.client.0.smithi110.stdout:2/721: link d2/d1a/d1f/l5b d2/d1a/d1f/d5f/dc0/le5 0 2024-09-21T04:15:27.097 INFO:tasks.workunit.client.0.smithi110.stdout:3/736: mknod d0/d51/d7b/c100 0 2024-09-21T04:15:27.098 INFO:tasks.workunit.client.1.smithi148.stdout:9/975: truncate d5/d2b/dbe/f9a 4079781 0 2024-09-21T04:15:27.098 INFO:tasks.workunit.client.0.smithi110.stdout:9/767: mknod d7/d96/d99/d3c/d14/c107 0 2024-09-21T04:15:27.100 INFO:tasks.workunit.client.0.smithi110.stdout:5/879: link d1/d7/d58/d9e/c12d d1/d49/d4d/d1e/d4a/dfb/c132 0 2024-09-21T04:15:27.102 INFO:tasks.workunit.client.0.smithi110.stdout:2/722: symlink d2/le6 0 2024-09-21T04:15:27.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/976: mkdir d5/d2b/d32/d103/d156 0 2024-09-21T04:15:27.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/977: write d5/d60/de1/f138 [939765,42787] 0 2024-09-21T04:15:27.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/982: sync 2024-09-21T04:15:27.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/983: write d5/d82/fc4 [1276377,102416] 0 2024-09-21T04:15:27.104 INFO:tasks.workunit.client.0.smithi110.stdout:3/737: rmdir d0/d12/d16/d1a/d95/dfa 0 2024-09-21T04:15:27.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/768: mkdir d7/d96/d99/d3c/d4f/d9f/d6f/d108 0 2024-09-21T04:15:27.106 INFO:tasks.workunit.client.0.smithi110.stdout:2/723: mkdir d2/de7 0 2024-09-21T04:15:27.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/978: dread d5/d1f/d81/f12a [0,4194304] 0 2024-09-21T04:15:27.109 INFO:tasks.workunit.client.0.smithi110.stdout:3/738: creat d0/d12/d3c/d5a/d65/f101 x:0 0 0 2024-09-21T04:15:27.110 INFO:tasks.workunit.client.0.smithi110.stdout:2/724: rmdir d2/d1a/d1f/d5f/d9d/d86/d9f/dc3 39 2024-09-21T04:15:27.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/739: write d0/d12/d2a/d80/d9c/fb8 [547044,39501] 0 2024-09-21T04:15:27.111 INFO:tasks.workunit.client.1.smithi148.stdout:0/984: link d5/dcb/de6/d110/d2a/dcd/df8/c117 d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/de8/df7/c138 0 2024-09-21T04:15:27.111 INFO:tasks.workunit.client.1.smithi148.stdout:0/985: truncate d5/dcb/de6/d110/d2a/dcd/df8/da5/f64 5353613 0 2024-09-21T04:15:27.112 INFO:tasks.workunit.client.1.smithi148.stdout:9/979: mkdir d5/d60/d28/d2f/d10a/d157 0 2024-09-21T04:15:27.112 INFO:tasks.workunit.client.1.smithi148.stdout:9/980: chown d5/d1e/d37/d50/c11e 3 1 2024-09-21T04:15:27.113 INFO:tasks.workunit.client.1.smithi148.stdout:9/981: fdatasync d5/d1f/d81/dcf/d106/d80/fa1 0 2024-09-21T04:15:27.113 INFO:tasks.workunit.client.1.smithi148.stdout:9/982: fsync d5/d1f/d81/f12a 0 2024-09-21T04:15:27.113 INFO:tasks.workunit.client.1.smithi148.stdout:9/983: write d5/d1f/f24 [5754060,118492] 0 2024-09-21T04:15:27.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/986: mkdir d5/dcb/de6/d110/d7/dd0/d139 0 2024-09-21T04:15:27.114 INFO:tasks.workunit.client.1.smithi148.stdout:9/984: mknod d5/d60/d28/d2f/d10a/d157/c158 0 2024-09-21T04:15:27.114 INFO:tasks.workunit.client.1.smithi148.stdout:9/985: write d5/d1e/d37/f88 [52022,93718] 0 2024-09-21T04:15:27.114 INFO:tasks.workunit.client.1.smithi148.stdout:0/987: creat d5/dcb/de6/d110/d2a/d45/f13a x:0 0 0 2024-09-21T04:15:27.123 INFO:tasks.workunit.client.1.smithi148.stdout:5/932: dwrite d3/d14/d1f/d26/d2e/d56/da7/fd9 [0,4194304] 0 2024-09-21T04:15:27.131 INFO:tasks.workunit.client.1.smithi148.stdout:5/933: dread d3/f9 [0,4194304] 0 2024-09-21T04:15:27.133 INFO:tasks.workunit.client.1.smithi148.stdout:5/934: symlink d3/d34/d78/dbf/d6f/l13b 0 2024-09-21T04:15:27.133 INFO:tasks.workunit.client.0.smithi110.stdout:2/725: dread d2/d1a/d1f/d5f/d9d/d86/d9b/fb4 [0,4194304] 0 2024-09-21T04:15:27.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/726: read - d2/d1a/d8b/da5/fdf zero size 2024-09-21T04:15:27.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/727: chown d2/d1a/d8b/da5 274337448 1 2024-09-21T04:15:27.135 INFO:tasks.workunit.client.0.smithi110.stdout:2/728: creat d2/d1a/d1f/d65/fe8 x:0 0 0 2024-09-21T04:15:27.135 INFO:tasks.workunit.client.0.smithi110.stdout:2/729: getdents d2/de7 0 2024-09-21T04:15:27.135 INFO:tasks.workunit.client.0.smithi110.stdout:2/730: creat d2/d1a/d1f/d5f/fe9 x:0 0 0 2024-09-21T04:15:27.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/731: creat d2/d1a/d1f/d5f/d9d/d86/d9f/fea x:0 0 0 2024-09-21T04:15:27.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/732: write d2/f44 [3072525,36933] 0 2024-09-21T04:15:27.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/733: fsync d2/d1a/d1f/d65/f80 0 2024-09-21T04:15:27.141 INFO:tasks.workunit.client.1.smithi148.stdout:7/979: dwrite d5/d14/ff2 [0,4194304] 0 2024-09-21T04:15:27.144 INFO:tasks.workunit.client.1.smithi148.stdout:7/980: symlink d5/d14/d25/d40/dab/dad/l161 0 2024-09-21T04:15:27.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/952: dwrite d3/d5/d6/d12/d43/fe2 [0,4194304] 0 2024-09-21T04:15:27.145 INFO:tasks.workunit.client.1.smithi148.stdout:7/981: unlink d5/d8/d6f/d7d/d120/d133/f13e 0 2024-09-21T04:15:27.145 INFO:tasks.workunit.client.1.smithi148.stdout:7/982: write d5/d8/d6f/ff3 [661301,68068] 0 2024-09-21T04:15:27.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/978: dwrite d17/d1e/f79 [0,4194304] 0 2024-09-21T04:15:27.151 INFO:tasks.workunit.client.0.smithi110.stdout:4/979: mknod d17/d1e/d77/dcf/c14b 0 2024-09-21T04:15:27.151 INFO:tasks.workunit.client.0.smithi110.stdout:4/980: write d17/d1e/d77/d10c/d57/fd2 [588771,5522] 0 2024-09-21T04:15:27.152 INFO:tasks.workunit.client.0.smithi110.stdout:4/981: write d17/d1e/d77/d10c/d20/dca/fd6 [809017,58434] 0 2024-09-21T04:15:27.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/953: creat d3/d1f/d4d/f145 x:0 0 0 2024-09-21T04:15:27.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/954: chown d3/d5/d6/d12/d43/d54/d9b/dd6/cd8 1 1 2024-09-21T04:15:27.154 INFO:tasks.workunit.client.0.smithi110.stdout:4/982: rename d17/fe9 to d17/d1e/d77/d10c/d20/d43/d51/d7a/d96/ddf/d114/f14c 0 2024-09-21T04:15:27.155 INFO:tasks.workunit.client.0.smithi110.stdout:4/983: mkdir d17/d1e/d21/de1/d14d 0 2024-09-21T04:15:27.155 INFO:tasks.workunit.client.0.smithi110.stdout:4/984: creat d17/d1e/d21/f14e x:0 0 0 2024-09-21T04:15:27.155 INFO:tasks.workunit.client.0.smithi110.stdout:4/985: dread - d17/d1e/d21/f54 zero size 2024-09-21T04:15:27.156 INFO:tasks.workunit.client.0.smithi110.stdout:4/986: truncate d17/d1e/d21/f2e 1060553 0 2024-09-21T04:15:27.157 INFO:tasks.workunit.client.0.smithi110.stdout:4/987: mknod d17/d1e/d77/d10c/d20/dca/df3/d107/c14f 0 2024-09-21T04:15:27.158 INFO:tasks.workunit.client.0.smithi110.stdout:4/988: truncate d17/d1e/d7b/fb8 60941 0 2024-09-21T04:15:27.158 INFO:tasks.workunit.client.0.smithi110.stdout:4/989: fsync d17/f3c 0 2024-09-21T04:15:27.162 INFO:tasks.workunit.client.1.smithi148.stdout:8/879: dwrite dc/d3e/f8b [0,4194304] 0 2024-09-21T04:15:27.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/983: dread d5/d8/d6f/d7d/d90/f6e [0,4194304] 0 2024-09-21T04:15:27.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/984: symlink d5/d8/d6f/d7d/d90/d13c/l162 0 2024-09-21T04:15:27.164 INFO:tasks.workunit.client.1.smithi148.stdout:3/955: dread d3/d1f/d4d/d82/d21/fd2 [0,4194304] 0 2024-09-21T04:15:27.164 INFO:tasks.workunit.client.0.smithi110.stdout:8/855: dwrite d0/d7/d13/d46/f51 [4194304,4194304] 0 2024-09-21T04:15:27.164 INFO:tasks.workunit.client.0.smithi110.stdout:5/880: dwrite d1/d7/d18/f41 [0,4194304] 0 2024-09-21T04:15:27.165 INFO:tasks.workunit.client.0.smithi110.stdout:8/856: mknod d0/d7/d23/d82/d85/d9b/c136 0 2024-09-21T04:15:27.165 INFO:tasks.workunit.client.1.smithi148.stdout:7/985: write d5/d14/d25/d40/d7b/f10f [347912,54220] 0 2024-09-21T04:15:27.165 INFO:tasks.workunit.client.1.smithi148.stdout:7/986: read - d5/daa/f11e zero size 2024-09-21T04:15:27.165 INFO:tasks.workunit.client.1.smithi148.stdout:0/988: dwrite d5/dcb/de6/d110/d2a/d2b/d99/f123 [0,4194304] 0 2024-09-21T04:15:27.166 INFO:tasks.workunit.client.1.smithi148.stdout:3/956: dread d3/d5/d6/ff3 [4194304,4194304] 0 2024-09-21T04:15:27.167 INFO:tasks.workunit.client.0.smithi110.stdout:2/734: dwrite d2/fbb [0,4194304] 0 2024-09-21T04:15:27.167 INFO:tasks.workunit.client.0.smithi110.stdout:3/740: dwrite d0/d12/d3c/f45 [0,4194304] 0 2024-09-21T04:15:27.167 INFO:tasks.workunit.client.0.smithi110.stdout:3/741: creat d0/d12/d2a/d80/da2/da7/f102 x:0 0 0 2024-09-21T04:15:27.167 INFO:tasks.workunit.client.0.smithi110.stdout:5/881: rename d1/dc/leb to d1/d49/d4d/d1e/d5a/d64/l133 0 2024-09-21T04:15:27.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/935: dwrite d3/d24/fb0 [0,4194304] 0 2024-09-21T04:15:27.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/936: read - d3/d34/d78/f107 zero size 2024-09-21T04:15:27.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/937: chown d3/d34/d78/f124 95 1 2024-09-21T04:15:27.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/938: chown d3/f1b 8112 1 2024-09-21T04:15:27.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/939: creat d3/d14/d1f/d26/d2e/d3d/f13c x:0 0 0 2024-09-21T04:15:27.173 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:26 smithi148 ceph-mon[34957]: pgmap v10: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 282 MiB/s wr, 420 op/s 2024-09-21T04:15:27.196 INFO:tasks.workunit.client.1.smithi148.stdout:8/880: dwrite dc/d13/d6a/fda [0,4194304] 0 2024-09-21T04:15:27.199 INFO:tasks.workunit.client.0.smithi110.stdout:8/857: getdents d0/d30/d34/d3e 0 2024-09-21T04:15:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:0/989: dwrite d5/dcb/de6/d110/d2a/d2b/fac [4194304,4194304] 0 2024-09-21T04:15:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:0/990: fsync d5/dcb/de6/d110/d2a/d3f/d6d/fd9 0 2024-09-21T04:15:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:0/991: write d5/dcb/de6/d110/d2a/d3f/d8b/dab/ffe [65494,93473] 0 2024-09-21T04:15:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:5/940: dwrite d3/d14/d1f/d26/d2e/d56/da7/fd9 [0,4194304] 0 2024-09-21T04:15:27.204 INFO:tasks.workunit.client.0.smithi110.stdout:2/735: mknod d2/de7/ceb 0 2024-09-21T04:15:27.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/987: mknod d5/d8/d64/c163 0 2024-09-21T04:15:27.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/988: read - d5/d8/d6f/d7d/f13f zero size 2024-09-21T04:15:27.213 INFO:tasks.workunit.client.1.smithi148.stdout:3/957: dread d3/d5/d6/d12/d15/d73/f7c [0,4194304] 0 2024-09-21T04:15:27.213 INFO:tasks.workunit.client.1.smithi148.stdout:3/958: dread - d3/d5/d6/d2b/d96/db8/f110 zero size 2024-09-21T04:15:27.216 INFO:tasks.workunit.client.1.smithi148.stdout:8/881: rename dc/d43/d4c/f105 to dc/d13/d3c/d81/f115 0 2024-09-21T04:15:27.225 INFO:tasks.workunit.client.0.smithi110.stdout:3/742: rename d0/d12/d16/f6a to d0/d12/d2a/d80/d9c/f103 0 2024-09-21T04:15:27.225 INFO:tasks.workunit.client.0.smithi110.stdout:3/743: chown d0/d12/d3c/d5a/f64 41 1 2024-09-21T04:15:27.225 INFO:tasks.workunit.client.0.smithi110.stdout:3/744: write d0/d12/d2a/d80/d9c/f103 [2319056,121159] 0 2024-09-21T04:15:27.226 INFO:tasks.workunit.client.0.smithi110.stdout:5/882: link d1/f4c d1/f134 0 2024-09-21T04:15:27.228 INFO:tasks.workunit.client.1.smithi148.stdout:0/992: creat d5/dcb/de6/d110/d2a/d3f/d6d/f13b x:0 0 0 2024-09-21T04:15:27.229 INFO:tasks.workunit.client.1.smithi148.stdout:0/993: write d5/dcb/de6/d110/d2a/d2b/fd1 [4715749,65062] 0 2024-09-21T04:15:27.230 INFO:tasks.workunit.client.0.smithi110.stdout:2/736: symlink d2/d50/lec 0 2024-09-21T04:15:27.231 INFO:tasks.workunit.client.1.smithi148.stdout:5/941: symlink d3/d14/d1f/l13d 0 2024-09-21T04:15:27.231 INFO:tasks.workunit.client.1.smithi148.stdout:5/942: creat d3/d34/d3c/f13e x:0 0 0 2024-09-21T04:15:27.231 INFO:tasks.workunit.client.0.smithi110.stdout:3/745: dread d0/d22/d29/f5f [0,4194304] 0 2024-09-21T04:15:27.232 INFO:tasks.workunit.client.0.smithi110.stdout:3/746: truncate d0/d12/d16/d1a/d5c/ff5 95116 0 2024-09-21T04:15:27.232 INFO:tasks.workunit.client.0.smithi110.stdout:8/858: read d0/d30/d54/fb3 [71508,56508] 0 2024-09-21T04:15:27.232 INFO:tasks.workunit.client.1.smithi148.stdout:7/989: mknod d5/d14/d25/d79/d86/c164 0 2024-09-21T04:15:27.233 INFO:tasks.workunit.client.1.smithi148.stdout:3/959: truncate d3/d1f/d4d/d127/f140 1106786 0 2024-09-21T04:15:27.234 INFO:tasks.workunit.client.1.smithi148.stdout:3/960: dread - d3/d1f/d4d/d82/d21/f63 zero size 2024-09-21T04:15:27.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/882: rename dc/d43/l6c to dc/d13/d76/d50/d80/l116 0 2024-09-21T04:15:27.237 INFO:tasks.workunit.client.0.smithi110.stdout:2/737: creat d2/d1a/d1f/d5f/d9d/d86/dad/fed x:0 0 0 2024-09-21T04:15:27.238 INFO:tasks.workunit.client.0.smithi110.stdout:3/747: stat d0/ld7 0 2024-09-21T04:15:27.239 INFO:tasks.workunit.client.0.smithi110.stdout:3/748: readlink d0/d22/d29/lf3 0 2024-09-21T04:15:27.239 INFO:tasks.workunit.client.1.smithi148.stdout:0/994: mkdir d5/dcb/de6/d110/d2a/dcd/df8/daf/dc8/de8/df7/d13c 0 2024-09-21T04:15:27.239 INFO:tasks.workunit.client.1.smithi148.stdout:0/995: chown d5/dcb/l130 10269341 1 2024-09-21T04:15:27.239 INFO:tasks.workunit.client.1.smithi148.stdout:4/981: sync 2024-09-21T04:15:27.241 INFO:tasks.workunit.client.0.smithi110.stdout:8/859: creat d0/d7/d13/d4b/d108/d12d/f137 x:0 0 0 2024-09-21T04:15:27.241 INFO:tasks.workunit.client.0.smithi110.stdout:8/860: fsync d0/d7/d13/d46/fd2 0 2024-09-21T04:15:27.243 INFO:tasks.workunit.client.1.smithi148.stdout:5/943: creat d3/d34/d78/dbf/d101/f13f x:0 0 0 2024-09-21T04:15:27.243 INFO:tasks.workunit.client.1.smithi148.stdout:5/944: creat d3/d14/d1a/f140 x:0 0 0 2024-09-21T04:15:27.243 INFO:tasks.workunit.client.1.smithi148.stdout:5/945: write d3/d34/d78/dbf/d101/f13f [725912,15712] 0 2024-09-21T04:15:27.245 INFO:tasks.workunit.client.0.smithi110.stdout:2/738: link d2/d1a/d8b/da5/fdf d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/fee 0 2024-09-21T04:15:27.246 INFO:tasks.workunit.client.1.smithi148.stdout:3/961: mknod d3/d5/db4/c146 0 2024-09-21T04:15:27.248 INFO:tasks.workunit.client.0.smithi110.stdout:8/861: rename d0/d7/d6f/d86/cc6 to d0/d30/d34/d3e/d100/c138 0 2024-09-21T04:15:27.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/990: dwrite d5/d8/d6f/d7d/d90/fb5 [0,4194304] 0 2024-09-21T04:15:27.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/991: chown d5/d14/d25/d40/d7b/f119 192073 1 2024-09-21T04:15:27.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/992: chown d5/d8/ff 4569 1 2024-09-21T04:15:27.251 INFO:tasks.workunit.client.1.smithi148.stdout:0/996: symlink d5/dcb/de6/d110/d2a/dcd/df8/db4/l13d 0 2024-09-21T04:15:27.254 INFO:tasks.workunit.client.0.smithi110.stdout:2/739: symlink d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/lef 0 2024-09-21T04:15:27.256 INFO:tasks.workunit.client.1.smithi148.stdout:3/962: rename d3/d1f/d4d/d82/cf7 to d3/d1f/d4d/d82/d21/c147 0 2024-09-21T04:15:27.257 INFO:tasks.workunit.client.0.smithi110.stdout:8/862: mknod d0/d7/d23/d82/d85/d9b/c139 0 2024-09-21T04:15:27.259 INFO:tasks.workunit.client.0.smithi110.stdout:8/863: symlink d0/d7/d89/d10f/l13a 0 2024-09-21T04:15:27.261 INFO:tasks.workunit.client.0.smithi110.stdout:8/864: write d0/d7/d13/d46/df8/faf [657239,78361] 0 2024-09-21T04:15:27.265 INFO:tasks.workunit.client.0.smithi110.stdout:2/740: dread d2/d1a/d1f/d29/f4f [0,4194304] 0 2024-09-21T04:15:27.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/993: mknod d5/d14/d25/d40/d7b/c165 0 2024-09-21T04:15:27.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/994: fsync d5/d8/d6f/d7d/d90/f6e 0 2024-09-21T04:15:27.265 INFO:tasks.workunit.client.1.smithi148.stdout:5/946: read d3/d14/d1f/d26/d2e/d56/da7/fd9 [3263191,6006] 0 2024-09-21T04:15:27.265 INFO:tasks.workunit.client.1.smithi148.stdout:5/947: fsync d3/d14/d1f/d26/d59/d97/f132 0 2024-09-21T04:15:27.267 INFO:tasks.workunit.client.1.smithi148.stdout:0/997: rename d5/dcb/de6/d110/f15 to d5/dcb/de6/d110/f13e 0 2024-09-21T04:15:27.268 INFO:tasks.workunit.client.1.smithi148.stdout:5/948: creat d3/d14/d1f/db1/f141 x:0 0 0 2024-09-21T04:15:27.271 INFO:tasks.workunit.client.1.smithi148.stdout:5/949: creat d3/d14/d1f/d26/f142 x:0 0 0 2024-09-21T04:15:27.271 INFO:tasks.workunit.client.1.smithi148.stdout:0/998: rmdir d5/dcb/de6/d110/d2a/d45 39 2024-09-21T04:15:27.273 INFO:tasks.workunit.client.1.smithi148.stdout:5/950: mknod d3/d24/dad/c143 0 2024-09-21T04:15:27.273 INFO:tasks.workunit.client.1.smithi148.stdout:5/951: fdatasync d3/d34/d3c/d5d/f8e 0 2024-09-21T04:15:27.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/999: creat d5/dcb/de6/d110/d2a/d3f/d8b/f13f x:0 0 0 2024-09-21T04:15:27.275 INFO:tasks.workunit.client.1.smithi148.stdout:5/952: symlink d3/d14/d1f/d26/d59/d97/d102/l144 0 2024-09-21T04:15:27.279 INFO:tasks.workunit.client.0.smithi110.stdout:3/749: dwrite d0/d12/d3c/d5a/d70/f97 [0,4194304] 0 2024-09-21T04:15:27.293 INFO:tasks.workunit.client.1.smithi148.stdout:4/982: dwrite d4/d12/d75/db4/fdf [0,4194304] 0 2024-09-21T04:15:27.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/983: readlink d4/d66/d7f/d2a/da9/lfd 0 2024-09-21T04:15:27.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/984: readlink d4/l7 0 2024-09-21T04:15:27.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/985: chown d4/d12/d16/d29/d61/d6c/d9d 10 1 2024-09-21T04:15:27.294 INFO:tasks.workunit.client.1.smithi148.stdout:4/986: chown d4/d66/d7f/d2a/da9/lfc 101893 1 2024-09-21T04:15:27.294 INFO:tasks.workunit.client.1.smithi148.stdout:8/883: dwrite dc/d3e/fe7 [0,4194304] 0 2024-09-21T04:15:27.297 INFO:tasks.workunit.client.1.smithi148.stdout:4/987: link d4/le0 d4/d12/d16/d29/d61/d6c/df3/l133 0 2024-09-21T04:15:27.301 INFO:tasks.workunit.client.1.smithi148.stdout:8/884: link dc/l4d dc/d13/d76/dd8/l117 0 2024-09-21T04:15:27.301 INFO:tasks.workunit.client.1.smithi148.stdout:8/885: readlink dc/d13/l37 0 2024-09-21T04:15:27.301 INFO:tasks.workunit.client.1.smithi148.stdout:8/886: fdatasync dc/d13/d3c/d81/d98/fa4 0 2024-09-21T04:15:27.303 INFO:tasks.workunit.client.1.smithi148.stdout:4/988: symlink d4/d12/d16/d29/d61/d6c/df1/d11f/l134 0 2024-09-21T04:15:27.304 INFO:tasks.workunit.client.1.smithi148.stdout:8/887: creat dc/d43/f118 x:0 0 0 2024-09-21T04:15:27.304 INFO:tasks.workunit.client.1.smithi148.stdout:8/888: readlink dc/d13/d3c/d81/db0/ldb 0 2024-09-21T04:15:27.306 INFO:tasks.workunit.client.1.smithi148.stdout:8/889: mkdir dc/d13/d76/d119 0 2024-09-21T04:15:27.306 INFO:tasks.workunit.client.1.smithi148.stdout:8/890: readlink dc/d13/d3c/d81/db0/lc9 0 2024-09-21T04:15:27.306 INFO:tasks.workunit.client.1.smithi148.stdout:8/891: chown dc/d3e 16018570 1 2024-09-21T04:15:27.310 INFO:tasks.workunit.client.1.smithi148.stdout:7/995: dwrite d5/d8/d6f/d7d/d90/d28/f87 [0,4194304] 0 2024-09-21T04:15:27.311 INFO:tasks.workunit.client.0.smithi110.stdout:2/741: dwrite d2/d1a/d1f/d65/fa0 [0,4194304] 0 2024-09-21T04:15:27.311 INFO:tasks.workunit.client.1.smithi148.stdout:7/996: mkdir d5/d8/d6f/d7d/d90/d9b/d166 0 2024-09-21T04:15:27.312 INFO:tasks.workunit.client.1.smithi148.stdout:4/989: write d4/d3a/f53 [151111,109831] 0 2024-09-21T04:15:27.312 INFO:tasks.workunit.client.1.smithi148.stdout:4/990: fdatasync d4/d12/d16/d29/dbd/f68 0 2024-09-21T04:15:27.312 INFO:tasks.workunit.client.1.smithi148.stdout:4/991: dread - d4/d3a/f12a zero size 2024-09-21T04:15:27.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/963: dread d3/d1f/d4d/d127/f140 [0,4194304] 0 2024-09-21T04:15:27.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/964: readlink d3/d5/d6/d2b/da2/da3/de4/d117/l118 0 2024-09-21T04:15:27.313 INFO:tasks.workunit.client.1.smithi148.stdout:9/986: sync 2024-09-21T04:15:27.314 INFO:tasks.workunit.client.0.smithi110.stdout:2/742: truncate d2/d1a/d1f/d5f/d9d/fbf 1761045 0 2024-09-21T04:15:27.314 INFO:tasks.workunit.client.0.smithi110.stdout:2/743: chown d2/d1a/d1f/d29/l6d 0 1 2024-09-21T04:15:27.315 INFO:tasks.workunit.client.0.smithi110.stdout:3/750: dwrite d0/fcb [0,4194304] 0 2024-09-21T04:15:27.315 INFO:tasks.workunit.client.0.smithi110.stdout:3/751: truncate d0/d12/d3c/dce/fe1 1466431 0 2024-09-21T04:15:27.315 INFO:tasks.workunit.client.0.smithi110.stdout:3/752: fdatasync d0/d51/d7b/ffc 0 2024-09-21T04:15:27.316 INFO:tasks.workunit.client.1.smithi148.stdout:5/953: dwrite d3/d14/d1f/d26/d2e/d3d/f67 [0,4194304] 0 2024-09-21T04:15:27.316 INFO:tasks.workunit.client.1.smithi148.stdout:5/954: creat d3/d14/d1f/d26/d2e/d3d/da0/f145 x:0 0 0 2024-09-21T04:15:27.316 INFO:tasks.workunit.client.1.smithi148.stdout:7/997: creat d5/d8/d6f/d7d/d90/d13c/f167 x:0 0 0 2024-09-21T04:15:27.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/883: dwrite d1/d22/d51/fa5 [0,4194304] 0 2024-09-21T04:15:27.317 INFO:tasks.workunit.client.1.smithi148.stdout:3/965: dread d3/d1f/f3e [0,4194304] 0 2024-09-21T04:15:27.319 INFO:tasks.workunit.client.0.smithi110.stdout:3/753: write d0/d12/d3c/d5a/d70/f84 [8287,7512] 0 2024-09-21T04:15:27.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/987: symlink d5/d2b/d4e/de4/l159 0 2024-09-21T04:15:27.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/988: chown d5/d2b/d4e/le0 146493 1 2024-09-21T04:15:27.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/989: read - d5/d60/d28/d73/fcd zero size 2024-09-21T04:15:27.326 INFO:tasks.workunit.client.1.smithi148.stdout:5/955: rename d3/d4a/cce to d3/d14/d1f/d26/d2e/ddb/deb/d105/d116/c146 0 2024-09-21T04:15:27.327 INFO:tasks.workunit.client.1.smithi148.stdout:5/956: write d3/d14/d1f/d26/d2e/d56/da7/fb3 [973552,49950] 0 2024-09-21T04:15:27.327 INFO:tasks.workunit.client.0.smithi110.stdout:6/822: sync 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:2/744: getdents d2/d1a/d8b/da5/da9 0 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:2/745: stat d2/d1a/d8b 0 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:7/774: sync 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:9/769: sync 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:7/775: creat dd/db1/ffe x:0 0 0 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:7/776: chown dd/d48/fb7 347444621 1 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:9/770: write d7/d96/d99/d3c/d35/f75 [1652801,37961] 0 2024-09-21T04:15:27.329 INFO:tasks.workunit.client.0.smithi110.stdout:7/777: chown dd/d1c/dad/db2/d56/d66/dea 2442 1 2024-09-21T04:15:27.330 INFO:tasks.workunit.client.1.smithi148.stdout:7/998: mknod d5/d8/d6f/d7d/df5/d149/d14d/c168 0 2024-09-21T04:15:27.331 INFO:tasks.workunit.client.0.smithi110.stdout:3/754: rename d0/d12/d16/d1a/d6f/f73 to d0/d12/d16/d1a/d95/f104 0 2024-09-21T04:15:27.336 INFO:tasks.workunit.client.0.smithi110.stdout:9/771: mkdir d7/d96/d99/d3c/d35/d109 0 2024-09-21T04:15:27.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/957: read d3/d34/d78/dbf/d6f/f10c [1507157,109751] 0 2024-09-21T04:15:27.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/958: creat d3/d34/d3c/d5d/d75/f147 x:0 0 0 2024-09-21T04:15:27.343 INFO:tasks.workunit.client.0.smithi110.stdout:3/755: unlink d0/d12/d16/d1a/d95/lcd 0 2024-09-21T04:15:27.344 INFO:tasks.workunit.client.0.smithi110.stdout:6/823: dread d4/d12/d17/d81/f108 [0,4194304] 0 2024-09-21T04:15:27.345 INFO:tasks.workunit.client.0.smithi110.stdout:6/824: chown d4/d12/d17/d33/l43 42831747 1 2024-09-21T04:15:27.345 INFO:tasks.workunit.client.0.smithi110.stdout:9/772: mknod d7/d96/d99/d3c/d14/d31/c10a 0 2024-09-21T04:15:27.346 INFO:tasks.workunit.client.0.smithi110.stdout:7/778: rmdir dd/d1c/d89 39 2024-09-21T04:15:27.347 INFO:tasks.workunit.client.1.smithi148.stdout:4/992: dwrite d4/d66/d7f/d9a/dce/d12f/d103/f123 [0,4194304] 0 2024-09-21T04:15:27.347 INFO:tasks.workunit.client.0.smithi110.stdout:5/884: dwrite d1/d7/d18/ff8 [4194304,4194304] 0 2024-09-21T04:15:27.347 INFO:tasks.workunit.client.1.smithi148.stdout:9/990: dwrite d5/d1e/f49 [0,4194304] 0 2024-09-21T04:15:27.348 INFO:tasks.workunit.client.0.smithi110.stdout:2/746: write d2/d1a/d1f/d5f/f58 [3391366,81469] 0 2024-09-21T04:15:27.348 INFO:tasks.workunit.client.0.smithi110.stdout:2/747: chown d2/d1a/d1f/d5f/fe9 371 1 2024-09-21T04:15:27.348 INFO:tasks.workunit.client.0.smithi110.stdout:2/748: creat d2/d1a/d1f/d2a/ff0 x:0 0 0 2024-09-21T04:15:27.348 INFO:tasks.workunit.client.0.smithi110.stdout:3/756: mknod d0/d51/c105 0 2024-09-21T04:15:27.352 INFO:tasks.workunit.client.0.smithi110.stdout:6/825: rename d4/d12/d16/d66/d7a/d83/f9f to d4/d12/d17/d33/f10b 0 2024-09-21T04:15:27.352 INFO:tasks.workunit.client.0.smithi110.stdout:6/826: dread - d4/d12/d17/d81/d90/fd9 zero size 2024-09-21T04:15:27.354 INFO:tasks.workunit.client.0.smithi110.stdout:9/773: creat d7/d10/d88/daa/f10b x:0 0 0 2024-09-21T04:15:27.354 INFO:tasks.workunit.client.1.smithi148.stdout:5/959: dread d3/d14/d1f/d26/d2e/ddb/dde/f106 [0,4194304] 0 2024-09-21T04:15:27.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/999: dwrite d5/d8/d64/f137 [0,4194304] 0 2024-09-21T04:15:27.364 INFO:tasks.workunit.client.0.smithi110.stdout:2/749: rename d2/d1a/d1f/d5f/d9d/cd3 to d2/d1a/d1f/d65/cf1 0 2024-09-21T04:15:27.364 INFO:tasks.workunit.client.0.smithi110.stdout:2/750: write d2/d81/fcb [870058,100056] 0 2024-09-21T04:15:27.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/827: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/d10c 0 2024-09-21T04:15:27.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/828: fsync d4/d12/d17/d2e/fb9 0 2024-09-21T04:15:27.367 INFO:tasks.workunit.client.0.smithi110.stdout:9/774: mkdir d7/d26/d10c 0 2024-09-21T04:15:27.368 INFO:tasks.workunit.client.0.smithi110.stdout:9/775: dread - d7/d10/fc0 zero size 2024-09-21T04:15:27.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/960: getdents d3/d24/dad 0 2024-09-21T04:15:27.368 INFO:tasks.workunit.client.0.smithi110.stdout:9/776: write d7/d10/d88/d8f/fd1 [68152,79419] 0 2024-09-21T04:15:27.370 INFO:tasks.workunit.client.0.smithi110.stdout:7/779: dwrite dd/d1c/dad/db2/d56/fac [0,4194304] 0 2024-09-21T04:15:27.370 INFO:tasks.workunit.client.1.smithi148.stdout:9/991: dwrite d5/d2b/d32/d103/d123/f150 [0,4194304] 0 2024-09-21T04:15:27.372 INFO:tasks.workunit.client.0.smithi110.stdout:3/757: rename d0/d12/d2a/d80/da2/da7/fe9 to d0/d12/d16/d1a/f106 0 2024-09-21T04:15:27.372 INFO:tasks.workunit.client.0.smithi110.stdout:3/758: write d0/d12/d3c/f4f [2477100,72547] 0 2024-09-21T04:15:27.372 INFO:tasks.workunit.client.0.smithi110.stdout:2/751: dread d2/d1a/d1f/d29/d19/fda [0,4194304] 0 2024-09-21T04:15:27.373 INFO:tasks.workunit.client.1.smithi148.stdout:3/966: rename d3/d1f/d4d/d82 to d3/d5/dac/d148 0 2024-09-21T04:15:27.375 INFO:tasks.workunit.client.1.smithi148.stdout:5/961: symlink d3/d14/d1f/d26/d2e/ddb/deb/d105/d116/l148 0 2024-09-21T04:15:27.376 INFO:tasks.workunit.client.0.smithi110.stdout:9/777: stat d7/d96/d99/d3c/d11/c15 0 2024-09-21T04:15:27.380 INFO:tasks.workunit.client.0.smithi110.stdout:3/759: dread d0/d12/d2a/d80/d9c/f103 [0,4194304] 0 2024-09-21T04:15:27.382 INFO:tasks.workunit.client.0.smithi110.stdout:2/752: dread d2/d1a/d1f/d5f/f58 [0,4194304] 0 2024-09-21T04:15:27.384 INFO:tasks.workunit.client.1.smithi148.stdout:9/992: mknod d5/d1e/c15a 0 2024-09-21T04:15:27.384 INFO:tasks.workunit.client.1.smithi148.stdout:9/993: stat d5/d1f/d81/f8c 0 2024-09-21T04:15:27.386 INFO:tasks.workunit.client.0.smithi110.stdout:6/829: rename d4/d12/d17/d2e/d40/f2f to d4/d12/d17/d9a/dbe/f10d 0 2024-09-21T04:15:27.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/993: rename d4/d12/d16/d29/d61/f8e to d4/d66/d7f/f135 0 2024-09-21T04:15:27.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/994: fdatasync d4/d12/d16/df0/dec/f10f 0 2024-09-21T04:15:27.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/995: chown d4/d12/d78/de4 5998 1 2024-09-21T04:15:27.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/996: chown d4/d12/d16/d29/d61/d6c/d9d/fb0 1064688 1 2024-09-21T04:15:27.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/967: creat d3/d5/dac/d148/d21/f149 x:0 0 0 2024-09-21T04:15:27.391 INFO:tasks.workunit.client.1.smithi148.stdout:3/968: write d3/d5/d5c/d126/f121 [447416,40636] 0 2024-09-21T04:15:27.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/997: write d4/d87/fba [1230,83494] 0 2024-09-21T04:15:27.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/998: write d4/d12/d16/df0/dec/f10f [137050,121207] 0 2024-09-21T04:15:27.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/999: dread - d4/d3a/f12a zero size 2024-09-21T04:15:27.391 INFO:tasks.workunit.client.0.smithi110.stdout:3/760: creat d0/d12/d2a/d80/da2/db0/f107 x:0 0 0 2024-09-21T04:15:27.393 INFO:tasks.workunit.client.1.smithi148.stdout:5/962: creat d3/d14/f149 x:0 0 0 2024-09-21T04:15:27.394 INFO:tasks.workunit.client.1.smithi148.stdout:9/994: read d5/d60/f3c [2108887,73731] 0 2024-09-21T04:15:27.396 INFO:tasks.workunit.client.0.smithi110.stdout:7/780: rename dd/f31 to dd/d14/d17/d8a/dd2/fff 0 2024-09-21T04:15:27.400 INFO:tasks.workunit.client.0.smithi110.stdout:7/781: symlink dd/d4d/dc1/l100 0 2024-09-21T04:15:27.401 INFO:tasks.workunit.client.1.smithi148.stdout:5/963: symlink d3/d14/d1f/d26/d2e/d56/l14a 0 2024-09-21T04:15:27.402 INFO:tasks.workunit.client.1.smithi148.stdout:3/969: dread d3/d5/d6/d12/d15/d73/f7c [0,4194304] 0 2024-09-21T04:15:27.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/782: rmdir dd/d1c/de8 0 2024-09-21T04:15:27.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/783: getdents dd/d1c/dad/db2/d56/d66/dea 0 2024-09-21T04:15:27.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/784: truncate dd/d14/ffa 156334 0 2024-09-21T04:15:27.405 INFO:tasks.workunit.client.1.smithi148.stdout:5/964: link d3/d4a/l8b d3/d14/d1f/d26/d2e/l14b 0 2024-09-21T04:15:27.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/970: dread d3/d5/d6/d12/f72 [0,4194304] 0 2024-09-21T04:15:27.406 INFO:tasks.workunit.client.0.smithi110.stdout:9/778: dwrite d7/d96/d99/d3c/d35/d71/ffc [0,4194304] 0 2024-09-21T04:15:27.406 INFO:tasks.workunit.client.0.smithi110.stdout:9/779: dread - d7/d96/d99/d3c/d35/fa7 zero size 2024-09-21T04:15:27.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/965: creat d3/f14c x:0 0 0 2024-09-21T04:15:27.406 INFO:tasks.workunit.client.1.smithi148.stdout:5/966: readlink d3/d14/d1f/db1/l100 0 2024-09-21T04:15:27.407 INFO:tasks.workunit.client.0.smithi110.stdout:9/780: creat d7/d96/d99/d3c/d1f/f10d x:0 0 0 2024-09-21T04:15:27.407 INFO:tasks.workunit.client.0.smithi110.stdout:9/781: truncate d7/d96/d99/d3c/d1f/f23 5001170 0 2024-09-21T04:15:27.408 INFO:tasks.workunit.client.1.smithi148.stdout:5/967: mknod d3/d24/dad/c14d 0 2024-09-21T04:15:27.408 INFO:tasks.workunit.client.1.smithi148.stdout:5/968: chown d3/d34/d3c/d5d/f95 2951446 1 2024-09-21T04:15:27.408 INFO:tasks.workunit.client.0.smithi110.stdout:9/782: link d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/le0 d7/d96/d99/d3c/d4f/d9f/d6f/l10e 0 2024-09-21T04:15:27.408 INFO:tasks.workunit.client.0.smithi110.stdout:9/783: chown d7/d96/d99/d3c/d1f 0 1 2024-09-21T04:15:27.409 INFO:tasks.workunit.client.0.smithi110.stdout:9/784: write d7/d96/d99/ff9 [419589,23877] 0 2024-09-21T04:15:27.409 INFO:tasks.workunit.client.0.smithi110.stdout:9/785: dread - d7/d10/fd8 zero size 2024-09-21T04:15:27.409 INFO:tasks.workunit.client.0.smithi110.stdout:9/786: rename d7/d96/d99/d3c/d4f/d9f/d6f/d78/d7d/f8a to d7/d10/d88/d8f/db7/f10f 0 2024-09-21T04:15:27.410 INFO:tasks.workunit.client.0.smithi110.stdout:9/787: truncate d7/d96/d99/d3c/d11/fc6 9671 0 2024-09-21T04:15:27.411 INFO:tasks.workunit.client.0.smithi110.stdout:9/788: truncate d7/d96/d99/d3c/d1f/d43/db5/ffe 556278 0 2024-09-21T04:15:27.412 INFO:tasks.workunit.client.0.smithi110.stdout:9/789: symlink d7/d10/l110 0 2024-09-21T04:15:27.413 INFO:tasks.workunit.client.0.smithi110.stdout:9/790: rename f2 to d7/d10/d88/f111 0 2024-09-21T04:15:27.413 INFO:tasks.workunit.client.0.smithi110.stdout:9/791: creat d7/d96/d99/d3c/d4f/f112 x:0 0 0 2024-09-21T04:15:27.414 INFO:tasks.workunit.client.0.smithi110.stdout:9/792: symlink d7/d96/d99/d3c/d4f/d9f/d6f/l113 0 2024-09-21T04:15:27.414 INFO:tasks.workunit.client.0.smithi110.stdout:9/793: fsync d7/d96/d99/d3c/d1f/d43/f6e 0 2024-09-21T04:15:27.417 INFO:tasks.workunit.client.0.smithi110.stdout:7/785: dread dd/f7e [0,4194304] 0 2024-09-21T04:15:27.417 INFO:tasks.workunit.client.0.smithi110.stdout:7/786: fdatasync dd/d1c/d89/fe2 0 2024-09-21T04:15:27.418 INFO:tasks.workunit.client.0.smithi110.stdout:7/787: fsync dd/f1a 0 2024-09-21T04:15:27.418 INFO:tasks.workunit.client.0.smithi110.stdout:7/788: link dd/d14/d17/l1f dd/d4d/dee/dbc/l101 0 2024-09-21T04:15:27.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/789: mkdir dd/d4d/dc1/d102 0 2024-09-21T04:15:27.419 INFO:tasks.workunit.client.1.smithi148.stdout:5/969: dread d3/d4a/fae [0,4194304] 0 2024-09-21T04:15:27.419 INFO:tasks.workunit.client.1.smithi148.stdout:5/970: write d3/d34/ff7 [981691,56408] 0 2024-09-21T04:15:27.420 INFO:tasks.workunit.client.0.smithi110.stdout:2/753: dwrite d2/d1a/d1f/d5f/d9d/d86/dad/fed [0,4194304] 0 2024-09-21T04:15:27.420 INFO:tasks.workunit.client.0.smithi110.stdout:9/794: dread d7/d10/fde [0,4194304] 0 2024-09-21T04:15:27.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/971: link d3/d88/lf1 d3/d34/l14e 0 2024-09-21T04:15:27.429 INFO:tasks.workunit.client.0.smithi110.stdout:3/761: dwrite d0/d12/d3c/dce/fdc [0,4194304] 0 2024-09-21T04:15:27.429 INFO:tasks.workunit.client.1.smithi148.stdout:3/971: dwrite d3/d5/db4/dfb/f136 [0,4194304] 0 2024-09-21T04:15:27.435 INFO:tasks.workunit.client.1.smithi148.stdout:3/972: fsync d3/d5/fe7 0 2024-09-21T04:15:27.436 INFO:tasks.workunit.client.0.smithi110.stdout:3/762: creat d0/d12/d3c/d5a/f108 x:0 0 0 2024-09-21T04:15:27.437 INFO:tasks.workunit.client.0.smithi110.stdout:2/754: dread d2/d1a/d1f/d5f/d9d/d86/dad/fed [0,4194304] 0 2024-09-21T04:15:27.437 INFO:tasks.workunit.client.0.smithi110.stdout:2/755: dread - d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/fee zero size 2024-09-21T04:15:27.439 INFO:tasks.workunit.client.1.smithi148.stdout:3/973: mkdir d3/d5/d6/d2b/da2/da3/de4/d117/d14a 0 2024-09-21T04:15:27.439 INFO:tasks.workunit.client.1.smithi148.stdout:3/974: chown d3/d5/d6/d2b/d96/l9e 12966628 1 2024-09-21T04:15:27.439 INFO:tasks.workunit.client.1.smithi148.stdout:3/975: truncate d3/d5/dac/d148/d21/f63 471422 0 2024-09-21T04:15:27.440 INFO:tasks.workunit.client.1.smithi148.stdout:3/976: write d3/d5/d5c/d126/f121 [377808,5174] 0 2024-09-21T04:15:27.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/972: dwrite d3/d14/d1a/fee [0,4194304] 0 2024-09-21T04:15:27.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/973: chown d3/d34/d3c/d5d 36284167 1 2024-09-21T04:15:27.440 INFO:tasks.workunit.client.1.smithi148.stdout:5/974: chown d3/d14/d1f/d26/d2e/d3d/da0/f134 238 1 2024-09-21T04:15:27.441 INFO:tasks.workunit.client.0.smithi110.stdout:3/763: creat d0/d12/d3c/d5a/d65/df0/f109 x:0 0 0 2024-09-21T04:15:27.444 INFO:tasks.workunit.client.1.smithi148.stdout:8/892: sync 2024-09-21T04:15:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/975: dread d3/d14/d1f/d26/d2e/d56/fc3 [0,4194304] 0 2024-09-21T04:15:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/976: creat d3/d34/f14f x:0 0 0 2024-09-21T04:15:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/977: creat d3/d14/d1f/d72/dc9/f150 x:0 0 0 2024-09-21T04:15:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/978: write d3/d14/d1f/d26/d2e/d3d/f138 [201768,11726] 0 2024-09-21T04:15:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/979: fdatasync d3/d14/d1f/d26/d2e/f10a 0 2024-09-21T04:15:27.447 INFO:tasks.workunit.client.0.smithi110.stdout:7/790: dwrite dd/f7e [0,4194304] 0 2024-09-21T04:15:27.452 INFO:tasks.workunit.client.1.smithi148.stdout:5/980: link d3/d88/lf1 d3/l151 0 2024-09-21T04:15:27.452 INFO:tasks.workunit.client.1.smithi148.stdout:5/981: dread - d3/d14/d1f/f123 zero size 2024-09-21T04:15:27.452 INFO:tasks.workunit.client.1.smithi148.stdout:5/982: fsync d3/d34/f11d 0 2024-09-21T04:15:27.452 INFO:tasks.workunit.client.1.smithi148.stdout:5/983: fsync d3/f8 0 2024-09-21T04:15:27.453 INFO:tasks.workunit.client.1.smithi148.stdout:5/984: creat d3/d34/d78/f152 x:0 0 0 2024-09-21T04:15:27.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/791: rename dd/d1c/dad/db2/d4a/l4e to dd/d4d/dc1/d102/l103 0 2024-09-21T04:15:27.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/792: write dd/d4d/f5e [1849175,28314] 0 2024-09-21T04:15:27.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/793: fsync dd/d1c/dad/db2/d4a/d52/f60 0 2024-09-21T04:15:27.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/893: write dc/d43/f47 [1501563,47516] 0 2024-09-21T04:15:27.457 INFO:tasks.workunit.client.0.smithi110.stdout:9/795: write d7/d96/d99/f2a [108688,6951] 0 2024-09-21T04:15:27.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/985: unlink d3/d14/d1f/ff0 0 2024-09-21T04:15:27.462 INFO:tasks.workunit.client.0.smithi110.stdout:7/794: symlink dd/d1c/dad/db2/d4a/l104 0 2024-09-21T04:15:27.463 INFO:tasks.workunit.client.1.smithi148.stdout:8/894: creat dc/d13/d76/f11a x:0 0 0 2024-09-21T04:15:27.464 INFO:tasks.workunit.client.0.smithi110.stdout:2/756: dwrite d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/fee [0,4194304] 0 2024-09-21T04:15:27.464 INFO:tasks.workunit.client.1.smithi148.stdout:3/977: dwrite d3/d5/dac/d148/d23/db5/d105/f130 [0,4194304] 0 2024-09-21T04:15:27.468 INFO:tasks.workunit.client.0.smithi110.stdout:9/796: symlink d7/d96/d99/d3c/d4f/l114 0 2024-09-21T04:15:27.469 INFO:tasks.workunit.client.1.smithi148.stdout:8/895: mkdir dc/d13/d3c/d81/d98/de5/d11b 0 2024-09-21T04:15:27.470 INFO:tasks.workunit.client.0.smithi110.stdout:7/795: getdents dd/d4d/dee 0 2024-09-21T04:15:27.476 INFO:tasks.workunit.client.0.smithi110.stdout:7/796: unlink dd/d14/ffa 0 2024-09-21T04:15:27.478 INFO:tasks.workunit.client.0.smithi110.stdout:7/797: symlink dd/d4d/l105 0 2024-09-21T04:15:27.480 INFO:tasks.workunit.client.0.smithi110.stdout:7/798: write dd/f3e [1207956,53500] 0 2024-09-21T04:15:27.480 INFO:tasks.workunit.client.0.smithi110.stdout:7/799: creat dd/d4d/dee/dbc/f106 x:0 0 0 2024-09-21T04:15:27.480 INFO:tasks.workunit.client.0.smithi110.stdout:7/800: chown dd/d1c/dad/db2/d4a/d52/f58 188 1 2024-09-21T04:15:27.482 INFO:tasks.workunit.client.1.smithi148.stdout:5/986: rename d3/d14/d1f/d26/d2e/d56 to d3/d153 0 2024-09-21T04:15:27.483 INFO:tasks.workunit.client.0.smithi110.stdout:7/801: rmdir dd/d4d/dee/ddc 39 2024-09-21T04:15:27.485 INFO:tasks.workunit.client.0.smithi110.stdout:7/802: mknod dd/d14/d17/d8a/dd2/c107 0 2024-09-21T04:15:27.486 INFO:tasks.workunit.client.1.smithi148.stdout:8/896: getdents dc/d13/d76/d50/df0 0 2024-09-21T04:15:27.487 INFO:tasks.workunit.client.1.smithi148.stdout:5/987: mkdir d3/d14/d1f/d26/d59/d97/d154 0 2024-09-21T04:15:27.488 INFO:tasks.workunit.client.0.smithi110.stdout:7/803: symlink dd/d4d/l108 0 2024-09-21T04:15:27.488 INFO:tasks.workunit.client.0.smithi110.stdout:4/990: sync 2024-09-21T04:15:27.489 INFO:tasks.workunit.client.0.smithi110.stdout:8/865: sync 2024-09-21T04:15:27.489 INFO:tasks.workunit.client.0.smithi110.stdout:4/991: fdatasync d17/d1e/d77/d10c/d20/d43/d13b/f12c 0 2024-09-21T04:15:27.489 INFO:tasks.workunit.client.0.smithi110.stdout:8/866: read d0/d30/d34/d3e/d32/da3/fa9 [651622,90571] 0 2024-09-21T04:15:27.489 INFO:tasks.workunit.client.0.smithi110.stdout:8/867: dread - d0/d7/d13/f10d zero size 2024-09-21T04:15:27.491 INFO:tasks.workunit.client.1.smithi148.stdout:8/897: link dc/d13/d3c/d81/d85/da1/fca dc/d43/d4c/d4f/dc5/f11c 0 2024-09-21T04:15:27.491 INFO:tasks.workunit.client.1.smithi148.stdout:8/898: stat dc/d13/d39/d6e/fd7 0 2024-09-21T04:15:27.491 INFO:tasks.workunit.client.1.smithi148.stdout:8/899: readlink dc/l27 0 2024-09-21T04:15:27.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/804: rename dd/d1c/dad/db2/d56/d66/da6/f8e to dd/d14/d17/f109 0 2024-09-21T04:15:27.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/805: creat dd/d1c/dad/db2/d4a/f10a x:0 0 0 2024-09-21T04:15:27.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/988: symlink d3/d153/l155 0 2024-09-21T04:15:27.493 INFO:tasks.workunit.client.0.smithi110.stdout:4/992: symlink d17/d34/l150 0 2024-09-21T04:15:27.493 INFO:tasks.workunit.client.0.smithi110.stdout:4/993: chown d17/d1e/d77/d10c/d20/d43/d51/d7a/l142 31733070 1 2024-09-21T04:15:27.493 INFO:tasks.workunit.client.0.smithi110.stdout:4/994: chown d17/d1e/d77/d10c/d20/d43/d51/d52/f59 5426340 1 2024-09-21T04:15:27.495 INFO:tasks.workunit.client.0.smithi110.stdout:8/868: rename d0/d7/c19 to d0/d30/d34/d3e/d32/da3/d117/c13b 0 2024-09-21T04:15:27.496 INFO:tasks.workunit.client.1.smithi148.stdout:5/989: creat d3/d153/da7/f156 x:0 0 0 2024-09-21T04:15:27.497 INFO:tasks.workunit.client.0.smithi110.stdout:7/806: mkdir dd/d10b 0 2024-09-21T04:15:27.498 INFO:tasks.workunit.client.0.smithi110.stdout:2/757: dwrite d2/d1a/d1f/d65/fe8 [0,4194304] 0 2024-09-21T04:15:27.498 INFO:tasks.workunit.client.1.smithi148.stdout:5/990: symlink d3/d37/l157 0 2024-09-21T04:15:27.501 INFO:tasks.workunit.client.1.smithi148.stdout:8/900: write f5 [2332294,22418] 0 2024-09-21T04:15:27.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/995: dread d17/d1e/d77/d10c/d20/f88 [0,4194304] 0 2024-09-21T04:15:27.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/996: write d17/d1e/d77/d10c/d20/d43/fa2 [1018457,81330] 0 2024-09-21T04:15:27.502 INFO:tasks.workunit.client.0.smithi110.stdout:8/869: unlink d0/d7/d23/d52/l123 0 2024-09-21T04:15:27.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/797: dwrite d7/d96/d99/d3c/d11/f8c [0,4194304] 0 2024-09-21T04:15:27.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/798: write d7/d96/d99/d3c/d1f/f10d [721014,37278] 0 2024-09-21T04:15:27.503 INFO:tasks.workunit.client.0.smithi110.stdout:7/807: mknod dd/d14/d17/d8a/c10c 0 2024-09-21T04:15:27.510 INFO:tasks.workunit.client.0.smithi110.stdout:2/758: symlink d2/de7/lf2 0 2024-09-21T04:15:27.511 INFO:tasks.workunit.client.0.smithi110.stdout:8/870: write d0/d7/d23/d6c/d12e/f104 [2582465,68807] 0 2024-09-21T04:15:27.511 INFO:tasks.workunit.client.0.smithi110.stdout:7/808: mkdir dd/d1c/dad/d10d 0 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:8/871: symlink d0/d30/d105/l13c 0 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:9/799: dread d7/d96/d99/d3c/d11/f4e [0,4194304] 0 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:9/800: truncate d7/d96/d99/d3c/d14/fc2 992741 0 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:9/801: truncate d7/d96/d99/f38 897683 0 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:9/802: chown d7/d10/fc0 63884 1 2024-09-21T04:15:27.518 INFO:tasks.workunit.client.0.smithi110.stdout:9/803: stat d7/d96/d99/d3c/d1f/d43/f6c 0 2024-09-21T04:15:27.519 INFO:tasks.workunit.client.0.smithi110.stdout:7/809: mknod dd/d1c/dad/db2/d4a/da5/c10e 0 2024-09-21T04:15:27.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/872: creat d0/d7/d23/dc1/f13d x:0 0 0 2024-09-21T04:15:27.523 INFO:tasks.workunit.client.0.smithi110.stdout:8/873: write d0/d30/d34/f12f [278532,67129] 0 2024-09-21T04:15:27.524 INFO:tasks.workunit.client.0.smithi110.stdout:4/997: dwrite d17/d1e/d77/d10c/d20/d43/fe2 [0,4194304] 0 2024-09-21T04:15:27.524 INFO:tasks.workunit.client.0.smithi110.stdout:4/998: truncate d17/d1e/d77/dcf/d12b/f134 746158 0 2024-09-21T04:15:27.528 INFO:tasks.workunit.client.1.smithi148.stdout:5/991: dwrite d3/d34/d3c/f13a [0,4194304] 0 2024-09-21T04:15:27.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/901: rename dc/d13/d6a to dc/d13/d3c/d81/db0/db7/d11d 0 2024-09-21T04:15:27.529 INFO:tasks.workunit.client.0.smithi110.stdout:9/804: dread d7/d10/d88/d8f/db7/f10f [0,4194304] 0 2024-09-21T04:15:27.531 INFO:tasks.workunit.client.1.smithi148.stdout:8/902: dread dc/d13/d76/d50/db1/fec [0,4194304] 0 2024-09-21T04:15:27.531 INFO:tasks.workunit.client.1.smithi148.stdout:8/903: stat dc/d43/f54 0 2024-09-21T04:15:27.541 INFO:tasks.workunit.client.0.smithi110.stdout:8/874: write d0/d7/d13/d46/f4c [669994,110858] 0 2024-09-21T04:15:27.542 INFO:tasks.workunit.client.0.smithi110.stdout:2/759: dwrite d2/d1a/d1f/d29/d19/fda [0,4194304] 0 2024-09-21T04:15:27.542 INFO:tasks.workunit.client.0.smithi110.stdout:8/875: readlink d0/d7/lc 0 2024-09-21T04:15:27.542 INFO:tasks.workunit.client.0.smithi110.stdout:2/760: chown d2/d1a/d1f/d5f/d9d/d86/d9f/dc3/lef 6388 1 2024-09-21T04:15:27.542 INFO:tasks.workunit.client.0.smithi110.stdout:4/999: creat d17/d1e/d77/d10c/d20/d43/f151 x:0 0 0 2024-09-21T04:15:27.543 INFO:tasks.workunit.client.0.smithi110.stdout:9/805: link d7/d96/d99/d3c/d4f/d9f/d6f/l10e d7/d96/l115 0 2024-09-21T04:15:27.544 INFO:tasks.workunit.client.1.smithi148.stdout:8/904: write dc/d13/d3c/d81/d85/da1/fb8 [3701692,120231] 0 2024-09-21T04:15:27.544 INFO:tasks.workunit.client.1.smithi148.stdout:8/905: truncate dc/d13/d28/d62/fb6 1118417 0 2024-09-21T04:15:27.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/810: dwrite dd/d14/f41 [0,4194304] 0 2024-09-21T04:15:27.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/811: fsync dd/d14/f2a 0 2024-09-21T04:15:27.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/812: truncate dd/d1c/dad/db2/d56/d66/fe1 635472 0 2024-09-21T04:15:27.546 INFO:tasks.workunit.client.0.smithi110.stdout:2/761: symlink d2/d1a/lf3 0 2024-09-21T04:15:27.546 INFO:tasks.workunit.client.0.smithi110.stdout:2/762: stat d2/d1a/da3/db9/ce1 0 2024-09-21T04:15:27.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/906: link dc/d13/d76/c3d dc/d13/d3c/d81/c11e 0 2024-09-21T04:15:27.552 INFO:tasks.workunit.client.0.smithi110.stdout:7/813: creat dd/d1c/f10f x:0 0 0 2024-09-21T04:15:27.559 INFO:tasks.workunit.client.1.smithi148.stdout:8/907: write dc/d13/d3c/d81/d98/fb9 [965122,84335] 0 2024-09-21T04:15:27.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/992: dwrite d3/d14/d1a/fe5 [0,4194304] 0 2024-09-21T04:15:27.561 INFO:tasks.workunit.client.0.smithi110.stdout:7/814: mknod dd/d1c/dad/db2/d56/d66/da6/da7/c110 0 2024-09-21T04:15:27.561 INFO:tasks.workunit.client.0.smithi110.stdout:7/815: chown dd/d4d/dc1/fca 60 1 2024-09-21T04:15:27.561 INFO:tasks.workunit.client.0.smithi110.stdout:2/763: dread d2/d1a/d8b/da5/fdf [0,4194304] 0 2024-09-21T04:15:27.561 INFO:tasks.workunit.client.0.smithi110.stdout:2/764: write d2/d50/d56/f7a [703203,6400] 0 2024-09-21T04:15:27.561 INFO:tasks.workunit.client.0.smithi110.stdout:2/765: truncate d2/d1a/d1f/d29/f48 2295374 0 2024-09-21T04:15:27.563 INFO:tasks.workunit.client.0.smithi110.stdout:7/816: mkdir dd/d1c/dad/db2/d4a/da5/d111 0 2024-09-21T04:15:27.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/817: mknod dd/d1c/dad/db2/d56/c112 0 2024-09-21T04:15:27.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/818: dread - dd/d4d/dc1/ff9 zero size 2024-09-21T04:15:27.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/819: chown dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/fbb 2 1 2024-09-21T04:15:27.568 INFO:tasks.workunit.client.1.smithi148.stdout:5/993: link d3/d14/d1f/d72/dc9/dd8/cf2 d3/d14/d1f/d26/d2e/ddb/deb/d105/d116/c158 0 2024-09-21T04:15:27.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/820: truncate dd/d14/d17/f109 893829 0 2024-09-21T04:15:27.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/821: creat dd/d1c/f113 x:0 0 0 2024-09-21T04:15:27.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/822: mknod dd/d10b/c114 0 2024-09-21T04:15:27.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/823: chown dd/d10b/c114 21982932 1 2024-09-21T04:15:27.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/824: truncate dd/fae 365659 0 2024-09-21T04:15:27.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/825: getdents dd/d1c/dad/db2/d56/d66/dea 0 2024-09-21T04:15:27.571 INFO:tasks.workunit.client.0.smithi110.stdout:7/826: truncate dd/d4d/dee/dbc/fc2 682529 0 2024-09-21T04:15:27.571 INFO:tasks.workunit.client.0.smithi110.stdout:7/827: fsync dd/d48/fc8 0 2024-09-21T04:15:27.571 INFO:tasks.workunit.client.0.smithi110.stdout:7/828: mkdir dd/d1c/dad/db2/d4a/d115 0 2024-09-21T04:15:27.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/908: read dc/d13/d76/dd8/ff5 [2469592,61192] 0 2024-09-21T04:15:27.575 INFO:tasks.workunit.client.1.smithi148.stdout:8/909: symlink dc/d13/d76/l11f 0 2024-09-21T04:15:27.576 INFO:tasks.workunit.client.1.smithi148.stdout:8/910: truncate dc/d13/f34 954564 0 2024-09-21T04:15:27.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/911: mkdir dc/d13/d3c/d81/d120 0 2024-09-21T04:15:27.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/912: symlink dc/de6/l121 0 2024-09-21T04:15:27.581 INFO:tasks.workunit.client.1.smithi148.stdout:8/913: mkdir dc/d13/d39/d6e/d122 0 2024-09-21T04:15:27.583 INFO:tasks.workunit.client.0.smithi110.stdout:8/876: dwrite d0/f3d [0,4194304] 0 2024-09-21T04:15:27.586 INFO:tasks.workunit.client.0.smithi110.stdout:8/877: mknod d0/d7/d13/d4b/ddc/c13e 0 2024-09-21T04:15:27.586 INFO:tasks.workunit.client.0.smithi110.stdout:8/878: write d0/d30/f31 [714605,15193] 0 2024-09-21T04:15:27.599 INFO:tasks.workunit.client.0.smithi110.stdout:2/766: dwrite d2/d1a/d1f/d65/fa0 [0,4194304] 0 2024-09-21T04:15:27.600 INFO:tasks.workunit.client.0.smithi110.stdout:2/767: mkdir d2/d1a/d1f/d5f/d9d/d86/d9f/df4 0 2024-09-21T04:15:27.603 INFO:tasks.workunit.client.1.smithi148.stdout:5/994: dwrite d3/d4a/fae [0,4194304] 0 2024-09-21T04:15:27.603 INFO:tasks.workunit.client.1.smithi148.stdout:5/995: chown d3/d14/d1f/d72 650525 1 2024-09-21T04:15:27.603 INFO:tasks.workunit.client.0.smithi110.stdout:2/768: symlink d2/d1a/d1f/d2a/lf5 0 2024-09-21T04:15:27.603 INFO:tasks.workunit.client.0.smithi110.stdout:2/769: write d2/d1a/d1f/d2a/f61 [522627,36895] 0 2024-09-21T04:15:27.603 INFO:tasks.workunit.client.0.smithi110.stdout:9/806: dwrite d7/d96/d99/d3c/d11/f8b [0,4194304] 0 2024-09-21T04:15:27.606 INFO:tasks.workunit.client.0.smithi110.stdout:9/807: link d7/d10/d88/d8f/la6 d7/d10/l116 0 2024-09-21T04:15:27.608 INFO:tasks.workunit.client.0.smithi110.stdout:7/829: dwrite dd/d14/fcb [0,4194304] 0 2024-09-21T04:15:27.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/808: symlink d7/d96/d99/d3c/d4f/l117 0 2024-09-21T04:15:27.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/809: fsync d7/d96/d99/d3c/d11/fc6 0 2024-09-21T04:15:27.610 INFO:tasks.workunit.client.0.smithi110.stdout:9/810: chown d7/d96/d99/d3c/f5a 38916623 1 2024-09-21T04:15:27.616 INFO:tasks.workunit.client.1.smithi148.stdout:5/996: link d3/d24/dad/c143 d3/d34/d3c/d5d/d75/c159 0 2024-09-21T04:15:27.617 INFO:tasks.workunit.client.0.smithi110.stdout:7/830: dread dd/d14/f41 [0,4194304] 0 2024-09-21T04:15:27.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/879: dwrite d0/d30/d34/d3e/d5d/f9f [0,4194304] 0 2024-09-21T04:15:27.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/880: dread - d0/d7/d13/dc5/ffe zero size 2024-09-21T04:15:27.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/881: truncate d0/d7/d13/d46/df8/fe5 1056145 0 2024-09-21T04:15:27.618 INFO:tasks.workunit.client.0.smithi110.stdout:8/882: chown d0/d7/d23/dc1/f13d 7470202 1 2024-09-21T04:15:27.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/997: rename d3/d14/d1a/l35 to d3/d34/d3c/d5d/d75/d121/l15a 0 2024-09-21T04:15:27.623 INFO:tasks.workunit.client.0.smithi110.stdout:9/811: unlink d7/d96/d99/d3c/d35/f76 0 2024-09-21T04:15:27.623 INFO:tasks.workunit.client.0.smithi110.stdout:9/812: read - d7/d96/d99/d3c/d14/ff0 zero size 2024-09-21T04:15:27.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/998: creat d3/d153/da7/f15b x:0 0 0 2024-09-21T04:15:27.625 INFO:tasks.workunit.client.0.smithi110.stdout:8/883: read d0/d7/d13/d4b/f4e [2038172,7628] 0 2024-09-21T04:15:27.625 INFO:tasks.workunit.client.0.smithi110.stdout:8/884: fsync d0/d30/d54/d6b/f75 0 2024-09-21T04:15:27.626 INFO:tasks.workunit.client.1.smithi148.stdout:5/999: rename d3/f83 to d3/d34/d3c/d5d/d75/d121/f15c 0 2024-09-21T04:15:27.630 INFO:tasks.workunit.client.0.smithi110.stdout:9/813: mkdir d7/d96/d99/d3c/d4f/d9f/d6f/d108/d118 0 2024-09-21T04:15:27.631 INFO:tasks.workunit.client.0.smithi110.stdout:7/831: getdents dd/d48/d9f 0 2024-09-21T04:15:27.632 INFO:tasks.workunit.client.0.smithi110.stdout:7/832: creat dd/db1/f116 x:0 0 0 2024-09-21T04:15:27.632 INFO:tasks.workunit.client.0.smithi110.stdout:7/833: truncate dd/d4d/dc1/ff9 603586 0 2024-09-21T04:15:27.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/885: symlink d0/d7/d13/dc5/l13f 0 2024-09-21T04:15:27.633 INFO:tasks.workunit.client.0.smithi110.stdout:7/834: dread dd/d14/d17/d8a/f8b [0,4194304] 0 2024-09-21T04:15:27.634 INFO:tasks.workunit.client.0.smithi110.stdout:9/814: rename d7/d96/d99/d3c/dae/lce to d7/d96/d99/d3c/d1f/d43/db5/dbf/dd0/l119 0 2024-09-21T04:15:27.636 INFO:tasks.workunit.client.0.smithi110.stdout:8/886: getdents d0/d30/d34/d3e/d32/da3/d117 0 2024-09-21T04:15:27.637 INFO:tasks.workunit.client.0.smithi110.stdout:7/835: rename dd/d1c/dad/db2/d4a/da5/ff6 to dd/d1c/dad/db2/d56/d66/da6/f117 0 2024-09-21T04:15:27.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/815: symlink d7/d96/d99/d3c/d4f/d9f/d6f/d78/l11a 0 2024-09-21T04:15:27.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/816: dread - d7/d10/fd8 zero size 2024-09-21T04:15:27.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/817: read d7/d96/d99/f2a [113827,25858] 0 2024-09-21T04:15:27.640 INFO:tasks.workunit.client.0.smithi110.stdout:9/818: truncate d7/d26/f7e 1188497 0 2024-09-21T04:15:27.640 INFO:tasks.workunit.client.0.smithi110.stdout:8/887: creat d0/d7/d13/d1c/f140 x:0 0 0 2024-09-21T04:15:27.641 INFO:tasks.workunit.client.0.smithi110.stdout:7/836: getdents dd/d1c/d89 0 2024-09-21T04:15:27.641 INFO:tasks.workunit.client.0.smithi110.stdout:7/837: chown dd/d14/c42 25 1 2024-09-21T04:15:27.641 INFO:tasks.workunit.client.0.smithi110.stdout:7/838: write dd/d14/d17/fc3 [2158714,111656] 0 2024-09-21T04:15:27.642 INFO:tasks.workunit.client.0.smithi110.stdout:9/819: link d7/d96/d99/l2b d7/d96/d99/d3c/d35/l11b 0 2024-09-21T04:15:27.642 INFO:tasks.workunit.client.0.smithi110.stdout:8/888: readlink d0/d30/d34/d3e/d60/ldf 0 2024-09-21T04:15:27.643 INFO:tasks.workunit.client.0.smithi110.stdout:8/889: creat d0/d30/d34/d3e/d5d/dd7/f141 x:0 0 0 2024-09-21T04:15:27.643 INFO:tasks.workunit.client.0.smithi110.stdout:8/890: fsync d0/d30/d34/d3e/d32/da3/fa9 0 2024-09-21T04:15:27.643 INFO:tasks.workunit.client.0.smithi110.stdout:9/820: truncate d7/d96/d99/d3c/d35/d71/ffc 4507965 0 2024-09-21T04:15:27.645 INFO:tasks.workunit.client.0.smithi110.stdout:8/891: creat d0/d30/d34/d3e/d5d/dd7/dda/f142 x:0 0 0 2024-09-21T04:15:27.645 INFO:tasks.workunit.client.0.smithi110.stdout:8/892: creat d0/d30/d34/d3e/d5d/f143 x:0 0 0 2024-09-21T04:15:27.645 INFO:tasks.workunit.client.0.smithi110.stdout:8/893: truncate d0/d30/d34/d3e/d5d/dd7/ff6 265096 0 2024-09-21T04:15:27.646 INFO:tasks.workunit.client.0.smithi110.stdout:9/821: getdents d7/d96/d99/d3c/d1f/d68 0 2024-09-21T04:15:27.646 INFO:tasks.workunit.client.1.smithi148.stdout:3/978: sync 2024-09-21T04:15:27.647 INFO:tasks.workunit.client.1.smithi148.stdout:9/995: sync 2024-09-21T04:15:27.648 INFO:tasks.workunit.client.1.smithi148.stdout:9/996: rmdir d5/d1f/d81/dcf 39 2024-09-21T04:15:27.649 INFO:tasks.workunit.client.0.smithi110.stdout:8/894: rmdir d0/d30/d54/d6b/da8 39 2024-09-21T04:15:27.649 INFO:tasks.workunit.client.0.smithi110.stdout:8/895: write d0/d7/d13/d46/f4c [1644894,14051] 0 2024-09-21T04:15:27.650 INFO:tasks.workunit.client.1.smithi148.stdout:3/979: rmdir d3/d5/d6/d12/d15/d13d 0 2024-09-21T04:15:27.651 INFO:tasks.workunit.client.1.smithi148.stdout:3/980: fsync d3/d5/d6/d12/d43/fbb 0 2024-09-21T04:15:27.652 INFO:tasks.workunit.client.1.smithi148.stdout:9/997: write d5/d2b/f5d [812435,120924] 0 2024-09-21T04:15:27.652 INFO:tasks.workunit.client.0.smithi110.stdout:5/885: sync 2024-09-21T04:15:27.653 INFO:tasks.workunit.client.0.smithi110.stdout:8/896: mkdir d0/d7/d13/d1c/d144 0 2024-09-21T04:15:27.655 INFO:tasks.workunit.client.0.smithi110.stdout:9/822: write d7/d96/d99/d3c/d14/d31/f6a [2923901,85782] 0 2024-09-21T04:15:27.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/981: link d3/d84/la9 d3/d5/dac/d148/d23/d28/d46/l14b 0 2024-09-21T04:15:27.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/982: write d3/d1f/f8e [2017514,64689] 0 2024-09-21T04:15:27.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/983: write d3/d5/d6/d2b/d96/db8/df4/f112 [1024176,50999] 0 2024-09-21T04:15:27.657 INFO:tasks.workunit.client.0.smithi110.stdout:6/830: sync 2024-09-21T04:15:27.657 INFO:tasks.workunit.client.0.smithi110.stdout:3/764: sync 2024-09-21T04:15:27.657 INFO:tasks.workunit.client.0.smithi110.stdout:3/765: write d0/d12/d16/f62 [2030557,121536] 0 2024-09-21T04:15:27.658 INFO:tasks.workunit.client.0.smithi110.stdout:3/766: fdatasync d0/d22/dd2/ff6 0 2024-09-21T04:15:27.658 INFO:tasks.workunit.client.0.smithi110.stdout:5/886: truncate d1/d49/d4d/d1e/f3b 1382855 0 2024-09-21T04:15:27.658 INFO:tasks.workunit.client.0.smithi110.stdout:5/887: stat d1/d49/d4d/d1e/d4a/l59 0 2024-09-21T04:15:27.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/839: dwrite dd/d4d/f68 [0,4194304] 0 2024-09-21T04:15:27.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/840: chown dd/d1c/d89 105 1 2024-09-21T04:15:27.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/841: truncate dd/d1c/dad/db2/f3c 5022624 0 2024-09-21T04:15:27.662 INFO:tasks.workunit.client.0.smithi110.stdout:6/831: mknod d4/d12/d17/d9a/dba/de6/c10e 0 2024-09-21T04:15:27.665 INFO:tasks.workunit.client.0.smithi110.stdout:3/767: getdents d0/d51 0 2024-09-21T04:15:27.666 INFO:tasks.workunit.client.1.smithi148.stdout:3/984: dread d3/d5/d6/d12/d15/d73/f7c [0,4194304] 0 2024-09-21T04:15:27.667 INFO:tasks.workunit.client.0.smithi110.stdout:7/842: truncate dd/f7e 3095004 0 2024-09-21T04:15:27.667 INFO:tasks.workunit.client.0.smithi110.stdout:7/843: stat dd/d1c/dad/db2/d56/d66/da6/d7a/c6e 0 2024-09-21T04:15:27.670 INFO:tasks.workunit.client.1.smithi148.stdout:3/985: mknod d3/d5/d6/d2b/c14c 0 2024-09-21T04:15:27.671 INFO:tasks.workunit.client.0.smithi110.stdout:3/768: mknod d0/d12/d3c/d5a/c10a 0 2024-09-21T04:15:27.676 INFO:tasks.workunit.client.1.smithi148.stdout:3/986: mkdir d3/d5/d6/d12/d15/da7/d14d 0 2024-09-21T04:15:27.676 INFO:tasks.workunit.client.1.smithi148.stdout:3/987: chown d3/d5/d6/d12/d43/d54/c79 82325515 1 2024-09-21T04:15:27.676 INFO:tasks.workunit.client.1.smithi148.stdout:3/988: readlink d3/d1f/l5d 0 2024-09-21T04:15:27.676 INFO:tasks.workunit.client.1.smithi148.stdout:3/989: chown d3/d5/dac/d148/d52/l2a 8078 1 2024-09-21T04:15:27.677 INFO:tasks.workunit.client.0.smithi110.stdout:3/769: getdents d0/d12/d2a/d80/da2/db0 0 2024-09-21T04:15:27.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/990: rename d3/d5/dac/d148/d52/f59 to d3/d5/db4/d113/f14e 0 2024-09-21T04:15:27.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/991: stat d3/d5/d6/ff3 0 2024-09-21T04:15:27.681 INFO:tasks.workunit.client.1.smithi148.stdout:9/998: dwrite d5/d2b/d4e/db3/fd7 [0,4194304] 0 2024-09-21T04:15:27.684 INFO:tasks.workunit.client.1.smithi148.stdout:3/992: dread d3/d5/d5c/fee [0,4194304] 0 2024-09-21T04:15:27.712 INFO:tasks.workunit.client.1.smithi148.stdout:3/993: write d3/fc0 [4029207,8925] 0 2024-09-21T04:15:27.713 INFO:tasks.workunit.client.1.smithi148.stdout:3/994: link d3/d5/dac/d148/d21/c147 d3/d5/dac/d148/d23/c14f 0 2024-09-21T04:15:27.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/995: symlink d3/d5/l150 0 2024-09-21T04:15:27.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/996: mknod d3/d5/d6/d2b/d96/db8/c151 0 2024-09-21T04:15:27.715 INFO:tasks.workunit.client.1.smithi148.stdout:3/997: link d3/d5/d6/d12/d43/f11f d3/d5/dac/d148/d23/db5/f152 0 2024-09-21T04:15:27.716 INFO:tasks.workunit.client.1.smithi148.stdout:3/998: write d3/d5/d5c/fc4 [1203666,102162] 0 2024-09-21T04:15:27.716 INFO:tasks.workunit.client.1.smithi148.stdout:3/999: write d3/d5/d6/d2b/da2/da3/fb2 [703842,99853] 0 2024-09-21T04:15:27.720 INFO:tasks.workunit.client.0.smithi110.stdout:5/888: dwrite d1/d7/d58/d72/fbe [0,4194304] 0 2024-09-21T04:15:27.720 INFO:tasks.workunit.client.0.smithi110.stdout:8/897: dwrite d0/d30/d34/d3e/d32/f7c [0,4194304] 0 2024-09-21T04:15:27.721 INFO:tasks.workunit.client.1.smithi148.stdout:9/999: dwrite d5/d2b/f91 [0,4194304] 0 2024-09-21T04:15:27.721 INFO:tasks.workunit.client.0.smithi110.stdout:9/823: dwrite d7/d26/f7e [0,4194304] 0 2024-09-21T04:15:27.722 INFO:tasks.workunit.client.0.smithi110.stdout:7/844: dwrite dd/d4d/dee/ddc/fec [0,4194304] 0 2024-09-21T04:15:27.722 INFO:tasks.workunit.client.0.smithi110.stdout:6/832: dwrite d4/d12/d17/d33/d76/fa6 [0,4194304] 0 2024-09-21T04:15:27.722 INFO:tasks.workunit.client.0.smithi110.stdout:3/770: dwrite d0/d22/dd2/ff6 [0,4194304] 0 2024-09-21T04:15:27.722 INFO:tasks.workunit.client.0.smithi110.stdout:7/845: write dd/d48/fc8 [874631,22816] 0 2024-09-21T04:15:27.727 INFO:tasks.workunit.client.0.smithi110.stdout:5/889: dread d1/d7/d18/d3f/f86 [0,4194304] 0 2024-09-21T04:15:27.727 INFO:tasks.workunit.client.0.smithi110.stdout:5/890: truncate d1/d39/f119 301187 0 2024-09-21T04:15:27.728 INFO:tasks.workunit.client.0.smithi110.stdout:8/898: unlink d0/d7/d6f/f93 0 2024-09-21T04:15:27.729 INFO:tasks.workunit.client.0.smithi110.stdout:8/899: write d0/d7/d13/d46/df8/f12c [669331,87027] 0 2024-09-21T04:15:27.734 INFO:tasks.workunit.client.0.smithi110.stdout:6/833: dwrite d4/d12/f1d [4194304,4194304] 0 2024-09-21T04:15:27.735 INFO:tasks.workunit.client.0.smithi110.stdout:6/834: dread d4/d12/d16/d66/fac [0,4194304] 0 2024-09-21T04:15:27.736 INFO:tasks.workunit.client.0.smithi110.stdout:6/835: dread d4/d12/d16/d66/fac [0,4194304] 0 2024-09-21T04:15:27.736 INFO:tasks.workunit.client.0.smithi110.stdout:6/836: fsync d4/d78/daa/f84 0 2024-09-21T04:15:27.738 INFO:tasks.workunit.client.0.smithi110.stdout:9/824: mknod d7/d96/d99/d3c/d4f/c11c 0 2024-09-21T04:15:27.742 INFO:tasks.workunit.client.0.smithi110.stdout:8/900: dwrite d0/d7/d23/d6c/d12e/f104 [0,4194304] 0 2024-09-21T04:15:27.742 INFO:tasks.workunit.client.0.smithi110.stdout:8/901: chown d0/d7/d23/d82/d85 15 1 2024-09-21T04:15:27.746 INFO:tasks.workunit.client.0.smithi110.stdout:3/771: mknod d0/d12/d3c/d5a/d70/dab/c10b 0 2024-09-21T04:15:27.747 INFO:tasks.workunit.client.0.smithi110.stdout:7/846: mknod dd/d1c/d89/c118 0 2024-09-21T04:15:27.747 INFO:tasks.workunit.client.0.smithi110.stdout:7/847: fsync dd/d1c/dad/db2/d4a/d52/f60 0 2024-09-21T04:15:27.749 INFO:tasks.workunit.client.0.smithi110.stdout:5/891: mkdir d1/d49/d4d/d1e/d4a/dd4/dd5/d10e/d135 0 2024-09-21T04:15:27.750 INFO:tasks.workunit.client.0.smithi110.stdout:6/837: mknod d4/d12/d17/d2e/d40/dad/c10f 0 2024-09-21T04:15:27.752 INFO:tasks.workunit.client.0.smithi110.stdout:9/825: creat d7/d96/d99/d3c/d4f/d9f/d6f/d108/d118/f11d x:0 0 0 2024-09-21T04:15:27.752 INFO:tasks.workunit.client.0.smithi110.stdout:9/826: write d7/d96/d99/d3c/d4f/d9f/d6f/fe8 [5015385,113523] 0 2024-09-21T04:15:27.753 INFO:tasks.workunit.client.0.smithi110.stdout:8/902: creat d0/d30/d34/d3e/d32/da3/d117/f145 x:0 0 0 2024-09-21T04:15:27.754 INFO:tasks.workunit.client.0.smithi110.stdout:7/848: mkdir dd/d1c/dad/db2/d4a/d52/d119 0 2024-09-21T04:15:27.756 INFO:tasks.workunit.client.0.smithi110.stdout:6/838: mkdir d4/d12/d16/d66/d7a/d4b/d110 0 2024-09-21T04:15:27.758 INFO:tasks.workunit.client.0.smithi110.stdout:9/827: unlink d7/d96/d99/d3c/d1f/d68/ce9 0 2024-09-21T04:15:27.758 INFO:tasks.workunit.client.0.smithi110.stdout:9/828: write d7/d96/d99/d3c/d11/f2e [5134204,22699] 0 2024-09-21T04:15:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/903: creat d0/d30/d34/d3e/d100/f146 x:0 0 0 2024-09-21T04:15:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/904: fsync d0/d7/d13/d4b/da7/fac 0 2024-09-21T04:15:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:6/839: creat d4/d12/d17/d33/dab/f111 x:0 0 0 2024-09-21T04:15:27.761 INFO:tasks.workunit.client.0.smithi110.stdout:6/840: creat d4/d3c/f112 x:0 0 0 2024-09-21T04:15:27.766 INFO:tasks.workunit.client.0.smithi110.stdout:7/849: dread dd/d14/d17/f3a [0,4194304] 0 2024-09-21T04:15:27.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/914: sync 2024-09-21T04:15:27.768 INFO:tasks.workunit.client.0.smithi110.stdout:7/850: rename dd/d1c/dad/db2/d4a/l104 to dd/d14/d17/d8a/dd2/l11a 0 2024-09-21T04:15:27.768 INFO:tasks.workunit.client.0.smithi110.stdout:7/851: fdatasync dd/d4d/dee/dbc/f106 0 2024-09-21T04:15:27.769 INFO:tasks.workunit.client.0.smithi110.stdout:7/852: symlink dd/d1c/dad/db2/d56/d66/da6/d7a/l11b 0 2024-09-21T04:15:27.770 INFO:tasks.workunit.client.0.smithi110.stdout:7/853: mkdir dd/d1c/dad/db2/d56/d66/da6/da7/d11c 0 2024-09-21T04:15:27.772 INFO:tasks.workunit.client.0.smithi110.stdout:7/854: unlink dd/d14/d17/d8a/c10c 0 2024-09-21T04:15:27.772 INFO:tasks.workunit.client.0.smithi110.stdout:9/829: write d7/d96/d99/d3c/d1f/d43/fad [3448258,70600] 0 2024-09-21T04:15:27.772 INFO:tasks.workunit.client.0.smithi110.stdout:9/830: write d7/d96/d99/d3c/d1f/d43/db5/dca/fea [507916,84503] 0 2024-09-21T04:15:27.772 INFO:tasks.workunit.client.0.smithi110.stdout:9/831: creat d7/d96/d99/d3c/d14/f11e x:0 0 0 2024-09-21T04:15:27.773 INFO:tasks.workunit.client.0.smithi110.stdout:9/832: mkdir d7/d96/d99/d3c/d4f/d9f/d6f/d11f 0 2024-09-21T04:15:27.774 INFO:tasks.workunit.client.0.smithi110.stdout:3/772: dwrite d0/d12/d16/d1a/f106 [0,4194304] 0 2024-09-21T04:15:27.774 INFO:tasks.workunit.client.0.smithi110.stdout:9/833: mkdir d7/d96/d99/d3c/d1f/d43/db5/dca/d120 0 2024-09-21T04:15:27.775 INFO:tasks.workunit.client.0.smithi110.stdout:9/834: write d7/d96/d99/d3c/d1f/d68/f6b [4415242,119451] 0 2024-09-21T04:15:27.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/773: symlink d0/d12/d3c/d5a/d65/df0/l10c 0 2024-09-21T04:15:27.777 INFO:tasks.workunit.client.0.smithi110.stdout:3/774: write d0/fe [1213180,108663] 0 2024-09-21T04:15:27.778 INFO:tasks.workunit.client.0.smithi110.stdout:7/855: dread dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:27.778 INFO:tasks.workunit.client.0.smithi110.stdout:6/841: write d4/d12/d16/d66/d7a/d4b/fa1 [1746699,3658] 0 2024-09-21T04:15:27.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/775: creat d0/d12/d16/d1a/d95/f10d x:0 0 0 2024-09-21T04:15:27.783 INFO:tasks.workunit.client.0.smithi110.stdout:3/776: write d0/d12/d2a/f3e [1776442,55450] 0 2024-09-21T04:15:27.784 INFO:tasks.workunit.client.0.smithi110.stdout:3/777: symlink d0/d12/d3c/dad/l10e 0 2024-09-21T04:15:27.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/778: unlink d0/d12/d16/d1a/dc6/lf8 0 2024-09-21T04:15:27.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/779: fsync d0/d22/f46 0 2024-09-21T04:15:27.786 INFO:tasks.workunit.client.0.smithi110.stdout:5/892: dwrite d1/d39/da2/fde [0,4194304] 0 2024-09-21T04:15:27.787 INFO:tasks.workunit.client.1.smithi148.stdout:8/915: dwrite dc/d13/d3c/f88 [0,4194304] 0 2024-09-21T04:15:27.787 INFO:tasks.workunit.client.0.smithi110.stdout:3/780: dread d0/d12/fd5 [0,4194304] 0 2024-09-21T04:15:27.787 INFO:tasks.workunit.client.0.smithi110.stdout:3/781: chown d0/d12/d3c/ff4 8 1 2024-09-21T04:15:27.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/856: dread dd/d1c/dad/db2/f4f [0,4194304] 0 2024-09-21T04:15:27.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/857: readlink dd/d14/d17/l1f 0 2024-09-21T04:15:27.794 INFO:tasks.workunit.client.0.smithi110.stdout:6/842: dwrite d4/d12/d17/d81/fcc [0,4194304] 0 2024-09-21T04:15:27.796 INFO:tasks.workunit.client.0.smithi110.stdout:7/858: dread dd/f3e [0,4194304] 0 2024-09-21T04:15:27.796 INFO:tasks.workunit.client.0.smithi110.stdout:7/859: stat dd/d48/fb7 0 2024-09-21T04:15:27.796 INFO:tasks.workunit.client.0.smithi110.stdout:6/843: mkdir d4/d12/d16/d66/d7a/d4b/d7f/d113 0 2024-09-21T04:15:27.797 INFO:tasks.workunit.client.0.smithi110.stdout:7/860: dread dd/d4d/dee/fd3 [0,4194304] 0 2024-09-21T04:15:27.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/844: dread d4/d12/d17/d2e/d40/fd8 [0,4194304] 0 2024-09-21T04:15:27.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/845: write d4/d78/daa/fcd [438345,99159] 0 2024-09-21T04:15:27.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/846: write d4/d12/d16/f50 [2762781,37772] 0 2024-09-21T04:15:27.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/847: dread d4/d12/d17/d2e/d40/fc9 [0,4194304] 0 2024-09-21T04:15:27.799 INFO:tasks.workunit.client.0.smithi110.stdout:6/848: truncate d4/d12/d17/d2e/d38/f98 952374 0 2024-09-21T04:15:27.799 INFO:tasks.workunit.client.0.smithi110.stdout:6/849: rmdir d4/d12/d17/d2e/d38 39 2024-09-21T04:15:27.799 INFO:tasks.workunit.client.1.smithi148.stdout:8/916: read dc/d13/d39/f66 [1868671,41050] 0 2024-09-21T04:15:27.800 INFO:tasks.workunit.client.0.smithi110.stdout:5/893: dread d1/f3e [0,4194304] 0 2024-09-21T04:15:27.801 INFO:tasks.workunit.client.0.smithi110.stdout:6/850: rmdir d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/d10c 0 2024-09-21T04:15:27.802 INFO:tasks.workunit.client.0.smithi110.stdout:3/782: write d0/d22/d29/fc5 [2484202,37580] 0 2024-09-21T04:15:27.802 INFO:tasks.workunit.client.0.smithi110.stdout:3/783: stat d0/d12/d16/d1a/f3b 0 2024-09-21T04:15:27.806 INFO:tasks.workunit.client.1.smithi148.stdout:8/917: mknod dc/d13/d28/d62/c123 0 2024-09-21T04:15:27.806 INFO:tasks.workunit.client.1.smithi148.stdout:8/918: truncate dc/d13/d39/f66 3110188 0 2024-09-21T04:15:27.806 INFO:tasks.workunit.client.1.smithi148.stdout:8/919: truncate dc/d13/d76/d5d/d78/f9f 4436420 0 2024-09-21T04:15:27.810 INFO:tasks.workunit.client.0.smithi110.stdout:5/894: symlink d1/d22/d51/l136 0 2024-09-21T04:15:27.810 INFO:tasks.workunit.client.0.smithi110.stdout:6/851: mknod d4/d3c/db0/c114 0 2024-09-21T04:15:27.810 INFO:tasks.workunit.client.0.smithi110.stdout:6/852: truncate d4/d86/d8f/fc7 1582600 0 2024-09-21T04:15:27.811 INFO:tasks.workunit.client.1.smithi148.stdout:8/920: symlink dc/d13/d3c/d81/d9d/l124 0 2024-09-21T04:15:27.811 INFO:tasks.workunit.client.0.smithi110.stdout:9/835: rmdir d7/d96/d99/d3c/d1f 39 2024-09-21T04:15:27.815 INFO:tasks.workunit.client.1.smithi148.stdout:8/921: rmdir dc/d13/d3c/d81/d85/da1 39 2024-09-21T04:15:27.815 INFO:tasks.workunit.client.1.smithi148.stdout:8/922: write dc/d3e/fe7 [4235490,11133] 0 2024-09-21T04:15:27.820 INFO:tasks.workunit.client.1.smithi148.stdout:8/923: getdents dc/d13/d76/d42 0 2024-09-21T04:15:27.821 INFO:tasks.workunit.client.1.smithi148.stdout:8/924: mknod dc/d13/d39/d6e/dac/c125 0 2024-09-21T04:15:27.822 INFO:tasks.workunit.client.1.smithi148.stdout:8/925: truncate dc/d13/d76/d50/faa 390217 0 2024-09-21T04:15:27.824 INFO:tasks.workunit.client.0.smithi110.stdout:3/784: dwrite d0/d12/d3c/d5a/d70/f97 [0,4194304] 0 2024-09-21T04:15:27.824 INFO:tasks.workunit.client.0.smithi110.stdout:7/861: dwrite dd/d48/f4b [0,4194304] 0 2024-09-21T04:15:27.826 INFO:tasks.workunit.client.0.smithi110.stdout:5/895: mkdir d1/d11f/d137 0 2024-09-21T04:15:27.827 INFO:tasks.workunit.client.0.smithi110.stdout:6/853: symlink d4/d12/d17/d2e/d40/dad/l115 0 2024-09-21T04:15:27.828 INFO:tasks.workunit.client.0.smithi110.stdout:9/836: link d7/d96/d99/d3c/d1f/d43/db5/dbf/cf8 d7/d96/d99/d3c/d1f/d43/db5/dca/d120/c121 0 2024-09-21T04:15:27.828 INFO:tasks.workunit.client.0.smithi110.stdout:9/837: creat d7/d26/f122 x:0 0 0 2024-09-21T04:15:27.828 INFO:tasks.workunit.client.0.smithi110.stdout:9/838: chown d7/d96/d99/d3c/d1f/d43/db5/dbf/dd0 20627 1 2024-09-21T04:15:27.831 INFO:tasks.workunit.client.1.smithi148.stdout:8/926: read dc/d13/d76/d42/f90 [2091955,66080] 0 2024-09-21T04:15:27.831 INFO:tasks.workunit.client.1.smithi148.stdout:8/927: truncate dc/d13/d76/d50/fff 456127 0 2024-09-21T04:15:27.832 INFO:tasks.workunit.client.1.smithi148.stdout:8/928: readlink dc/d13/d3c/d81/d85/d104/l10a 0 2024-09-21T04:15:27.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/862: unlink dd/d48/fb7 0 2024-09-21T04:15:27.832 INFO:tasks.workunit.client.0.smithi110.stdout:7/863: readlink dd/d1c/dad/db2/d4a/l79 0 2024-09-21T04:15:27.832 INFO:tasks.workunit.client.0.smithi110.stdout:5/896: truncate d1/d7/f2b 388506 0 2024-09-21T04:15:27.836 INFO:tasks.workunit.client.0.smithi110.stdout:9/839: mknod d7/d96/c123 0 2024-09-21T04:15:27.837 INFO:tasks.workunit.client.0.smithi110.stdout:9/840: truncate d7/d96/d99/d3c/d1f/d43/f92 521746 0 2024-09-21T04:15:27.837 INFO:tasks.workunit.client.0.smithi110.stdout:5/897: read d1/f3e [3008583,106039] 0 2024-09-21T04:15:27.837 INFO:tasks.workunit.client.0.smithi110.stdout:3/785: getdents d0/d12/d3c/dad 0 2024-09-21T04:15:27.842 INFO:tasks.workunit.client.0.smithi110.stdout:9/841: link d7/d96/d99/d3c/d14/ff0 d7/d26/f124 0 2024-09-21T04:15:27.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/898: getdents d1/d49/d4d/d1e/d4a/dd4/dd5/d10e 0 2024-09-21T04:15:27.844 INFO:tasks.workunit.client.0.smithi110.stdout:9/842: dread d7/d96/d99/d3c/d1f/d72/f63 [0,4194304] 0 2024-09-21T04:15:27.846 INFO:tasks.workunit.client.0.smithi110.stdout:6/854: rename d4/d78/daa to d4/d12/d17/d2e/dfb/d116 0 2024-09-21T04:15:27.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/786: dread d0/d12/d16/d1a/dc6/fe0 [0,4194304] 0 2024-09-21T04:15:27.847 INFO:tasks.workunit.client.0.smithi110.stdout:6/855: dread d4/d86/d8f/fc7 [0,4194304] 0 2024-09-21T04:15:27.848 INFO:tasks.workunit.client.0.smithi110.stdout:9/843: unlink d7/d96/f9a 0 2024-09-21T04:15:27.848 INFO:tasks.workunit.client.0.smithi110.stdout:9/844: read - d7/d10/d88/d8f/da0/fe6 zero size 2024-09-21T04:15:27.850 INFO:tasks.workunit.client.0.smithi110.stdout:3/787: mkdir d0/d12/d2a/d10f 0 2024-09-21T04:15:27.850 INFO:tasks.workunit.client.0.smithi110.stdout:3/788: chown d0/d12/l44 2007 1 2024-09-21T04:15:27.852 INFO:tasks.workunit.client.0.smithi110.stdout:7/864: dwrite dd/db1/ff1 [0,4194304] 0 2024-09-21T04:15:27.852 INFO:tasks.workunit.client.0.smithi110.stdout:7/865: write dd/d4d/dc1/fe4 [218196,28817] 0 2024-09-21T04:15:27.852 INFO:tasks.workunit.client.1.smithi148.stdout:8/929: dwrite dc/d13/d76/f40 [0,4194304] 0 2024-09-21T04:15:27.852 INFO:tasks.workunit.client.1.smithi148.stdout:8/930: write dc/d13/d39/f51 [4158140,19843] 0 2024-09-21T04:15:27.853 INFO:tasks.workunit.client.0.smithi110.stdout:9/845: rmdir d7/d10/d88/d8f 39 2024-09-21T04:15:27.854 INFO:tasks.workunit.client.0.smithi110.stdout:3/789: dread d0/d12/d2a/d80/d9c/f103 [0,4194304] 0 2024-09-21T04:15:27.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/931: mkdir dc/d13/d39/d6e/d126 0 2024-09-21T04:15:27.859 INFO:tasks.workunit.client.1.smithi148.stdout:8/932: creat dc/d13/d3c/d81/d98/de5/d11b/f127 x:0 0 0 2024-09-21T04:15:27.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/933: rename dc/c1f to dc/d13/d3c/d81/db0/db7/d11d/c128 0 2024-09-21T04:15:27.876 INFO:tasks.workunit.client.0.smithi110.stdout:3/790: dread d0/f88 [0,4194304] 0 2024-09-21T04:15:27.877 INFO:tasks.workunit.client.0.smithi110.stdout:3/791: symlink d0/d22/d29/d76/l110 0 2024-09-21T04:15:27.877 INFO:tasks.workunit.client.0.smithi110.stdout:3/792: truncate d0/d12/d16/f96 836944 0 2024-09-21T04:15:27.878 INFO:tasks.workunit.client.0.smithi110.stdout:3/793: dread d0/d12/d16/f96 [0,4194304] 0 2024-09-21T04:15:27.878 INFO:tasks.workunit.client.0.smithi110.stdout:3/794: fsync d0/d12/d3c/fe8 0 2024-09-21T04:15:27.891 INFO:tasks.workunit.client.0.smithi110.stdout:9/846: dwrite d7/d96/d99/d3c/d35/f75 [0,4194304] 0 2024-09-21T04:15:27.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/847: symlink d7/d96/d99/d3c/d4f/d9f/d6f/d78/l125 0 2024-09-21T04:15:27.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/848: chown d7/d96/d99/d3c/d4f/d9f/d6f/d11f 5475982 1 2024-09-21T04:15:27.894 INFO:tasks.workunit.client.0.smithi110.stdout:3/795: write d0/fcb [731438,82749] 0 2024-09-21T04:15:27.895 INFO:tasks.workunit.client.0.smithi110.stdout:5/899: dwrite d1/d49/d6f/fc4 [0,4194304] 0 2024-09-21T04:15:27.895 INFO:tasks.workunit.client.0.smithi110.stdout:5/900: fsync d1/d7/f2b 0 2024-09-21T04:15:27.896 INFO:tasks.workunit.client.0.smithi110.stdout:9/849: rmdir d7/d10/dfb 39 2024-09-21T04:15:27.896 INFO:tasks.workunit.client.0.smithi110.stdout:9/850: chown d7/d96/d99/d3c/d1f/d43/db5/ffe 2082045874 1 2024-09-21T04:15:27.896 INFO:tasks.workunit.client.0.smithi110.stdout:9/851: write d7/d10/d88/d8f/da0/fc8 [854566,12336] 0 2024-09-21T04:15:27.900 INFO:tasks.workunit.client.1.smithi148.stdout:8/934: dwrite dc/d13/d28/f48 [0,4194304] 0 2024-09-21T04:15:27.900 INFO:tasks.workunit.client.1.smithi148.stdout:8/935: chown dc/d13/d39/d6e/dac/c125 1022249792 1 2024-09-21T04:15:27.901 INFO:tasks.workunit.client.0.smithi110.stdout:6/856: dwrite d4/d12/d16/d66/f93 [0,4194304] 0 2024-09-21T04:15:27.901 INFO:tasks.workunit.client.0.smithi110.stdout:6/857: write d4/d12/d17/d2e/dfb/d116/f11 [4482572,108425] 0 2024-09-21T04:15:27.902 INFO:tasks.workunit.client.0.smithi110.stdout:7/866: dwrite dd/d1c/dad/db2/d56/d66/fe1 [0,4194304] 0 2024-09-21T04:15:27.908 INFO:tasks.workunit.client.0.smithi110.stdout:7/867: mknod dd/d1c/dad/db2/d4a/d115/c11d 0 2024-09-21T04:15:27.909 INFO:tasks.workunit.client.0.smithi110.stdout:3/796: dread d0/d12/d3c/f5e [0,4194304] 0 2024-09-21T04:15:27.911 INFO:tasks.workunit.client.1.smithi148.stdout:8/936: write dc/d13/d3c/d81/d85/fcc [422410,80538] 0 2024-09-21T04:15:27.912 INFO:tasks.workunit.client.0.smithi110.stdout:7/868: unlink dd/d1c/d89/fe2 0 2024-09-21T04:15:27.913 INFO:tasks.workunit.client.0.smithi110.stdout:7/869: creat dd/d48/f11e x:0 0 0 2024-09-21T04:15:27.913 INFO:tasks.workunit.client.0.smithi110.stdout:7/870: dread - dd/d4d/dee/dbc/ff7 zero size 2024-09-21T04:15:27.914 INFO:tasks.workunit.client.0.smithi110.stdout:3/797: rename d0/d12/d3c/dad/fb7 to d0/d12/d2a/d80/f111 0 2024-09-21T04:15:27.915 INFO:tasks.workunit.client.1.smithi148.stdout:8/937: symlink dc/d43/d4c/l129 0 2024-09-21T04:15:27.915 INFO:tasks.workunit.client.1.smithi148.stdout:8/938: chown dc/d13/d28/de4/fea 1304042 1 2024-09-21T04:15:27.916 INFO:tasks.workunit.client.0.smithi110.stdout:6/858: rename d4/d12/d17/d33 to d4/d12/d17/d117 0 2024-09-21T04:15:27.919 INFO:tasks.workunit.client.0.smithi110.stdout:3/798: symlink d0/d12/d16/d1a/d5c/l112 0 2024-09-21T04:15:27.920 INFO:tasks.workunit.client.0.smithi110.stdout:5/901: dwrite d1/f3e [4194304,4194304] 0 2024-09-21T04:15:27.920 INFO:tasks.workunit.client.0.smithi110.stdout:9/852: dwrite d7/d96/d99/d3c/d4f/d9f/d6f/d108/d118/f11d [0,4194304] 0 2024-09-21T04:15:27.921 INFO:tasks.workunit.client.0.smithi110.stdout:3/799: rename d0/d12/d2a/d80/da2/db0/lbd to d0/d12/d3c/dad/l113 0 2024-09-21T04:15:27.923 INFO:tasks.workunit.client.0.smithi110.stdout:5/902: unlink d1/d22/d51/l104 0 2024-09-21T04:15:27.925 INFO:tasks.workunit.client.0.smithi110.stdout:7/871: dread dd/fc6 [0,4194304] 0 2024-09-21T04:15:27.925 INFO:tasks.workunit.client.0.smithi110.stdout:9/853: creat d7/d96/d99/d3c/d4f/d9f/d6f/d11f/f126 x:0 0 0 2024-09-21T04:15:27.926 INFO:tasks.workunit.client.0.smithi110.stdout:3/800: mkdir d0/d12/d114 0 2024-09-21T04:15:27.926 INFO:tasks.workunit.client.0.smithi110.stdout:3/801: creat d0/d22/dd2/f115 x:0 0 0 2024-09-21T04:15:27.926 INFO:tasks.workunit.client.0.smithi110.stdout:3/802: fsync d0/f6 0 2024-09-21T04:15:27.927 INFO:tasks.workunit.client.0.smithi110.stdout:5/903: unlink d1/d49/d4d/d1e/d4a/c85 0 2024-09-21T04:15:27.927 INFO:tasks.workunit.client.0.smithi110.stdout:5/904: fsync d1/d49/d4d/f46 0 2024-09-21T04:15:27.927 INFO:tasks.workunit.client.0.smithi110.stdout:7/872: creat dd/d1c/dad/d10d/f11f x:0 0 0 2024-09-21T04:15:27.927 INFO:tasks.workunit.client.0.smithi110.stdout:7/873: fdatasync dd/d1c/f75 0 2024-09-21T04:15:27.928 INFO:tasks.workunit.client.0.smithi110.stdout:9/854: rmdir d7/d96/d99/d3c/d1f/d43/db5/dbf/dd0 39 2024-09-21T04:15:27.929 INFO:tasks.workunit.client.0.smithi110.stdout:3/803: mkdir d0/d12/d2a/d80/da2/db0/d116 0 2024-09-21T04:15:27.930 INFO:tasks.workunit.client.0.smithi110.stdout:5/905: unlink d1/d7/l90 0 2024-09-21T04:15:27.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/855: rename d7/d10/l57 to d7/d26/l127 0 2024-09-21T04:15:27.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/856: stat d7/d96/d99/d3c/d4f/d9f/d6f/d78/l125 0 2024-09-21T04:15:27.932 INFO:tasks.workunit.client.0.smithi110.stdout:3/804: dread d0/d12/d16/d1a/d5c/faa [0,4194304] 0 2024-09-21T04:15:27.932 INFO:tasks.workunit.client.0.smithi110.stdout:3/805: fsync d0/d12/d16/d1a/d5c/f60 0 2024-09-21T04:15:27.933 INFO:tasks.workunit.client.0.smithi110.stdout:9/857: rename d7/d96/d99/fd7 to d7/d96/d99/d3c/d1f/d72/f128 0 2024-09-21T04:15:27.936 INFO:tasks.workunit.client.1.smithi148.stdout:8/939: dwrite dc/d13/d3c/f93 [0,4194304] 0 2024-09-21T04:15:27.936 INFO:tasks.workunit.client.0.smithi110.stdout:7/874: getdents dd/d1c/d89 0 2024-09-21T04:15:27.937 INFO:tasks.workunit.client.0.smithi110.stdout:6/859: dwrite d4/d12/d17/d81/fcc [0,4194304] 0 2024-09-21T04:15:27.940 INFO:tasks.workunit.client.0.smithi110.stdout:6/860: creat d4/d3c/db0/f118 x:0 0 0 2024-09-21T04:15:27.945 INFO:tasks.workunit.client.1.smithi148.stdout:8/940: symlink dc/d13/d39/l12a 0 2024-09-21T04:15:27.945 INFO:tasks.workunit.client.1.smithi148.stdout:8/941: stat dc/d13/f35 0 2024-09-21T04:15:27.946 INFO:tasks.workunit.client.0.smithi110.stdout:5/906: dread d1/d22/d51/fa5 [0,4194304] 0 2024-09-21T04:15:27.946 INFO:tasks.workunit.client.0.smithi110.stdout:6/861: creat d4/d78/f119 x:0 0 0 2024-09-21T04:15:27.946 INFO:tasks.workunit.client.0.smithi110.stdout:6/862: write d4/d12/d17/d2e/dfb/d116/fcd [1308484,43736] 0 2024-09-21T04:15:27.946 INFO:tasks.workunit.client.0.smithi110.stdout:6/863: fsync d4/d12/d17/d2e/dfb/d116/fda 0 2024-09-21T04:15:27.946 INFO:tasks.workunit.client.1.smithi148.stdout:8/942: creat dc/d13/d3c/d81/d98/dc1/f12b x:0 0 0 2024-09-21T04:15:27.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/907: unlink d1/dc/ded/lf6 0 2024-09-21T04:15:27.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/908: chown d1/d22/d6e/cbb 0 1 2024-09-21T04:15:27.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/909: chown d1/d49/d4d/d1e/d4a/dae/l103 394 1 2024-09-21T04:15:27.948 INFO:tasks.workunit.client.0.smithi110.stdout:6/864: symlink d4/d12/d16/d66/d7a/d4b/d110/l11a 0 2024-09-21T04:15:27.950 INFO:tasks.workunit.client.0.smithi110.stdout:5/910: creat d1/d49/d69/df5/f138 x:0 0 0 2024-09-21T04:15:27.951 INFO:tasks.workunit.client.0.smithi110.stdout:6/865: symlink d4/d12/d16/d6d/l11b 0 2024-09-21T04:15:27.952 INFO:tasks.workunit.client.0.smithi110.stdout:3/806: dwrite d0/d12/d16/d1a/f3b [0,4194304] 0 2024-09-21T04:15:27.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/943: getdents dc/d13/dbd 0 2024-09-21T04:15:27.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/944: getdents dc/d13/d76/d119 0 2024-09-21T04:15:27.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/945: write dc/f41 [261743,24967] 0 2024-09-21T04:15:27.953 INFO:tasks.workunit.client.0.smithi110.stdout:5/911: link d1/d7/d18/d8f/lcd d1/d22/d6e/l139 0 2024-09-21T04:15:27.955 INFO:tasks.workunit.client.0.smithi110.stdout:6/866: creat d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca/f11c x:0 0 0 2024-09-21T04:15:27.957 INFO:tasks.workunit.client.1.smithi148.stdout:8/946: dread f7 [4194304,4194304] 0 2024-09-21T04:15:27.957 INFO:tasks.workunit.client.1.smithi148.stdout:8/947: write dc/fc8 [873276,89983] 0 2024-09-21T04:15:27.958 INFO:tasks.workunit.client.0.smithi110.stdout:5/912: rmdir d1/d49/de4 39 2024-09-21T04:15:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/913: getdents d1/d7/d58/d72/dcb 0 2024-09-21T04:15:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/914: chown d1/d49/d4d/d1e/d4a/dd4/cdb 995278672 1 2024-09-21T04:15:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/915: getdents d1/d11f/d137 0 2024-09-21T04:15:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/916: truncate d1/d49/d4d/d1e/d5a/d64/f73 4511495 0 2024-09-21T04:15:27.967 INFO:tasks.workunit.client.0.smithi110.stdout:6/867: dwrite d4/d86/d8f/fc7 [0,4194304] 0 2024-09-21T04:15:27.967 INFO:tasks.workunit.client.0.smithi110.stdout:6/868: fdatasync f3 0 2024-09-21T04:15:27.971 INFO:tasks.workunit.client.0.smithi110.stdout:3/807: write d0/d12/d16/d1a/d5c/faa [279338,79162] 0 2024-09-21T04:15:27.973 INFO:tasks.workunit.client.1.smithi148.stdout:8/948: dwrite dc/d13/d28/d62/f97 [4194304,4194304] 0 2024-09-21T04:15:27.973 INFO:tasks.workunit.client.1.smithi148.stdout:8/949: write dc/d13/d76/d5d/d78/f10f [91853,67681] 0 2024-09-21T04:15:27.974 INFO:tasks.workunit.client.0.smithi110.stdout:5/917: rmdir d1 39 2024-09-21T04:15:27.975 INFO:tasks.workunit.client.1.smithi148.stdout:8/950: symlink dc/d13/d28/d62/l12c 0 2024-09-21T04:15:27.977 INFO:tasks.workunit.client.1.smithi148.stdout:8/951: mkdir dc/d13/d39/d6e/dac/d12d 0 2024-09-21T04:15:27.978 INFO:tasks.workunit.client.0.smithi110.stdout:6/869: mknod d4/d12/d16/d66/d7a/d4b/d7f/d113/c11d 0 2024-09-21T04:15:27.978 INFO:tasks.workunit.client.0.smithi110.stdout:6/870: fsync d4/d12/f2b 0 2024-09-21T04:15:27.978 INFO:tasks.workunit.client.0.smithi110.stdout:6/871: readlink d4/d12/d16/d66/d7a/d54/l9c 0 2024-09-21T04:15:27.979 INFO:tasks.workunit.client.0.smithi110.stdout:6/872: fdatasync d4/d12/d16/f7e 0 2024-09-21T04:15:27.979 INFO:tasks.workunit.client.0.smithi110.stdout:3/808: mknod d0/d51/c117 0 2024-09-21T04:15:27.979 INFO:tasks.workunit.client.0.smithi110.stdout:5/918: mkdir d1/d13a 0 2024-09-21T04:15:27.980 INFO:tasks.workunit.client.0.smithi110.stdout:6/873: creat d4/d12/d16/d66/d7a/d54/f11e x:0 0 0 2024-09-21T04:15:27.980 INFO:tasks.workunit.client.0.smithi110.stdout:6/874: write d4/d12/d16/d66/d7a/d83/fe0 [462934,128410] 0 2024-09-21T04:15:27.981 INFO:tasks.workunit.client.0.smithi110.stdout:3/809: symlink d0/d12/d3c/d5a/d70/d81/l118 0 2024-09-21T04:15:27.981 INFO:tasks.workunit.client.0.smithi110.stdout:5/919: getdents d1/d49/d4d/d1e/d4a/dd4 0 2024-09-21T04:15:27.982 INFO:tasks.workunit.client.0.smithi110.stdout:6/875: mknod d4/d12/d17/d9a/dbe/c11f 0 2024-09-21T04:15:27.982 INFO:tasks.workunit.client.0.smithi110.stdout:6/876: readlink d4/d12/la8 0 2024-09-21T04:15:27.982 INFO:tasks.workunit.client.0.smithi110.stdout:3/810: mknod d0/d12/d16/d1a/d5c/c119 0 2024-09-21T04:15:27.983 INFO:tasks.workunit.client.0.smithi110.stdout:5/920: symlink d1/d7/d58/d72/dcb/l13b 0 2024-09-21T04:15:27.983 INFO:tasks.workunit.client.0.smithi110.stdout:5/921: write d1/d7/d58/f82 [4349681,43146] 0 2024-09-21T04:15:27.986 INFO:tasks.workunit.client.0.smithi110.stdout:5/922: symlink d1/d7/d18/d8f/dc2/l13c 0 2024-09-21T04:15:27.987 INFO:tasks.workunit.client.0.smithi110.stdout:5/923: getdents d1/d49/d69/d100 0 2024-09-21T04:15:27.989 INFO:tasks.workunit.client.0.smithi110.stdout:5/924: getdents d1/d22/db6/dc6 0 2024-09-21T04:15:27.989 INFO:tasks.workunit.client.0.smithi110.stdout:5/925: dread - d1/d39/fe6 zero size 2024-09-21T04:15:27.989 INFO:tasks.workunit.client.0.smithi110.stdout:5/926: fsync d1/d22/d51/fd1 0 2024-09-21T04:15:27.992 INFO:tasks.workunit.client.0.smithi110.stdout:5/927: rename d1/dc/ded/f114 to d1/d49/d4d/d1e/d5a/d64/f13d 0 2024-09-21T04:15:27.992 INFO:tasks.workunit.client.0.smithi110.stdout:5/928: chown d1/f15 1948804907 1 2024-09-21T04:15:27.992 INFO:tasks.workunit.client.0.smithi110.stdout:6/877: read d4/d12/d17/d2e/dfb/d116/fb4 [1456500,97936] 0 2024-09-21T04:15:27.993 INFO:tasks.workunit.client.0.smithi110.stdout:6/878: mkdir d4/d12/d17/d2e/dfb/d116/d120 0 2024-09-21T04:15:27.993 INFO:tasks.workunit.client.0.smithi110.stdout:6/879: write d4/d12/d16/d66/d7a/d4b/f5e [5090459,110262] 0 2024-09-21T04:15:28.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/929: write d1/d7/d18/d3f/f93 [196302,66573] 0 2024-09-21T04:15:28.002 INFO:tasks.workunit.client.0.smithi110.stdout:3/811: dwrite d0/d12/d16/d1a/d5c/f85 [0,4194304] 0 2024-09-21T04:15:28.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/930: dread - d1/d22/d6e/fcf zero size 2024-09-21T04:15:28.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/952: dwrite dc/d13/fe0 [4194304,4194304] 0 2024-09-21T04:15:28.004 INFO:tasks.workunit.client.0.smithi110.stdout:3/812: dread d0/d12/fd5 [0,4194304] 0 2024-09-21T04:15:28.005 INFO:tasks.workunit.client.1.smithi148.stdout:8/953: unlink dc/d13/l20 0 2024-09-21T04:15:28.005 INFO:tasks.workunit.client.1.smithi148.stdout:8/954: truncate dc/d13/f9b 4361812 0 2024-09-21T04:15:28.006 INFO:tasks.workunit.client.1.smithi148.stdout:8/955: creat dc/f12e x:0 0 0 2024-09-21T04:15:28.007 INFO:tasks.workunit.client.0.smithi110.stdout:3/813: rename d0/d12/d3c/dce/cd1 to d0/d12/d3c/d5a/d65/df0/c11a 0 2024-09-21T04:15:28.010 INFO:tasks.workunit.client.0.smithi110.stdout:3/814: mknod d0/d12/d114/c11b 0 2024-09-21T04:15:28.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/815: readlink d0/d12/d2a/d80/d9c/lc2 0 2024-09-21T04:15:28.011 INFO:tasks.workunit.client.0.smithi110.stdout:5/931: write d1/d22/d51/fa5 [3840644,65686] 0 2024-09-21T04:15:28.011 INFO:tasks.workunit.client.0.smithi110.stdout:5/932: chown d1/d7/d18/d8f/lcd 447515 1 2024-09-21T04:15:28.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/816: symlink d0/d12/d2a/d80/dc4/l11c 0 2024-09-21T04:15:28.013 INFO:tasks.workunit.client.0.smithi110.stdout:5/933: creat d1/d49/d4d/d1e/d4a/dd4/dd5/f13e x:0 0 0 2024-09-21T04:15:28.013 INFO:tasks.workunit.client.0.smithi110.stdout:6/880: dwrite d4/d12/d16/ff6 [0,4194304] 0 2024-09-21T04:15:28.013 INFO:tasks.workunit.client.0.smithi110.stdout:6/881: chown d4/d12/d17/d9a/dbe 1736124287 1 2024-09-21T04:15:28.014 INFO:tasks.workunit.client.0.smithi110.stdout:3/817: symlink d0/d12/d2a/d80/da2/db0/l11d 0 2024-09-21T04:15:28.014 INFO:tasks.workunit.client.0.smithi110.stdout:3/818: truncate d0/d12/d3c/dce/fe1 1562689 0 2024-09-21T04:15:28.015 INFO:tasks.workunit.client.0.smithi110.stdout:5/934: dread d1/d7/d18/f74 [0,4194304] 0 2024-09-21T04:15:28.019 INFO:tasks.workunit.client.0.smithi110.stdout:6/882: mknod d4/d12/d17/d81/d90/c121 0 2024-09-21T04:15:28.019 INFO:tasks.workunit.client.0.smithi110.stdout:6/883: stat d4/d86/d8f 0 2024-09-21T04:15:28.038 INFO:tasks.workunit.client.0.smithi110.stdout:3/819: dwrite d0/d12/f23 [0,4194304] 0 2024-09-21T04:15:28.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/884: dwrite d4/d78/fe9 [0,4194304] 0 2024-09-21T04:15:28.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/885: creat d4/d12/d16/f122 x:0 0 0 2024-09-21T04:15:28.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/886: creat d4/d86/f123 x:0 0 0 2024-09-21T04:15:28.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/887: dread - d4/d12/d17/d81/d90/fd9 zero size 2024-09-21T04:15:28.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/888: creat d4/d12/d17/f124 x:0 0 0 2024-09-21T04:15:28.044 INFO:tasks.workunit.client.0.smithi110.stdout:6/889: creat d4/d12/d17/d2e/d40/f125 x:0 0 0 2024-09-21T04:15:28.044 INFO:tasks.workunit.client.0.smithi110.stdout:6/890: read d4/d12/d17/d2e/d38/f98 [568397,41385] 0 2024-09-21T04:15:28.044 INFO:tasks.workunit.client.0.smithi110.stdout:6/891: chown d4/d12/d17/d2e 87560650 1 2024-09-21T04:15:28.046 INFO:tasks.workunit.client.0.smithi110.stdout:6/892: rmdir d4/d12/d17/d81/d90 39 2024-09-21T04:15:28.046 INFO:tasks.workunit.client.0.smithi110.stdout:6/893: stat d4/d12/d17/d2e/dfb/d116/ld4 0 2024-09-21T04:15:28.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/820: dwrite d0/d12/d16/d1a/d95/f10d [0,4194304] 0 2024-09-21T04:15:28.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/821: chown d0/d12/d2a/d80/da2 3997361 1 2024-09-21T04:15:28.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/894: dwrite d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:28.074 INFO:tasks.workunit.client.0.smithi110.stdout:3/822: dwrite d0/d12/d3c/ff4 [0,4194304] 0 2024-09-21T04:15:28.074 INFO:tasks.workunit.client.0.smithi110.stdout:3/823: truncate d0/f77 838590 0 2024-09-21T04:15:28.075 INFO:tasks.workunit.client.0.smithi110.stdout:3/824: stat d0/d22/c55 0 2024-09-21T04:15:28.075 INFO:tasks.workunit.client.0.smithi110.stdout:6/895: mknod d4/d12/d17/d2e/dfb/c126 0 2024-09-21T04:15:28.076 INFO:tasks.workunit.client.0.smithi110.stdout:3/825: creat d0/d12/d3c/d5a/d70/dab/f11e x:0 0 0 2024-09-21T04:15:28.076 INFO:tasks.workunit.client.0.smithi110.stdout:3/826: fdatasync d0/d12/f99 0 2024-09-21T04:15:28.076 INFO:tasks.workunit.client.0.smithi110.stdout:3/827: dread - d0/d12/d2a/d80/da2/da7/f102 zero size 2024-09-21T04:15:28.076 INFO:tasks.workunit.client.0.smithi110.stdout:6/896: creat d4/d12/d17/d117/f127 x:0 0 0 2024-09-21T04:15:28.077 INFO:tasks.workunit.client.0.smithi110.stdout:3/828: mkdir d0/d12/d3c/dad/d11f 0 2024-09-21T04:15:28.099 INFO:tasks.workunit.client.0.smithi110.stdout:6/897: dwrite d4/f79 [0,4194304] 0 2024-09-21T04:15:28.099 INFO:tasks.workunit.client.0.smithi110.stdout:6/898: chown d4/d12/d17/d9a/fb2 144885407 1 2024-09-21T04:15:28.100 INFO:tasks.workunit.client.0.smithi110.stdout:6/899: mknod d4/d12/d16/d66/d7a/d54/c128 0 2024-09-21T04:15:28.104 INFO:tasks.workunit.client.0.smithi110.stdout:3/829: dwrite d0/d12/d2a/d80/da2/db0/fd8 [0,4194304] 0 2024-09-21T04:15:28.106 INFO:tasks.workunit.client.0.smithi110.stdout:3/830: symlink d0/d12/d16/d1a/d95/l120 0 2024-09-21T04:15:28.108 INFO:tasks.workunit.client.0.smithi110.stdout:3/831: link d0/d12/d3c/dad/l10e d0/d12/d16/d1a/d6f/dd9/l121 0 2024-09-21T04:15:28.108 INFO:tasks.workunit.client.0.smithi110.stdout:3/832: dread - d0/d12/d2a/d80/d9c/ff1 zero size 2024-09-21T04:15:28.108 INFO:tasks.workunit.client.0.smithi110.stdout:6/900: write d4/d12/d16/d66/fd5 [2391148,33164] 0 2024-09-21T04:15:28.108 INFO:tasks.workunit.client.0.smithi110.stdout:6/901: chown d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca/f11c 51332156 1 2024-09-21T04:15:28.109 INFO:tasks.workunit.client.0.smithi110.stdout:3/833: mkdir d0/d12/d3c/d5a/d70/d81/d122 0 2024-09-21T04:15:28.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/834: mkdir d0/d12/d3c/dce/d123 0 2024-09-21T04:15:28.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/835: chown d0/d22/cb6 12406 1 2024-09-21T04:15:28.111 INFO:tasks.workunit.client.0.smithi110.stdout:3/836: mknod d0/d12/d16/d1a/d5c/c124 0 2024-09-21T04:15:28.113 INFO:tasks.workunit.client.0.smithi110.stdout:3/837: symlink d0/d12/d3c/d5a/d70/d81/d122/l125 0 2024-09-21T04:15:28.113 INFO:tasks.workunit.client.0.smithi110.stdout:3/838: write d0/d12/d3c/f45 [5029753,31647] 0 2024-09-21T04:15:28.115 INFO:tasks.workunit.client.0.smithi110.stdout:6/902: write d4/d86/d8f/fc7 [3274554,67601] 0 2024-09-21T04:15:28.115 INFO:tasks.workunit.client.0.smithi110.stdout:3/839: unlink d0/d12/d3c/c8e 0 2024-09-21T04:15:28.118 INFO:tasks.workunit.client.0.smithi110.stdout:6/903: link d4/d12/d17/d9a/dba/fe2 d4/d12/d16/d66/d7a/f129 0 2024-09-21T04:15:28.118 INFO:tasks.workunit.client.0.smithi110.stdout:6/904: chown d4/d12/d16/d66/d7a/d4b/d7f 43162 1 2024-09-21T04:15:28.119 INFO:tasks.workunit.client.0.smithi110.stdout:3/840: creat d0/d12/d2a/f126 x:0 0 0 2024-09-21T04:15:28.119 INFO:tasks.workunit.client.0.smithi110.stdout:3/841: dread - d0/d12/d16/d1a/fc1 zero size 2024-09-21T04:15:28.119 INFO:tasks.workunit.client.0.smithi110.stdout:6/905: symlink d4/d78/l12a 0 2024-09-21T04:15:28.121 INFO:tasks.workunit.client.0.smithi110.stdout:3/842: rename d0/d12/d3c/d5a/d70/cf2 to d0/d12/d16/d1a/d6f/c127 0 2024-09-21T04:15:28.121 INFO:tasks.workunit.client.0.smithi110.stdout:8/905: sync 2024-09-21T04:15:28.121 INFO:tasks.workunit.client.0.smithi110.stdout:2/770: sync 2024-09-21T04:15:28.121 INFO:tasks.workunit.client.0.smithi110.stdout:2/771: write d2/d1a/f88 [507975,107953] 0 2024-09-21T04:15:28.122 INFO:tasks.workunit.client.0.smithi110.stdout:2/772: creat d2/d50/ff6 x:0 0 0 2024-09-21T04:15:28.122 INFO:tasks.workunit.client.0.smithi110.stdout:2/773: write d2/d1a/d1f/d65/f68 [2130985,98763] 0 2024-09-21T04:15:28.123 INFO:tasks.workunit.client.0.smithi110.stdout:2/774: mkdir d2/d1a/d1f/d5f/d9d/d86/d9f/df7 0 2024-09-21T04:15:28.124 INFO:tasks.workunit.client.0.smithi110.stdout:2/775: rename d2/de7/ceb to d2/de7/cf8 0 2024-09-21T04:15:28.124 INFO:tasks.workunit.client.0.smithi110.stdout:2/776: chown d2/d1a/d1f/d29/d19/fdb 75006 1 2024-09-21T04:15:28.124 INFO:tasks.workunit.client.0.smithi110.stdout:2/777: mknod d2/d1a/cf9 0 2024-09-21T04:15:28.125 INFO:tasks.workunit.client.0.smithi110.stdout:2/778: rmdir d2/d1a/d8b/da5 39 2024-09-21T04:15:28.125 INFO:tasks.workunit.client.0.smithi110.stdout:2/779: fdatasync d2/d1a/d1f/d5f/f2b 0 2024-09-21T04:15:28.125 INFO:tasks.workunit.client.0.smithi110.stdout:2/780: mkdir d2/d81/dfa 0 2024-09-21T04:15:28.126 INFO:tasks.workunit.client.0.smithi110.stdout:2/781: mknod d2/d1a/da3/db9/cfb 0 2024-09-21T04:15:28.126 INFO:tasks.workunit.client.0.smithi110.stdout:6/906: write d4/d12/f1d [7292243,121818] 0 2024-09-21T04:15:28.127 INFO:tasks.workunit.client.0.smithi110.stdout:2/782: symlink d2/d1a/d1f/d5f/d9d/d86/dad/lfc 0 2024-09-21T04:15:28.128 INFO:tasks.workunit.client.0.smithi110.stdout:2/783: mkdir d2/d1a/d60/dfd 0 2024-09-21T04:15:28.129 INFO:tasks.workunit.client.0.smithi110.stdout:2/784: truncate d2/d50/d56/f51 4304339 0 2024-09-21T04:15:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:8/906: dread d0/d30/d34/d3e/d32/fb9 [0,4194304] 0 2024-09-21T04:15:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:8/907: fdatasync d0/d7/d23/d82/d92/fc4 0 2024-09-21T04:15:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:8/908: creat d0/d30/d9d/f147 x:0 0 0 2024-09-21T04:15:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:8/909: fdatasync d0/d30/d34/d3e/d32/da3/fa9 0 2024-09-21T04:15:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:3/843: write d0/d12/d16/d1a/dac/fb5 [2903719,95631] 0 2024-09-21T04:15:28.137 INFO:tasks.workunit.client.0.smithi110.stdout:3/844: rmdir d0/d12/d16/d1a/d6f 39 2024-09-21T04:15:28.137 INFO:tasks.workunit.client.0.smithi110.stdout:3/845: stat d0/d12/d16/l7c 0 2024-09-21T04:15:28.137 INFO:tasks.workunit.client.0.smithi110.stdout:3/846: truncate d0/d12/d2a/fde 1264244 0 2024-09-21T04:15:28.143 INFO:tasks.workunit.client.0.smithi110.stdout:2/785: dread d2/d1a/d1f/d5f/f1d [0,4194304] 0 2024-09-21T04:15:28.146 INFO:tasks.workunit.client.0.smithi110.stdout:8/910: write d0/d30/d34/d3e/d32/fb9 [53017,110689] 0 2024-09-21T04:15:28.147 INFO:tasks.workunit.client.0.smithi110.stdout:8/911: creat d0/d7/d23/d82/d92/f148 x:0 0 0 2024-09-21T04:15:28.151 INFO:tasks.workunit.client.0.smithi110.stdout:6/907: dwrite d4/d12/d16/ff6 [0,4194304] 0 2024-09-21T04:15:28.153 INFO:tasks.workunit.client.0.smithi110.stdout:8/912: dread d0/d7/f20 [0,4194304] 0 2024-09-21T04:15:28.153 INFO:tasks.workunit.client.0.smithi110.stdout:8/913: stat d0/d7/d13/d4b/l6e 0 2024-09-21T04:15:28.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/914: rmdir d0/d30/d34/d3e/d5d/dd7/de8 39 2024-09-21T04:15:28.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/915: fsync d0/d7/d13/d4b/f67 0 2024-09-21T04:15:28.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/916: chown d0/d7/d6f/f120 7094922 1 2024-09-21T04:15:28.159 INFO:tasks.workunit.client.0.smithi110.stdout:2/786: dread d2/d1a/d1f/d5f/d9d/d86/d9b/fb4 [0,4194304] 0 2024-09-21T04:15:28.159 INFO:tasks.workunit.client.0.smithi110.stdout:3/847: dwrite d0/d22/dd2/ff6 [0,4194304] 0 2024-09-21T04:15:28.160 INFO:tasks.workunit.client.0.smithi110.stdout:2/787: symlink d2/d1a/d1f/d5f/lfe 0 2024-09-21T04:15:28.161 INFO:tasks.workunit.client.0.smithi110.stdout:3/848: rename d0/d12/d3c/d5a/d70/d81/d122 to d0/d12/d2a/d80/da2/da7/d128 0 2024-09-21T04:15:28.161 INFO:tasks.workunit.client.0.smithi110.stdout:2/788: mkdir d2/d1a/d1f/d5f/d9d/d86/d9f/df7/dff 0 2024-09-21T04:15:28.162 INFO:tasks.workunit.client.1.smithi148.stdout:8/956: sync 2024-09-21T04:15:28.162 INFO:tasks.workunit.client.0.smithi110.stdout:3/849: creat d0/f129 x:0 0 0 2024-09-21T04:15:28.163 INFO:tasks.workunit.client.0.smithi110.stdout:2/789: mknod d2/d1a/d1f/d5f/d9d/d86/d9f/df7/c100 0 2024-09-21T04:15:28.163 INFO:tasks.workunit.client.0.smithi110.stdout:2/790: readlink d2/d1a/d1f/d5f/l59 0 2024-09-21T04:15:28.163 INFO:tasks.workunit.client.0.smithi110.stdout:2/791: fsync d2/d1a/d1f/d5f/d9d/d86/d9b/fb4 0 2024-09-21T04:15:28.163 INFO:tasks.workunit.client.0.smithi110.stdout:2/792: dread - d2/d50/ff6 zero size 2024-09-21T04:15:28.164 INFO:tasks.workunit.client.1.smithi148.stdout:8/957: creat dc/d13/d3c/d81/db0/f12f x:0 0 0 2024-09-21T04:15:28.165 INFO:tasks.workunit.client.0.smithi110.stdout:2/793: mknod d2/de7/c101 0 2024-09-21T04:15:28.165 INFO:tasks.workunit.client.0.smithi110.stdout:2/794: fsync d2/d1a/d1f/d29/f71 0 2024-09-21T04:15:28.165 INFO:tasks.workunit.client.1.smithi148.stdout:8/958: mknod dc/de6/c130 0 2024-09-21T04:15:28.167 INFO:tasks.workunit.client.0.smithi110.stdout:2/795: rename d2/d50/d56/f51 to d2/d50/d56/f102 0 2024-09-21T04:15:28.168 INFO:tasks.workunit.client.1.smithi148.stdout:8/959: rmdir dc/d13/d39/d6e/d108 0 2024-09-21T04:15:28.169 INFO:tasks.workunit.client.1.smithi148.stdout:8/960: creat dc/d13/d39/d6e/f131 x:0 0 0 2024-09-21T04:15:28.170 INFO:tasks.workunit.client.1.smithi148.stdout:8/961: rmdir dc/d13/d3c/d81/d120 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/796: write d2/d1a/d1f/d5f/f1d [1462038,113632] 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/797: fdatasync d2/d1a/d1f/d29/f71 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/798: truncate d2/d1a/da3/fb1 4832308 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/799: creat d2/d1a/d1f/d5f/d9d/d86/f103 x:0 0 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/800: fsync d2/d1a/d1f/d29/f45 0 2024-09-21T04:15:28.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/801: write d2/d1a/d1f/d29/d19/fb7 [514579,118555] 0 2024-09-21T04:15:28.175 INFO:tasks.workunit.client.0.smithi110.stdout:2/802: creat d2/d50/d56/f104 x:0 0 0 2024-09-21T04:15:28.175 INFO:tasks.workunit.client.0.smithi110.stdout:2/803: fsync d2/d1a/d1f/d29/f71 0 2024-09-21T04:15:28.175 INFO:tasks.workunit.client.0.smithi110.stdout:2/804: fsync d2/d1a/d1f/d29/f4f 0 2024-09-21T04:15:28.175 INFO:tasks.workunit.client.0.smithi110.stdout:2/805: link d2/d1a/da3/fb1 d2/d1a/f105 0 2024-09-21T04:15:28.177 INFO:tasks.workunit.client.0.smithi110.stdout:2/806: unlink d2/f44 0 2024-09-21T04:15:28.177 INFO:tasks.workunit.client.0.smithi110.stdout:3/850: write d0/f10 [4323046,33036] 0 2024-09-21T04:15:28.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/917: dwrite d0/d7/d13/d4b/f17 [0,4194304] 0 2024-09-21T04:15:28.179 INFO:tasks.workunit.client.0.smithi110.stdout:6/908: dwrite d4/d78/f119 [0,4194304] 0 2024-09-21T04:15:28.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/918: truncate d0/d7/d23/d82/d92/f148 647215 0 2024-09-21T04:15:28.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/919: write d0/d7/d13/d4b/f134 [72688,2759] 0 2024-09-21T04:15:28.180 INFO:tasks.workunit.client.0.smithi110.stdout:3/851: link d0/d12/d2a/d80/da2/db0/fd8 d0/d22/dd2/f12a 0 2024-09-21T04:15:28.183 INFO:tasks.workunit.client.0.smithi110.stdout:6/909: mknod d4/d12/d17/d9a/de4/c12b 0 2024-09-21T04:15:28.184 INFO:tasks.workunit.client.0.smithi110.stdout:8/920: mknod d0/d7/d23/d6c/c149 0 2024-09-21T04:15:28.184 INFO:tasks.workunit.client.0.smithi110.stdout:8/921: write d0/d7/d13/d46/df8/f12c [707531,76522] 0 2024-09-21T04:15:28.185 INFO:tasks.workunit.client.0.smithi110.stdout:3/852: creat d0/d12/d3c/dad/d11f/f12b x:0 0 0 2024-09-21T04:15:28.185 INFO:tasks.workunit.client.0.smithi110.stdout:3/853: stat d0/d12/d16/d1a/d95/c98 0 2024-09-21T04:15:28.187 INFO:tasks.workunit.client.0.smithi110.stdout:6/910: creat d4/d12/d16/d6d/f12c x:0 0 0 2024-09-21T04:15:28.187 INFO:tasks.workunit.client.0.smithi110.stdout:6/911: dread - d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca/f11c zero size 2024-09-21T04:15:28.188 INFO:tasks.workunit.client.0.smithi110.stdout:6/912: mkdir d4/d12/d17/dfe/d12d 0 2024-09-21T04:15:28.188 INFO:tasks.workunit.client.0.smithi110.stdout:6/913: chown d4/d12/d17/d9a/fb2 1398958 1 2024-09-21T04:15:28.192 INFO:tasks.workunit.client.0.smithi110.stdout:6/914: mkdir d4/d3c/d12e 0 2024-09-21T04:15:28.192 INFO:tasks.workunit.client.0.smithi110.stdout:6/915: getdents d4/d12/d17/d2e/dfb/d116/d120 0 2024-09-21T04:15:28.192 INFO:tasks.workunit.client.0.smithi110.stdout:6/916: truncate d4/d12/d17/d2e/dfb/d116/ffd 384076 0 2024-09-21T04:15:28.196 INFO:tasks.workunit.client.0.smithi110.stdout:2/807: dwrite d2/fbb [0,4194304] 0 2024-09-21T04:15:28.197 INFO:tasks.workunit.client.1.smithi148.stdout:8/962: dwrite dc/d13/d3c/d81/d85/da1/fb8 [0,4194304] 0 2024-09-21T04:15:28.198 INFO:tasks.workunit.client.0.smithi110.stdout:2/808: unlink d2/d50/d56/f7e 0 2024-09-21T04:15:28.198 INFO:tasks.workunit.client.1.smithi148.stdout:8/963: dread dc/f41 [0,4194304] 0 2024-09-21T04:15:28.201 INFO:tasks.workunit.client.0.smithi110.stdout:2/809: mknod d2/d1a/d1f/d5f/d9d/d86/c106 0 2024-09-21T04:15:28.203 INFO:tasks.workunit.client.0.smithi110.stdout:2/810: creat d2/d1a/d1f/d5f/d9d/d86/d9f/df7/f107 x:0 0 0 2024-09-21T04:15:28.208 INFO:tasks.workunit.client.0.smithi110.stdout:2/811: rmdir d2/d1a/da3 39 2024-09-21T04:15:28.208 INFO:tasks.workunit.client.0.smithi110.stdout:2/812: chown d2/d50/d56/fc4 7938 1 2024-09-21T04:15:28.210 INFO:tasks.workunit.client.0.smithi110.stdout:2/813: mkdir d2/d81/dfa/d108 0 2024-09-21T04:15:28.210 INFO:tasks.workunit.client.0.smithi110.stdout:2/814: write d2/d50/ff6 [1012471,32995] 0 2024-09-21T04:15:28.210 INFO:tasks.workunit.client.0.smithi110.stdout:6/917: dread d4/d78/f119 [0,4194304] 0 2024-09-21T04:15:28.210 INFO:tasks.workunit.client.0.smithi110.stdout:6/918: dread - d4/d12/d16/d66/d7a/f129 zero size 2024-09-21T04:15:28.210 INFO:tasks.workunit.client.0.smithi110.stdout:2/815: creat d2/d1a/d1f/d5f/d9d/f109 x:0 0 0 2024-09-21T04:15:28.211 INFO:tasks.workunit.client.0.smithi110.stdout:6/919: symlink d4/d12/d17/d2e/dfb/l12f 0 2024-09-21T04:15:28.212 INFO:tasks.workunit.client.0.smithi110.stdout:2/816: unlink d2/d1a/d1f/d65/c67 0 2024-09-21T04:15:28.214 INFO:tasks.workunit.client.0.smithi110.stdout:3/854: dwrite d0/d12/d3c/d5a/d65/df0/f109 [0,4194304] 0 2024-09-21T04:15:28.215 INFO:tasks.workunit.client.0.smithi110.stdout:8/922: dwrite d0/d7/fd [0,4194304] 0 2024-09-21T04:15:28.217 INFO:tasks.workunit.client.0.smithi110.stdout:2/817: write d2/d1a/d1f/d29/d19/fb7 [166530,84390] 0 2024-09-21T04:15:28.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/920: truncate d4/d12/d16/d66/fd5 3998136 0 2024-09-21T04:15:28.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/921: write d4/d86/f123 [509640,8666] 0 2024-09-21T04:15:28.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/922: stat d4/d12/d17/d81/d90/fd9 0 2024-09-21T04:15:28.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/923: creat d4/d86/f130 x:0 0 0 2024-09-21T04:15:28.226 INFO:tasks.workunit.client.0.smithi110.stdout:8/923: symlink d0/d7/d89/d10f/l14a 0 2024-09-21T04:15:28.226 INFO:tasks.workunit.client.0.smithi110.stdout:8/924: fdatasync d0/d30/d9d/f147 0 2024-09-21T04:15:28.227 INFO:tasks.workunit.client.0.smithi110.stdout:2/818: symlink d2/d1a/d60/dfd/l10a 0 2024-09-21T04:15:28.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/855: dwrite d0/d12/d2a/f3e [0,4194304] 0 2024-09-21T04:15:28.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/856: creat d0/d12/d3c/dad/d11f/f12c x:0 0 0 2024-09-21T04:15:28.229 INFO:tasks.workunit.client.0.smithi110.stdout:6/924: unlink d4/d3c/db0/c114 0 2024-09-21T04:15:28.231 INFO:tasks.workunit.client.0.smithi110.stdout:8/925: rename d0/d7/d13/f2d to d0/d7/d13/d46/f14b 0 2024-09-21T04:15:28.231 INFO:tasks.workunit.client.0.smithi110.stdout:8/926: chown d0/d7/d6f/f101 126310 1 2024-09-21T04:15:28.233 INFO:tasks.workunit.client.0.smithi110.stdout:8/927: dread d0/d7/d89/fb2 [0,4194304] 0 2024-09-21T04:15:28.234 INFO:tasks.workunit.client.0.smithi110.stdout:3/857: mknod d0/d12/d3c/d5a/d65/df0/c12d 0 2024-09-21T04:15:28.240 INFO:tasks.workunit.client.0.smithi110.stdout:8/928: mkdir d0/d7/d89/d10f/d14c 0 2024-09-21T04:15:28.241 INFO:tasks.workunit.client.0.smithi110.stdout:3/858: unlink d0/d51/d7b/ffc 0 2024-09-21T04:15:28.241 INFO:tasks.workunit.client.0.smithi110.stdout:3/859: creat d0/d12/d3c/d5a/dc8/f12e x:0 0 0 2024-09-21T04:15:28.242 INFO:tasks.workunit.client.0.smithi110.stdout:6/925: dwrite d4/d12/d16/d66/fac [0,4194304] 0 2024-09-21T04:15:28.243 INFO:tasks.workunit.client.0.smithi110.stdout:8/929: dread d0/d7/d13/d4b/f134 [0,4194304] 0 2024-09-21T04:15:28.252 INFO:tasks.workunit.client.0.smithi110.stdout:3/860: mkdir d0/d12/d3c/dad/d12f 0 2024-09-21T04:15:28.255 INFO:tasks.workunit.client.0.smithi110.stdout:2/819: rename d2/d1a/d1f to d2/d1a/d8b/d10b 0 2024-09-21T04:15:28.255 INFO:tasks.workunit.client.0.smithi110.stdout:3/861: rmdir d0/d12/d2a/d80/da2/db0 39 2024-09-21T04:15:28.256 INFO:tasks.workunit.client.0.smithi110.stdout:3/862: mknod d0/d12/d3c/dce/c130 0 2024-09-21T04:15:28.258 INFO:tasks.workunit.client.0.smithi110.stdout:3/863: mkdir d0/d12/d114/d131 0 2024-09-21T04:15:28.258 INFO:tasks.workunit.client.0.smithi110.stdout:3/864: write d0/d12/d2a/fdd [1006557,3150] 0 2024-09-21T04:15:28.258 INFO:tasks.workunit.client.0.smithi110.stdout:3/865: creat d0/d12/d3c/dad/d11f/f132 x:0 0 0 2024-09-21T04:15:28.258 INFO:tasks.workunit.client.0.smithi110.stdout:3/866: creat d0/d12/d16/d1a/d95/f133 x:0 0 0 2024-09-21T04:15:28.264 INFO:tasks.workunit.client.0.smithi110.stdout:9/858: sync 2024-09-21T04:15:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:2/820: write d2/d1a/da3/fb1 [2323399,12808] 0 2024-09-21T04:15:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:6/926: dwrite d4/d12/d17/d117/f127 [0,4194304] 0 2024-09-21T04:15:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:6/927: getdents d4/d12/d17/d81/db6 0 2024-09-21T04:15:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:8/930: dwrite d0/d7/d13/d4b/f28 [0,4194304] 0 2024-09-21T04:15:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:8/931: creat d0/d7/d6f/f14d x:0 0 0 2024-09-21T04:15:28.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/821: truncate d2/d1a/d8b/d10b/d5f/d9d/d86/d9f/dc3/fee 4756578 0 2024-09-21T04:15:28.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/822: readlink d2/d1a/d8b/d10b/d29/d19/l84 0 2024-09-21T04:15:28.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/928: mkdir d4/d12/d17/d9a/d131 0 2024-09-21T04:15:28.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/929: readlink d4/d12/d16/d66/d7a/d54/le7 0 2024-09-21T04:15:28.274 INFO:tasks.workunit.client.0.smithi110.stdout:6/930: stat d4/d12/d17/d9a/dbe/f10d 0 2024-09-21T04:15:28.275 INFO:tasks.workunit.client.0.smithi110.stdout:9/859: dread d7/d96/d99/d3c/f5a [0,4194304] 0 2024-09-21T04:15:28.275 INFO:tasks.workunit.client.0.smithi110.stdout:9/860: dread - d7/d26/fb1 zero size 2024-09-21T04:15:28.275 INFO:tasks.workunit.client.0.smithi110.stdout:9/861: truncate d7/d96/d99/d3c/d14/f11e 188704 0 2024-09-21T04:15:28.276 INFO:tasks.workunit.client.0.smithi110.stdout:8/932: creat d0/d7/d23/d82/d85/d9b/f14e x:0 0 0 2024-09-21T04:15:28.277 INFO:tasks.workunit.client.0.smithi110.stdout:6/931: mknod d4/d12/d17/d9a/c132 0 2024-09-21T04:15:28.278 INFO:tasks.workunit.client.0.smithi110.stdout:9/862: unlink d7/d96/d99/d3c/d4f/d9f/d6f/ld9 0 2024-09-21T04:15:28.278 INFO:tasks.workunit.client.0.smithi110.stdout:9/863: write d7/d96/d99/d3c/d1f/d43/db5/fba [211002,37487] 0 2024-09-21T04:15:28.278 INFO:tasks.workunit.client.0.smithi110.stdout:8/933: mkdir d0/d30/d34/d14f 0 2024-09-21T04:15:28.279 INFO:tasks.workunit.client.0.smithi110.stdout:6/932: creat d4/d12/d17/d81/d90/dc3/f133 x:0 0 0 2024-09-21T04:15:28.280 INFO:tasks.workunit.client.0.smithi110.stdout:8/934: creat d0/d7/d23/d82/d85/d9b/d11b/f150 x:0 0 0 2024-09-21T04:15:28.281 INFO:tasks.workunit.client.0.smithi110.stdout:8/935: mknod d0/d7/d13/d46/df8/dbe/c151 0 2024-09-21T04:15:28.285 INFO:tasks.workunit.client.1.smithi148.stdout:8/964: sync 2024-09-21T04:15:28.285 INFO:tasks.workunit.client.0.smithi110.stdout:5/935: sync 2024-09-21T04:15:28.285 INFO:tasks.workunit.client.0.smithi110.stdout:7/875: sync 2024-09-21T04:15:28.285 INFO:tasks.workunit.client.0.smithi110.stdout:5/936: write d1/d49/d4d/d1e/d5a/d64/d7b/fb5 [648556,48701] 0 2024-09-21T04:15:28.286 INFO:tasks.workunit.client.0.smithi110.stdout:2/823: dread d2/d1a/d8b/d10b/d29/d19/fda [0,4194304] 0 2024-09-21T04:15:28.286 INFO:tasks.workunit.client.0.smithi110.stdout:3/867: dwrite d0/d12/d16/d1a/f86 [0,4194304] 0 2024-09-21T04:15:28.287 INFO:tasks.workunit.client.0.smithi110.stdout:5/937: mknod d1/dc/ded/d109/c13f 0 2024-09-21T04:15:28.288 INFO:tasks.workunit.client.0.smithi110.stdout:2/824: mknod d2/d81/dfa/d108/c10c 0 2024-09-21T04:15:28.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/868: mknod d0/d12/d16/d1a/d6f/dd9/c134 0 2024-09-21T04:15:28.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/938: rename d1/dc/fea to d1/d49/d4d/d1e/d4a/dfb/f140 0 2024-09-21T04:15:28.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/939: fdatasync d1/d49/d4d/d1e/d4a/f9f 0 2024-09-21T04:15:28.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/940: getdents d1/d49/de4 0 2024-09-21T04:15:28.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/941: fsync d1/d22/ffa 0 2024-09-21T04:15:28.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/942: chown d1/dc/de9/fee 0 1 2024-09-21T04:15:28.293 INFO:tasks.workunit.client.0.smithi110.stdout:8/936: write d0/d7/d23/d6c/d12e/f104 [629899,89856] 0 2024-09-21T04:15:28.293 INFO:tasks.workunit.client.0.smithi110.stdout:5/943: mknod d1/d49/d4d/d1e/d4a/dd4/dda/c141 0 2024-09-21T04:15:28.294 INFO:tasks.workunit.client.0.smithi110.stdout:8/937: mknod d0/d7/d23/d82/d85/c152 0 2024-09-21T04:15:28.295 INFO:tasks.workunit.client.0.smithi110.stdout:5/944: unlink d1/d39/f2d 0 2024-09-21T04:15:28.297 INFO:tasks.workunit.client.0.smithi110.stdout:5/945: mknod d1/d7/d18/d8f/c142 0 2024-09-21T04:15:28.297 INFO:tasks.workunit.client.1.smithi148.stdout:8/965: write dc/f7d [6959577,37667] 0 2024-09-21T04:15:28.298 INFO:tasks.workunit.client.0.smithi110.stdout:8/938: rmdir d0/d7/d89 39 2024-09-21T04:15:28.298 INFO:tasks.workunit.client.0.smithi110.stdout:8/939: getdents d0/d30/d34/d3e 0 2024-09-21T04:15:28.298 INFO:tasks.workunit.client.0.smithi110.stdout:8/940: readlink d0/d30/d54/l63 0 2024-09-21T04:15:28.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/966: mkdir dc/d13/d76/d50/db1/d132 0 2024-09-21T04:15:28.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/967: readlink dc/d13/d76/d5d/la3 0 2024-09-21T04:15:28.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/968: write dc/d13/d76/d5d/d78/f112 [760794,119717] 0 2024-09-21T04:15:28.299 INFO:tasks.workunit.client.0.smithi110.stdout:8/941: mkdir d0/d30/d54/d6b/d153 0 2024-09-21T04:15:28.301 INFO:tasks.workunit.client.0.smithi110.stdout:8/942: symlink d0/d30/d54/l154 0 2024-09-21T04:15:28.301 INFO:tasks.workunit.client.0.smithi110.stdout:8/943: write d0/d30/d34/f11a [596804,25584] 0 2024-09-21T04:15:28.302 INFO:tasks.workunit.client.1.smithi148.stdout:8/969: rename dc/d13/d39/d107 to dc/d13/d3c/d81/d98/de5/d11b/d133 0 2024-09-21T04:15:28.302 INFO:tasks.workunit.client.1.smithi148.stdout:8/970: creat dc/d13/d3c/d81/f134 x:0 0 0 2024-09-21T04:15:28.303 INFO:tasks.workunit.client.0.smithi110.stdout:8/944: dread d0/d7/d13/d4b/f134 [0,4194304] 0 2024-09-21T04:15:28.303 INFO:tasks.workunit.client.0.smithi110.stdout:8/945: creat d0/d7/d23/d82/d85/f155 x:0 0 0 2024-09-21T04:15:28.303 INFO:tasks.workunit.client.1.smithi148.stdout:8/971: mkdir dc/d13/d3c/d81/d9d/d135 0 2024-09-21T04:15:28.304 INFO:tasks.workunit.client.1.smithi148.stdout:8/972: creat dc/d13/d3c/d81/d85/da1/f136 x:0 0 0 2024-09-21T04:15:28.306 INFO:tasks.workunit.client.0.smithi110.stdout:3/869: read d0/fd3 [2970217,41680] 0 2024-09-21T04:15:28.308 INFO:tasks.workunit.client.0.smithi110.stdout:3/870: rename d0/d12/d16/c3f to d0/d51/d7b/c135 0 2024-09-21T04:15:28.314 INFO:tasks.workunit.client.0.smithi110.stdout:5/946: dread d1/d22/f27 [0,4194304] 0 2024-09-21T04:15:28.316 INFO:tasks.workunit.client.0.smithi110.stdout:9/864: dwrite d7/d10/fc0 [0,4194304] 0 2024-09-21T04:15:28.317 INFO:tasks.workunit.client.0.smithi110.stdout:6/933: dwrite d4/d12/d16/f122 [0,4194304] 0 2024-09-21T04:15:28.317 INFO:tasks.workunit.client.0.smithi110.stdout:2/825: dwrite d2/d1a/d8b/d10b/d65/f68 [0,4194304] 0 2024-09-21T04:15:28.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/947: dread d1/f4c [0,4194304] 0 2024-09-21T04:15:28.320 INFO:tasks.workunit.client.0.smithi110.stdout:9/865: rename d7/d96/d99/d3c/d1f/d43/l62 to d7/d96/d99/d3c/d4f/d9f/l129 0 2024-09-21T04:15:28.320 INFO:tasks.workunit.client.0.smithi110.stdout:9/866: chown d7/d96/d99/f52 98680 1 2024-09-21T04:15:28.322 INFO:tasks.workunit.client.0.smithi110.stdout:2/826: mkdir d2/d1a/d8b/d10b/d5f/d9d/d86/d10d 0 2024-09-21T04:15:28.323 INFO:tasks.workunit.client.0.smithi110.stdout:5/948: mkdir d1/d11f/d137/d143 0 2024-09-21T04:15:28.325 INFO:tasks.workunit.client.0.smithi110.stdout:3/871: dwrite d0/d12/d3c/f48 [0,4194304] 0 2024-09-21T04:15:28.325 INFO:tasks.workunit.client.0.smithi110.stdout:3/872: truncate d0/fe 5122094 0 2024-09-21T04:15:28.326 INFO:tasks.workunit.client.0.smithi110.stdout:2/827: creat d2/d1a/d60/f10e x:0 0 0 2024-09-21T04:15:28.333 INFO:tasks.workunit.client.0.smithi110.stdout:2/828: mkdir d2/d1a/da3/db9/d10f 0 2024-09-21T04:15:28.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/934: dwrite d4/d86/f123 [0,4194304] 0 2024-09-21T04:15:28.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/973: dwrite dc/d13/ff9 [0,4194304] 0 2024-09-21T04:15:28.339 INFO:tasks.workunit.client.1.smithi148.stdout:8/974: chown dc/f41 838 1 2024-09-21T04:15:28.340 INFO:tasks.workunit.client.0.smithi110.stdout:6/935: creat d4/d12/d17/dfe/d12d/f134 x:0 0 0 2024-09-21T04:15:28.346 INFO:tasks.workunit.client.0.smithi110.stdout:9/867: rename d7/d96/d99/d3c to d7/d10/d12a 0 2024-09-21T04:15:28.348 INFO:tasks.workunit.client.0.smithi110.stdout:3/873: rename d0/d12/d16/c36 to d0/d12/d3c/d5a/c136 0 2024-09-21T04:15:28.348 INFO:tasks.workunit.client.0.smithi110.stdout:9/868: dread d7/d10/d12a/d1f/d72/f4d [0,4194304] 0 2024-09-21T04:15:28.349 INFO:tasks.workunit.client.0.smithi110.stdout:9/869: write d7/d10/d12a/d1f/d72/f4d [1169947,117250] 0 2024-09-21T04:15:28.349 INFO:tasks.workunit.client.0.smithi110.stdout:9/870: write d7/d10/f12 [1223476,102271] 0 2024-09-21T04:15:28.349 INFO:tasks.workunit.client.0.smithi110.stdout:9/871: write d7/d10/d88/f104 [581458,44512] 0 2024-09-21T04:15:28.350 INFO:tasks.workunit.client.0.smithi110.stdout:3/874: mkdir d0/d12/d16/d1a/dc6/d137 0 2024-09-21T04:15:28.350 INFO:tasks.workunit.client.0.smithi110.stdout:3/875: truncate d0/d12/d2a/d80/da2/da7/f102 482489 0 2024-09-21T04:15:28.351 INFO:tasks.workunit.client.0.smithi110.stdout:6/936: write d4/f6c [1005417,76635] 0 2024-09-21T04:15:28.355 INFO:tasks.workunit.client.0.smithi110.stdout:2/829: dwrite d2/d1a/d8b/d10b/f5d [4194304,4194304] 0 2024-09-21T04:15:28.358 INFO:tasks.workunit.client.0.smithi110.stdout:3/876: write d0/d12/d16/d1a/d95/f10d [1635402,32120] 0 2024-09-21T04:15:28.360 INFO:tasks.workunit.client.0.smithi110.stdout:3/877: rmdir d0/d12/d2a/db4 39 2024-09-21T04:15:28.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/975: sync 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/872: dwrite d7/d10/f12 [0,4194304] 0 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.0.smithi110.stdout:9/873: chown d7/d10/d12a/d1f/d72/d59/l65 2584 1 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/937: dwrite d4/d12/d17/d81/d90/fd9 [0,4194304] 0 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.0.smithi110.stdout:2/830: dwrite d2/d1a/d8b/d10b/d2a/f61 [0,4194304] 0 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/938: truncate d4/d12/d17/d2e/d40/fd8 884155 0 2024-09-21T04:15:28.387 INFO:tasks.workunit.client.1.smithi148.stdout:8/976: rename dc/d13/d28/d62/fe8 to dc/d13/d39/df8/f137 0 2024-09-21T04:15:28.388 INFO:tasks.workunit.client.0.smithi110.stdout:9/874: mkdir d7/d10/d12a/d1f/d43/db5/d12b 0 2024-09-21T04:15:28.389 INFO:tasks.workunit.client.1.smithi148.stdout:8/977: getdents dc/d13/d3c/d81/db0/db7 0 2024-09-21T04:15:28.391 INFO:tasks.workunit.client.0.smithi110.stdout:2/831: mkdir d2/d1a/da3/db9/d10f/d110 0 2024-09-21T04:15:28.392 INFO:tasks.workunit.client.0.smithi110.stdout:6/939: creat d4/d12/d17/d9a/dfa/f135 x:0 0 0 2024-09-21T04:15:28.394 INFO:tasks.workunit.client.0.smithi110.stdout:9/875: rename d7/d10/d12a/d4f/d9f/d6f/d11f to d7/d10/d88/d8f/d12c 0 2024-09-21T04:15:28.394 INFO:tasks.workunit.client.0.smithi110.stdout:9/876: truncate d7/d10/d12a/d1f/d72/f4d 1730284 0 2024-09-21T04:15:28.395 INFO:tasks.workunit.client.0.smithi110.stdout:6/940: rename d4/d12/d17/d81/d90/fd9 to d4/d12/d16/d66/d7a/d54/f136 0 2024-09-21T04:15:28.396 INFO:tasks.workunit.client.1.smithi148.stdout:8/978: dread dc/d13/d3c/d81/d85/da1/fb8 [0,4194304] 0 2024-09-21T04:15:28.396 INFO:tasks.workunit.client.1.smithi148.stdout:8/979: chown dc/d13/d3c/f5f 536148160 1 2024-09-21T04:15:28.396 INFO:tasks.workunit.client.0.smithi110.stdout:9/877: creat d7/d10/d12a/d4f/d9f/d6f/d78/f12d x:0 0 0 2024-09-21T04:15:28.396 INFO:tasks.workunit.client.0.smithi110.stdout:9/878: fsync d7/d10/d88/f104 0 2024-09-21T04:15:28.397 INFO:tasks.workunit.client.0.smithi110.stdout:6/941: symlink d4/d12/d17/d9a/l137 0 2024-09-21T04:15:28.399 INFO:tasks.workunit.client.0.smithi110.stdout:6/942: mknod d4/d12/d17/d81/d90/dc3/c138 0 2024-09-21T04:15:28.405 INFO:tasks.workunit.client.0.smithi110.stdout:9/879: dread d7/d10/d12a/f20 [0,4194304] 0 2024-09-21T04:15:28.409 INFO:tasks.workunit.client.0.smithi110.stdout:6/943: dread d4/d78/feb [0,4194304] 0 2024-09-21T04:15:28.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/944: mkdir d4/d3c/d12e/d139 0 2024-09-21T04:15:28.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/945: dread - d4/d12/d16/d66/d7a/d9e/f10a zero size 2024-09-21T04:15:28.412 INFO:tasks.workunit.client.1.smithi148.stdout:8/980: dwrite dc/d13/d3c/d81/d98/de5/fee [0,4194304] 0 2024-09-21T04:15:28.412 INFO:tasks.workunit.client.1.smithi148.stdout:8/981: truncate dc/d3e/ffd 844212 0 2024-09-21T04:15:28.412 INFO:tasks.workunit.client.1.smithi148.stdout:8/982: write dc/d13/d39/fc6 [495347,78329] 0 2024-09-21T04:15:28.412 INFO:tasks.workunit.client.0.smithi110.stdout:2/832: dwrite d2/d50/f7f [0,4194304] 0 2024-09-21T04:15:28.413 INFO:tasks.workunit.client.0.smithi110.stdout:6/946: rename d4/c1b to d4/d12/d17/d117/c13a 0 2024-09-21T04:15:28.413 INFO:tasks.workunit.client.0.smithi110.stdout:2/833: write d2/d1a/d8b/d10b/d5f/d9d/fbf [69301,8072] 0 2024-09-21T04:15:28.413 INFO:tasks.workunit.client.0.smithi110.stdout:2/834: chown d2/d1a/d8b/d10b/d5f/d9d/d86/d9f/df7/f107 3871915 1 2024-09-21T04:15:28.415 INFO:tasks.workunit.client.0.smithi110.stdout:2/835: mkdir d2/d1a/d60/dfd/d111 0 2024-09-21T04:15:28.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/983: rename dc/d13/d3c/f9c to dc/d13/d3c/d81/db0/db7/d11d/f138 0 2024-09-21T04:15:28.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/984: chown dc/d3e/c63 609 1 2024-09-21T04:15:28.417 INFO:tasks.workunit.client.0.smithi110.stdout:2/836: getdents d2 0 2024-09-21T04:15:28.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/837: dread d2/d1a/f105 [4194304,4194304] 0 2024-09-21T04:15:28.419 INFO:tasks.workunit.client.0.smithi110.stdout:2/838: write d2/d1a/d8b/d10b/d5f/d9d/d86/d9f/fea [868147,116231] 0 2024-09-21T04:15:28.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/985: mkdir dc/d13/d3c/d81/d98/de5/d11b/d139 0 2024-09-21T04:15:28.420 INFO:tasks.workunit.client.0.smithi110.stdout:2/839: mkdir d2/d1a/d8b/d10b/d29/d112 0 2024-09-21T04:15:28.420 INFO:tasks.workunit.client.0.smithi110.stdout:9/880: dwrite d7/d10/d88/d8f/fd1 [0,4194304] 0 2024-09-21T04:15:28.420 INFO:tasks.workunit.client.0.smithi110.stdout:2/840: getdents d2/d1a/d8b 0 2024-09-21T04:15:28.422 INFO:tasks.workunit.client.0.smithi110.stdout:9/881: mkdir d7/d10/d88/d8f/d12e 0 2024-09-21T04:15:28.422 INFO:tasks.workunit.client.0.smithi110.stdout:9/882: dread - d7/d10/d12a/d14/fc7 zero size 2024-09-21T04:15:28.423 INFO:tasks.workunit.client.0.smithi110.stdout:6/947: write d4/d12/d17/d2e/d38/f4f [3145357,59935] 0 2024-09-21T04:15:28.423 INFO:tasks.workunit.client.0.smithi110.stdout:6/948: getdents d4/d12/d17/d81/d90/dc3/df1 0 2024-09-21T04:15:28.425 INFO:tasks.workunit.client.1.smithi148.stdout:8/986: write dc/d13/d3c/d81/d98/de5/fee [2203599,9872] 0 2024-09-21T04:15:28.427 INFO:tasks.workunit.client.0.smithi110.stdout:6/949: symlink d4/d78/l13b 0 2024-09-21T04:15:28.427 INFO:tasks.workunit.client.0.smithi110.stdout:6/950: truncate d4/d12/d17/d9a/dfa/f135 375850 0 2024-09-21T04:15:28.428 INFO:tasks.workunit.client.0.smithi110.stdout:9/883: rmdir d7/d96 39 2024-09-21T04:15:28.433 INFO:tasks.workunit.client.0.smithi110.stdout:6/951: unlink d4/d12/d17/d9a/dbe/c11f 0 2024-09-21T04:15:28.434 INFO:tasks.workunit.client.0.smithi110.stdout:6/952: read - d4/d12/d16/d6d/f12c zero size 2024-09-21T04:15:28.434 INFO:tasks.workunit.client.0.smithi110.stdout:9/884: symlink d7/d10/d12a/d1f/d43/db5/d12b/l12f 0 2024-09-21T04:15:28.437 INFO:tasks.workunit.client.0.smithi110.stdout:6/953: link d4/d12/d17/f124 d4/d86/f13c 0 2024-09-21T04:15:28.437 INFO:tasks.workunit.client.0.smithi110.stdout:6/954: truncate d4/d12/d17/d2e/dfb/d116/fd1 907500 0 2024-09-21T04:15:28.437 INFO:tasks.workunit.client.0.smithi110.stdout:6/955: readlink d4/d12/d16/d66/d7a/d54/l9c 0 2024-09-21T04:15:28.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/885: mkdir d7/d10/d12a/d1f/d72/d130 0 2024-09-21T04:15:28.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/886: fsync d7/d10/d12a/d1f/d43/db5/dca/fea 0 2024-09-21T04:15:28.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/887: chown d7/d10/d12a/d1f/d43/db5/d12b/l12f 70724 1 2024-09-21T04:15:28.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/888: fsync d7/d10/d12a/d4f/f64 0 2024-09-21T04:15:28.439 INFO:tasks.workunit.client.0.smithi110.stdout:6/956: symlink d4/d12/d17/d2e/d38/l13d 0 2024-09-21T04:15:28.439 INFO:tasks.workunit.client.0.smithi110.stdout:6/957: fdatasync d4/d12/d17/d2e/d38/f98 0 2024-09-21T04:15:28.446 INFO:tasks.workunit.client.0.smithi110.stdout:9/889: creat d7/d10/d12a/dae/f131 x:0 0 0 2024-09-21T04:15:28.448 INFO:tasks.workunit.client.0.smithi110.stdout:9/890: mkdir d7/d10/d12a/d1f/d68/d132 0 2024-09-21T04:15:28.449 INFO:tasks.workunit.client.0.smithi110.stdout:6/958: rmdir d4/d12/d17/d81/d90/dc3 39 2024-09-21T04:15:28.449 INFO:tasks.workunit.client.0.smithi110.stdout:6/959: dread - d4/d12/d17/f124 zero size 2024-09-21T04:15:28.449 INFO:tasks.workunit.client.0.smithi110.stdout:6/960: fsync d4/f6a 0 2024-09-21T04:15:28.451 INFO:tasks.workunit.client.0.smithi110.stdout:9/891: symlink d7/d10/d12a/d4f/d9f/d6f/l133 0 2024-09-21T04:15:28.454 INFO:tasks.workunit.client.0.smithi110.stdout:6/961: link d4/d12/d17/d2e/d40/dad/lf9 d4/d12/l13e 0 2024-09-21T04:15:28.454 INFO:tasks.workunit.client.0.smithi110.stdout:6/962: chown d4/d12/d17/d2e/dfb/d116/f84 2 1 2024-09-21T04:15:28.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/987: dwrite dc/d13/dbd/fde [0,4194304] 0 2024-09-21T04:15:28.455 INFO:tasks.workunit.client.0.smithi110.stdout:6/963: unlink d4/d12/d17/d117/d76/l91 0 2024-09-21T04:15:28.457 INFO:tasks.workunit.client.1.smithi148.stdout:8/988: mkdir dc/d13/d76/d50/df0/d13a 0 2024-09-21T04:15:28.459 INFO:tasks.workunit.client.1.smithi148.stdout:8/989: dread dc/d13/d3c/d81/db0/db7/d11d/f138 [0,4194304] 0 2024-09-21T04:15:28.465 INFO:tasks.workunit.client.0.smithi110.stdout:6/964: write d4/d12/d16/d6d/f87 [3907821,102209] 0 2024-09-21T04:15:28.486 INFO:tasks.workunit.client.1.smithi148.stdout:8/990: dwrite dc/d13/d3c/d81/db0/db7/d11d/f86 [0,4194304] 0 2024-09-21T04:15:28.487 INFO:tasks.workunit.client.0.smithi110.stdout:6/965: dwrite d4/d12/d16/f56 [0,4194304] 0 2024-09-21T04:15:28.501 INFO:tasks.workunit.client.1.smithi148.stdout:8/991: dwrite dc/d13/d3c/f93 [0,4194304] 0 2024-09-21T04:15:28.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/992: symlink dc/d13/d3c/d81/d9d/l13b 0 2024-09-21T04:15:28.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/993: write dc/d13/d39/f51 [682556,117580] 0 2024-09-21T04:15:28.512 INFO:tasks.workunit.client.0.smithi110.stdout:6/966: dwrite d4/d12/d16/d66/d7a/d54/f136 [0,4194304] 0 2024-09-21T04:15:28.512 INFO:tasks.workunit.client.0.smithi110.stdout:6/967: read d4/d12/d17/d9a/dbe/f10d [1968564,126847] 0 2024-09-21T04:15:28.532 INFO:tasks.workunit.client.0.smithi110.stdout:6/968: dwrite d4/d12/d16/ff6 [0,4194304] 0 2024-09-21T04:15:28.533 INFO:tasks.workunit.client.0.smithi110.stdout:6/969: truncate d4/d12/d16/d66/d7a/d9e/ff2 960837 0 2024-09-21T04:15:28.533 INFO:tasks.workunit.client.0.smithi110.stdout:6/970: mknod d4/d12/d17/d117/c13f 0 2024-09-21T04:15:28.535 INFO:tasks.workunit.client.0.smithi110.stdout:6/971: creat d4/d12/d16/d66/d7a/d4b/d110/f140 x:0 0 0 2024-09-21T04:15:28.535 INFO:tasks.workunit.client.0.smithi110.stdout:6/972: chown d4/d12/d16/f7e 3563181 1 2024-09-21T04:15:28.536 INFO:tasks.workunit.client.0.smithi110.stdout:6/973: link d4/d12/d17/d2e/d38/c57 d4/d12/d16/d66/d7a/d4b/d7f/c141 0 2024-09-21T04:15:28.537 INFO:tasks.workunit.client.0.smithi110.stdout:6/974: truncate d4/d12/d17/d2e/d38/f4f 2558109 0 2024-09-21T04:15:28.550 INFO:tasks.workunit.client.0.smithi110.stdout:6/975: dwrite d4/d12/d16/d66/d7a/d83/fa3 [0,4194304] 0 2024-09-21T04:15:28.568 INFO:tasks.workunit.client.1.smithi148.stdout:8/994: sync 2024-09-21T04:15:28.568 INFO:tasks.workunit.client.1.smithi148.stdout:8/995: fsync dc/d13/d39/ff2 0 2024-09-21T04:15:28.572 INFO:tasks.workunit.client.0.smithi110.stdout:6/976: dwrite d4/d12/d17/d2e/dfb/d116/fda [0,4194304] 0 2024-09-21T04:15:28.573 INFO:tasks.workunit.client.0.smithi110.stdout:6/977: creat d4/d12/d16/d66/d7a/d4b/d7f/d113/f142 x:0 0 0 2024-09-21T04:15:28.574 INFO:tasks.workunit.client.0.smithi110.stdout:6/978: write d4/d12/d17/d2e/dfb/d116/fb4 [5175966,22003] 0 2024-09-21T04:15:28.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/979: truncate d4/d12/d16/d66/ffc 171496 0 2024-09-21T04:15:28.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/980: creat d4/d12/d17/d9a/f143 x:0 0 0 2024-09-21T04:15:28.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/981: read d4/d12/d16/d66/d7a/d83/fe0 [492027,37924] 0 2024-09-21T04:15:28.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/982: stat d4/d12/d16/d66/d7a/d54/c5f 0 2024-09-21T04:15:28.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/983: fdatasync d4/d12/d17/d9a/dfa/f135 0 2024-09-21T04:15:28.578 INFO:tasks.workunit.client.0.smithi110.stdout:6/984: creat d4/d3c/d12e/d139/f144 x:0 0 0 2024-09-21T04:15:28.578 INFO:tasks.workunit.client.0.smithi110.stdout:6/985: chown d4/d12/d16/d66/d7a/d83/lef 36 1 2024-09-21T04:15:28.587 INFO:tasks.workunit.client.1.smithi148.stdout:8/996: dwrite dc/f14 [4194304,4194304] 0 2024-09-21T04:15:28.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/997: link dc/d13/d3c/d81/f115 dc/d13/d3c/d81/d98/dc1/f13c 0 2024-09-21T04:15:28.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/998: read dc/d13/d3c/d81/d98/de5/fee [1883140,19317] 0 2024-09-21T04:15:28.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/999: chown dc/d13/dbd/fde 699724484 1 2024-09-21T04:15:28.592 INFO:tasks.workunit.client.1.smithi148.stderr:+ rm -rf -- ./tmp.9ZUkyyQeE2 2024-09-21T04:15:28.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/986: dwrite d4/d12/d17/f8e [0,4194304] 0 2024-09-21T04:15:28.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/987: fsync d4/d12/d16/d66/d7a/d4b/d7f/d89/d95/d99/dca/f11c 0 2024-09-21T04:15:28.600 INFO:tasks.workunit.client.0.smithi110.stdout:6/988: chown d4/d12/d16/d66/d7a/d4b/d7f/d113/c11d 0 1 2024-09-21T04:15:28.611 INFO:tasks.workunit.client.0.smithi110.stdout:6/989: dread d4/d12/f34 [0,4194304] 0 2024-09-21T04:15:28.611 INFO:tasks.workunit.client.0.smithi110.stdout:6/990: chown d4/d12/d16/d66/fac 6786719 1 2024-09-21T04:15:28.615 INFO:tasks.workunit.client.0.smithi110.stdout:6/991: dread d4/d12/d16/f59 [0,4194304] 0 2024-09-21T04:15:28.617 INFO:tasks.workunit.client.0.smithi110.stdout:6/992: symlink d4/d12/d17/d2e/l145 0 2024-09-21T04:15:28.620 INFO:tasks.workunit.client.0.smithi110.stdout:6/993: rename d4/d12/d17/d2e/d40/l45 to d4/d12/d17/l146 0 2024-09-21T04:15:28.620 INFO:tasks.workunit.client.0.smithi110.stdout:6/994: chown d4/d12/d17/d2e/d40/l3b 633880 1 2024-09-21T04:15:28.622 INFO:tasks.workunit.client.0.smithi110.stdout:6/995: creat d4/d12/d17/d81/f147 x:0 0 0 2024-09-21T04:15:28.627 INFO:tasks.workunit.client.0.smithi110.stdout:6/996: rmdir d4/d12/d16/d6d 39 2024-09-21T04:15:28.627 INFO:tasks.workunit.client.0.smithi110.stdout:6/997: dread - d4/d3c/d12e/d139/f144 zero size 2024-09-21T04:15:28.627 INFO:tasks.workunit.client.0.smithi110.stdout:6/998: write d4/d12/d17/d81/f147 [814401,108260] 0 2024-09-21T04:15:28.640 INFO:tasks.workunit.client.0.smithi110.stdout:6/999: dwrite d4/d12/d16/d66/d7a/d9e/ff2 [0,4194304] 0 2024-09-21T04:15:28.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/876: sync 2024-09-21T04:15:28.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/877: chown dd/d1c/c43 0 1 2024-09-21T04:15:28.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/878: chown dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/ca3 3 1 2024-09-21T04:15:28.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/879: mknod dd/d1c/dad/db2/d4a/da5/d111/c120 0 2024-09-21T04:15:28.712 INFO:tasks.workunit.client.0.smithi110.stdout:7/880: dwrite dd/d4d/dee/dbc/f106 [0,4194304] 0 2024-09-21T04:15:28.716 INFO:tasks.workunit.client.0.smithi110.stdout:7/881: rename dd/d1c/dad/db2/d56/d66/da6/d7a/d9b/la8 to dd/d4d/l121 0 2024-09-21T04:15:28.716 INFO:tasks.workunit.client.0.smithi110.stdout:7/882: creat dd/d48/f122 x:0 0 0 2024-09-21T04:15:28.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/883: rmdir dd/d14/d17/d8a/dd2 39 2024-09-21T04:15:28.725 INFO:tasks.workunit.client.0.smithi110.stdout:7/884: rmdir dd/d1c/dad/db2/d56/d66/dea 0 2024-09-21T04:15:28.725 INFO:tasks.workunit.client.0.smithi110.stdout:7/885: creat dd/d1c/dad/db2/d56/d66/da6/d7a/f123 x:0 0 0 2024-09-21T04:15:28.733 INFO:tasks.workunit.client.0.smithi110.stdout:9/892: sync 2024-09-21T04:15:28.733 INFO:tasks.workunit.client.0.smithi110.stdout:3/878: sync 2024-09-21T04:15:28.733 INFO:tasks.workunit.client.0.smithi110.stdout:5/949: sync 2024-09-21T04:15:28.733 INFO:tasks.workunit.client.0.smithi110.stdout:8/946: sync 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:8/947: write d0/d30/d9d/ff0 [256512,38368] 0 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:2/841: sync 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:3/879: creat d0/f138 x:0 0 0 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:9/893: creat d7/d10/d12a/d14/f134 x:0 0 0 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:9/894: dread - d7/d10/d12a/d14/ff0 zero size 2024-09-21T04:15:28.734 INFO:tasks.workunit.client.0.smithi110.stdout:7/886: dread dd/d1c/f8c [0,4194304] 0 2024-09-21T04:15:28.736 INFO:tasks.workunit.client.0.smithi110.stdout:3/880: mkdir d0/d12/d2a/d80/dc4/d139 0 2024-09-21T04:15:28.737 INFO:tasks.workunit.client.0.smithi110.stdout:9/895: symlink d7/d10/d12a/d1f/d43/db5/dca/d120/l135 0 2024-09-21T04:15:28.739 INFO:tasks.workunit.client.0.smithi110.stdout:7/887: mkdir dd/d1c/dad/db2/d56/d124 0 2024-09-21T04:15:28.740 INFO:tasks.workunit.client.0.smithi110.stdout:3/881: mkdir d0/d12/d2a/d80/da2/db0/d116/d13a 0 2024-09-21T04:15:28.742 INFO:tasks.workunit.client.0.smithi110.stdout:8/948: dread d0/d7/d23/f33 [0,4194304] 0 2024-09-21T04:15:28.742 INFO:tasks.workunit.client.0.smithi110.stdout:8/949: write d0/d7/d13/d4b/ddc/f130 [449254,66281] 0 2024-09-21T04:15:28.743 INFO:tasks.workunit.client.0.smithi110.stdout:9/896: creat d7/d10/d12a/d4f/d101/f136 x:0 0 0 2024-09-21T04:15:28.743 INFO:tasks.workunit.client.0.smithi110.stdout:3/882: symlink d0/d12/d3c/d5a/d70/l13b 0 2024-09-21T04:15:28.745 INFO:tasks.workunit.client.0.smithi110.stdout:9/897: rename d7/d10/d12a/f20 to d7/d10/d12a/d35/f137 0 2024-09-21T04:15:28.745 INFO:tasks.workunit.client.0.smithi110.stdout:9/898: write d7/d10/d88/f104 [1500235,37396] 0 2024-09-21T04:15:28.748 INFO:tasks.workunit.client.0.smithi110.stdout:9/899: mkdir d7/d10/d12a/d4f/d9f/d6f/d78/d138 0 2024-09-21T04:15:28.748 INFO:tasks.workunit.client.0.smithi110.stdout:9/900: write d7/d10/d12a/d4f/f64 [1515907,47682] 0 2024-09-21T04:15:28.748 INFO:tasks.workunit.client.0.smithi110.stdout:9/901: chown d7/d10/d12a/d4f/l117 7 1 2024-09-21T04:15:28.749 INFO:tasks.workunit.client.0.smithi110.stdout:9/902: mknod d7/d10/d12a/d4f/d9f/d6f/d108/c139 0 2024-09-21T04:15:28.754 INFO:tasks.workunit.client.0.smithi110.stdout:7/888: dread dd/d14/f46 [0,4194304] 0 2024-09-21T04:15:28.757 INFO:tasks.workunit.client.0.smithi110.stdout:7/889: rename dd/d1c/dad/db2/d56/d66/da6/d7a/lfc to dd/d1c/l125 0 2024-09-21T04:15:28.759 INFO:tasks.workunit.client.0.smithi110.stdout:7/890: rename dd/d14/c16 to dd/d4d/c126 0 2024-09-21T04:15:28.765 INFO:tasks.workunit.client.0.smithi110.stdout:2/842: dwrite d2/d1a/d8b/d10b/d5f/d9d/f109 [0,4194304] 0 2024-09-21T04:15:28.765 INFO:tasks.workunit.client.0.smithi110.stdout:7/891: unlink dd/d1c/dad/db2/d4a/da5/ffd 0 2024-09-21T04:15:28.769 INFO:tasks.workunit.client.0.smithi110.stdout:2/843: symlink d2/d1a/d8b/d10b/l113 0 2024-09-21T04:15:28.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/892: mkdir dd/df3/d127 0 2024-09-21T04:15:28.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/893: fdatasync dd/d14/d17/fc3 0 2024-09-21T04:15:28.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/894: chown dd/d14/d17/fb9 2369258 1 2024-09-21T04:15:28.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/844: link d2/d1a/d8b/d10b/d5f/f6f d2/d50/d56/f114 0 2024-09-21T04:15:28.772 INFO:tasks.workunit.client.0.smithi110.stdout:8/950: dwrite d0/d7/d13/d46/f14b [0,4194304] 0 2024-09-21T04:15:28.772 INFO:tasks.workunit.client.0.smithi110.stdout:8/951: dread - d0/d30/d34/d3e/d5d/dd7/dda/d119/f129 zero size 2024-09-21T04:15:28.773 INFO:tasks.workunit.client.0.smithi110.stdout:7/895: unlink dd/d1c/f75 0 2024-09-21T04:15:28.776 INFO:tasks.workunit.client.0.smithi110.stdout:5/950: dwrite d1/d22/f27 [0,4194304] 0 2024-09-21T04:15:28.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/952: link d0/d7/d13/d46/l11c d0/d7/d23/d52/l156 0 2024-09-21T04:15:28.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/953: write d0/d7/d13/d4b/f1f [5776830,72152] 0 2024-09-21T04:15:28.778 INFO:tasks.workunit.client.0.smithi110.stdout:3/883: dwrite d0/d12/d2a/d80/da2/db0/f107 [0,4194304] 0 2024-09-21T04:15:28.780 INFO:tasks.workunit.client.0.smithi110.stdout:5/951: unlink d1/d49/d4d/d1e/d4a/f83 0 2024-09-21T04:15:28.782 INFO:tasks.workunit.client.0.smithi110.stdout:8/954: unlink d0/d30/ccf 0 2024-09-21T04:15:28.787 INFO:tasks.workunit.client.0.smithi110.stdout:2/845: write d2/d1a/d8b/d10b/d5f/d9d/f78 [2507158,82860] 0 2024-09-21T04:15:28.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/955: symlink d0/d7/d13/d46/df8/l157 0 2024-09-21T04:15:28.790 INFO:tasks.workunit.client.0.smithi110.stdout:5/952: write d1/d49/d6f/fc4 [1154379,95813] 0 2024-09-21T04:15:28.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/896: dwrite dd/d48/f11e [0,4194304] 0 2024-09-21T04:15:28.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/897: fdatasync dd/fc6 0 2024-09-21T04:15:28.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/956: unlink d0/d30/d34/d3e/d5d/cd4 0 2024-09-21T04:15:28.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/957: creat d0/d30/d34/d3e/d100/f158 x:0 0 0 2024-09-21T04:15:28.795 INFO:tasks.workunit.client.0.smithi110.stdout:3/884: rename d0/d12/d3c/d5a/d70/d81 to d0/d12/d16/d1a/d13c 0 2024-09-21T04:15:28.797 INFO:tasks.workunit.client.0.smithi110.stdout:5/953: mknod d1/c144 0 2024-09-21T04:15:28.801 INFO:tasks.workunit.client.0.smithi110.stdout:7/898: symlink dd/d4d/dee/dbc/l128 0 2024-09-21T04:15:28.801 INFO:tasks.workunit.client.0.smithi110.stdout:7/899: write dd/d14/f46 [4285614,95367] 0 2024-09-21T04:15:28.802 INFO:tasks.workunit.client.0.smithi110.stdout:2/846: dwrite d2/d1a/d8b/d10b/fd1 [0,4194304] 0 2024-09-21T04:15:28.803 INFO:tasks.workunit.client.0.smithi110.stdout:8/958: write d0/d7/d89/f99 [1771451,31722] 0 2024-09-21T04:15:28.803 INFO:tasks.workunit.client.0.smithi110.stdout:5/954: creat d1/d7/d18/d8f/dc2/f145 x:0 0 0 2024-09-21T04:15:28.809 INFO:tasks.workunit.client.0.smithi110.stdout:2/847: mknod d2/d1a/d8b/d10b/d5f/d9d/d86/d9f/df4/c115 0 2024-09-21T04:15:28.810 INFO:tasks.workunit.client.0.smithi110.stdout:5/955: mknod d1/d22/db6/c146 0 2024-09-21T04:15:28.813 INFO:tasks.workunit.client.0.smithi110.stdout:5/956: symlink d1/d49/d4d/d1e/d4a/dd4/l147 0 2024-09-21T04:15:28.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/885: dwrite d0/d12/d2a/d80/da2/da7/f102 [0,4194304] 0 2024-09-21T04:15:28.816 INFO:tasks.workunit.client.0.smithi110.stdout:5/957: dread d1/d7/d58/f77 [0,4194304] 0 2024-09-21T04:15:28.816 INFO:tasks.workunit.client.0.smithi110.stdout:5/958: fdatasync d1/d49/de4/df9/f10c 0 2024-09-21T04:15:28.816 INFO:tasks.workunit.client.0.smithi110.stdout:8/959: dwrite d0/d7/d13/d4b/f4e [0,4194304] 0 2024-09-21T04:15:28.817 INFO:tasks.workunit.client.0.smithi110.stdout:7/900: rename dd/d4d/dc1 to dd/d1c/dad/db2/d4a/d129 0 2024-09-21T04:15:28.823 INFO:tasks.workunit.client.0.smithi110.stdout:3/886: creat d0/d51/f13d x:0 0 0 2024-09-21T04:15:28.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/887: dread d0/d12/d2a/fdd [0,4194304] 0 2024-09-21T04:15:28.829 INFO:tasks.workunit.client.0.smithi110.stdout:8/960: dwrite d0/d30/d9d/ff0 [0,4194304] 0 2024-09-21T04:15:28.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/888: link d0/d12/d16/d1a/dac/lc7 d0/d12/d3c/dad/d12f/l13e 0 2024-09-21T04:15:28.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/889: readlink d0/d12/d2a/d80/d9c/lc2 0 2024-09-21T04:15:28.834 INFO:tasks.workunit.client.0.smithi110.stdout:8/961: symlink d0/d30/d34/d3e/d60/dec/l159 0 2024-09-21T04:15:28.835 INFO:tasks.workunit.client.0.smithi110.stdout:2/848: rename d2/d1a to d2/de7/d116 0 2024-09-21T04:15:28.836 INFO:tasks.workunit.client.0.smithi110.stdout:3/890: mknod d0/d12/d3c/c13f 0 2024-09-21T04:15:28.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/962: creat d0/d30/d34/d3e/d32/f15a x:0 0 0 2024-09-21T04:15:28.840 INFO:tasks.workunit.client.0.smithi110.stdout:5/959: rename d1/d49/d4d/f48 to d1/d49/d4d/d1e/d4a/dae/f148 0 2024-09-21T04:15:28.840 INFO:tasks.workunit.client.0.smithi110.stdout:5/960: chown d1/f8d 82459727 1 2024-09-21T04:15:28.840 INFO:tasks.workunit.client.0.smithi110.stdout:5/961: chown d1/d22/db6/dc6/d12b/d12c 651 1 2024-09-21T04:15:28.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/962: link d1/l75 d1/d11f/l149 0 2024-09-21T04:15:28.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/963: chown d1/d49/l111 11165 1 2024-09-21T04:15:28.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/964: getdents d1/d22/db6/dc6/d12b/d123 0 2024-09-21T04:15:28.843 INFO:tasks.workunit.client.0.smithi110.stdout:5/965: chown d1/d7/d18/f8b 1 1 2024-09-21T04:15:28.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/849: link d2/de7/d116/d8b/d10b/cb2 d2/de7/d116/d8b/d10b/d29/d19/c117 0 2024-09-21T04:15:28.846 INFO:tasks.workunit.client.0.smithi110.stdout:3/891: read d0/d12/d16/d1a/d5c/f79 [498479,4836] 0 2024-09-21T04:15:28.846 INFO:tasks.workunit.client.0.smithi110.stdout:3/892: write d0/d12/d3c/dad/d11f/f132 [787189,56473] 0 2024-09-21T04:15:28.846 INFO:tasks.workunit.client.0.smithi110.stdout:3/893: creat d0/d12/d3c/d5a/d70/dab/f140 x:0 0 0 2024-09-21T04:15:28.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/850: dread d2/de7/d116/d8b/da5/fdf [0,4194304] 0 2024-09-21T04:15:28.850 INFO:tasks.workunit.client.0.smithi110.stdout:3/894: link d0/d12/d2a/c4c d0/d12/d16/d1a/dc6/d137/c141 0 2024-09-21T04:15:28.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/895: mkdir d0/d12/d2a/d80/da2/d142 0 2024-09-21T04:15:28.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/896: stat d0/d12/d3c/ff4 0 2024-09-21T04:15:28.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/897: chown d0/d12/d3c/d5a/d65/f75 2 1 2024-09-21T04:15:28.864 INFO:tasks.workunit.client.0.smithi110.stdout:2/851: dread d2/de7/d116/d8b/d10b/d5f/d9d/f109 [0,4194304] 0 2024-09-21T04:15:28.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/852: creat d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/f118 x:0 0 0 2024-09-21T04:15:28.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/853: write d2/d50/d56/f94 [770827,115781] 0 2024-09-21T04:15:28.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/854: mknod d2/de7/d116/d60/dfd/d111/c119 0 2024-09-21T04:15:28.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/855: readlink d2/lc5 0 2024-09-21T04:15:28.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/856: chown d2/le6 4 1 2024-09-21T04:15:28.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/857: truncate d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df7/f107 1022346 0 2024-09-21T04:15:28.867 INFO:tasks.workunit.client.0.smithi110.stdout:8/963: dwrite d0/d7/d13/d46/f4c [0,4194304] 0 2024-09-21T04:15:28.867 INFO:tasks.workunit.client.0.smithi110.stdout:8/964: fsync d0/d7/d13/d46/f106 0 2024-09-21T04:15:28.867 INFO:tasks.workunit.client.0.smithi110.stdout:5/966: dwrite d1/fa7 [0,4194304] 0 2024-09-21T04:15:28.867 INFO:tasks.workunit.client.0.smithi110.stdout:5/967: chown d1/d7/d58/d72/dcb 61936605 1 2024-09-21T04:15:28.868 INFO:tasks.workunit.client.0.smithi110.stdout:2/858: getdents d2/de7/d116/da3/db9/d10f 0 2024-09-21T04:15:28.869 INFO:tasks.workunit.client.0.smithi110.stdout:8/965: creat d0/f15b x:0 0 0 2024-09-21T04:15:28.874 INFO:tasks.workunit.client.0.smithi110.stdout:8/966: mknod d0/d7/d23/c15c 0 2024-09-21T04:15:28.877 INFO:tasks.workunit.client.0.smithi110.stdout:8/967: mkdir d0/d7/d13/d4b/d15d 0 2024-09-21T04:15:28.878 INFO:tasks.workunit.client.0.smithi110.stdout:2/859: dread d2/de7/d116/d8b/d10b/d65/f75 [4194304,4194304] 0 2024-09-21T04:15:28.879 INFO:tasks.workunit.client.0.smithi110.stdout:8/968: truncate d0/d30/d34/d3e/f88 4025311 0 2024-09-21T04:15:28.880 INFO:tasks.workunit.client.0.smithi110.stdout:2/860: write d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/fea [676100,42405] 0 2024-09-21T04:15:28.881 INFO:tasks.workunit.client.0.smithi110.stdout:8/969: symlink d0/d30/d34/d3e/d5d/l15e 0 2024-09-21T04:15:28.883 INFO:tasks.workunit.client.0.smithi110.stdout:5/968: dwrite d1/d49/d4d/d1e/d4a/dd4/dd5/f118 [0,4194304] 0 2024-09-21T04:15:28.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/861: truncate d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9b/fb4 2327216 0 2024-09-21T04:15:28.884 INFO:tasks.workunit.client.0.smithi110.stdout:8/970: mkdir d0/d7/d89/d10f/d14c/d15f 0 2024-09-21T04:15:28.884 INFO:tasks.workunit.client.0.smithi110.stdout:8/971: creat d0/d30/d34/d3e/d32/da3/d117/f160 x:0 0 0 2024-09-21T04:15:28.886 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 282 MiB/s wr, 420 op/s 2024-09-21T04:15:28.886 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:28.886 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.886 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.887 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.888 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi148 to 2240M 2024-09-21T04:15:28.888 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:28 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.893 INFO:tasks.workunit.client.0.smithi110.stdout:5/969: symlink d1/d11f/l14a 0 2024-09-21T04:15:28.896 INFO:tasks.workunit.client.0.smithi110.stdout:5/970: mkdir d1/d13a/d14b 0 2024-09-21T04:15:28.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/971: rename d1/d7/d58/f84 to d1/d49/d4d/d1e/d5a/d64/f14c 0 2024-09-21T04:15:28.898 INFO:tasks.workunit.client.0.smithi110.stdout:5/972: chown d1/l91 49052 1 2024-09-21T04:15:28.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/862: dwrite d2/de7/d116/d8b/d10b/d65/f75 [4194304,4194304] 0 2024-09-21T04:15:28.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/863: readlink d2/l93 0 2024-09-21T04:15:28.900 INFO:tasks.workunit.client.0.smithi110.stdout:5/973: symlink d1/d49/d4d/d1e/d4a/dd4/d129/l14d 0 2024-09-21T04:15:28.900 INFO:tasks.workunit.client.0.smithi110.stdout:2/864: write d2/d50/ff6 [838993,14618] 0 2024-09-21T04:15:28.900 INFO:tasks.workunit.client.0.smithi110.stdout:5/974: read d1/d22/fb9 [274490,125275] 0 2024-09-21T04:15:28.901 INFO:tasks.workunit.client.0.smithi110.stdout:2/865: rename d2/de7/d116/d8b/d10b/d5f/d9d/d86/fae to d2/de7/d116/d8b/d10b/d5f/dc0/f11a 0 2024-09-21T04:15:28.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/866: truncate d2/de7/d116/d8b/d10b/d5f/fba 117968 0 2024-09-21T04:15:28.902 INFO:tasks.workunit.client.0.smithi110.stdout:5/975: mknod d1/d49/d4d/d1e/d4a/c14e 0 2024-09-21T04:15:28.906 INFO:tasks.workunit.client.0.smithi110.stdout:2/867: truncate d2/de7/d116/d8b/d10b/d29/f71 153054 0 2024-09-21T04:15:28.906 INFO:tasks.workunit.client.0.smithi110.stdout:2/868: stat d2/de7/d116/d8b/d10b/d5f/f2b 0 2024-09-21T04:15:28.906 INFO:tasks.workunit.client.0.smithi110.stdout:2/869: chown d2/d50/d56/f94 8845 1 2024-09-21T04:15:28.906 INFO:tasks.workunit.client.0.smithi110.stdout:2/870: write d2/de7/d116/d8b/d10b/d2a/fe0 [433953,44083] 0 2024-09-21T04:15:28.907 INFO:tasks.workunit.client.0.smithi110.stdout:8/972: dwrite d0/d7/d13/d4b/f5b [0,4194304] 0 2024-09-21T04:15:28.908 INFO:tasks.workunit.client.0.smithi110.stdout:5/976: mkdir d1/d7/d58/d72/dcb/d14f 0 2024-09-21T04:15:28.908 INFO:tasks.workunit.client.0.smithi110.stdout:5/977: dread - d1/d7/d18/d3f/d117/f11b zero size 2024-09-21T04:15:28.908 INFO:tasks.workunit.client.0.smithi110.stdout:5/978: chown d1/d49/fab 537765 1 2024-09-21T04:15:28.908 INFO:tasks.workunit.client.0.smithi110.stdout:8/973: dread d0/d7/d13/d4b/f27 [0,4194304] 0 2024-09-21T04:15:28.909 INFO:tasks.workunit.client.0.smithi110.stdout:2/871: symlink d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/l11b 0 2024-09-21T04:15:28.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/872: unlink d2/de7/d116/d8b/d10b/d5f/fe9 0 2024-09-21T04:15:28.919 INFO:tasks.workunit.client.0.smithi110.stdout:5/979: truncate d1/d7/f13 7102184 0 2024-09-21T04:15:28.920 INFO:tasks.workunit.client.0.smithi110.stdout:5/980: chown d1/d49/de4 2056 1 2024-09-21T04:15:28.920 INFO:tasks.workunit.client.0.smithi110.stdout:5/981: readlink d1/d22/l127 0 2024-09-21T04:15:28.922 INFO:tasks.workunit.client.0.smithi110.stdout:2/873: symlink d2/d50/d56/l11c 0 2024-09-21T04:15:28.922 INFO:tasks.workunit.client.0.smithi110.stdout:2/874: truncate d2/de7/d116/d8b/d10b/d2a/fe0 686895 0 2024-09-21T04:15:28.927 INFO:tasks.workunit.client.0.smithi110.stdout:2/875: mknod d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/c11d 0 2024-09-21T04:15:28.936 INFO:tasks.workunit.client.0.smithi110.stdout:2/876: dread d2/de7/d116/d8b/d10b/d5f/f24 [4194304,4194304] 0 2024-09-21T04:15:28.936 INFO:tasks.workunit.client.0.smithi110.stdout:5/982: unlink d1/d22/f3c 0 2024-09-21T04:15:28.937 INFO:tasks.workunit.client.0.smithi110.stdout:2/877: creat d2/d81/dfa/f11e x:0 0 0 2024-09-21T04:15:28.939 INFO:tasks.workunit.client.0.smithi110.stdout:5/983: rename d1/d7/d58/d72/dcb/c122 to d1/d49/de4/c150 0 2024-09-21T04:15:28.939 INFO:tasks.workunit.client.0.smithi110.stdout:5/984: chown d1/d39/fa4 682 1 2024-09-21T04:15:28.939 INFO:tasks.workunit.client.0.smithi110.stdout:5/985: fdatasync d1/d49/de4/df9/f106 0 2024-09-21T04:15:28.939 INFO:tasks.workunit.client.0.smithi110.stdout:5/986: fsync d1/fb1 0 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 282 MiB/s wr, 420 op/s 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:28.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi148 to 2240M 2024-09-21T04:15:28.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:28 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:28.941 INFO:tasks.workunit.client.0.smithi110.stdout:2/878: creat d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/f11f x:0 0 0 2024-09-21T04:15:28.942 INFO:tasks.workunit.client.0.smithi110.stdout:5/987: mkdir d1/d49/d4d/d1e/d4a/dae/df2/d151 0 2024-09-21T04:15:28.942 INFO:tasks.workunit.client.0.smithi110.stdout:5/988: stat d1/d49/d4d/d1e/d5a/d11e 0 2024-09-21T04:15:28.943 INFO:tasks.workunit.client.0.smithi110.stdout:2/879: rename d2/de7/d116/d8b/d10b/d29/c13 to d2/de7/d116/d8b/d10b/d29/d112/c120 0 2024-09-21T04:15:28.946 INFO:tasks.workunit.client.0.smithi110.stdout:5/989: unlink d1/d49/d4d/d1e/d4a/dae/cb0 0 2024-09-21T04:15:28.946 INFO:tasks.workunit.client.0.smithi110.stdout:5/990: write d1/d22/d6e/fcf [273501,103436] 0 2024-09-21T04:15:28.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/880: dwrite d2/de7/d116/d8b/d10b/d5f/d9d/d86/f103 [0,4194304] 0 2024-09-21T04:15:28.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/881: fdatasync d2/de7/d116/d8b/d10b/d5f/d9d/fbf 0 2024-09-21T04:15:28.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/882: fdatasync d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/fea 0 2024-09-21T04:15:28.983 INFO:tasks.workunit.client.0.smithi110.stdout:2/883: dwrite d2/de7/d116/d8b/d10b/d65/fa0 [0,4194304] 0 2024-09-21T04:15:28.983 INFO:tasks.workunit.client.0.smithi110.stdout:5/991: dwrite d1/d49/d4d/d1e/d5a/f67 [0,4194304] 0 2024-09-21T04:15:28.983 INFO:tasks.workunit.client.0.smithi110.stdout:5/992: write d1/d7/d18/d8f/dc2/fc3 [2174374,66010] 0 2024-09-21T04:15:28.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/884: link d2/d81/c85 d2/de7/d116/d8b/d10b/d29/d19/c121 0 2024-09-21T04:15:28.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/885: symlink d2/de7/d116/d8b/d10b/d5f/d9d/l122 0 2024-09-21T04:15:28.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/886: mknod d2/de7/d116/da3/db9/c123 0 2024-09-21T04:15:29.013 INFO:tasks.workunit.client.0.smithi110.stdout:5/993: dwrite d1/dc/fd6 [0,4194304] 0 2024-09-21T04:15:29.014 INFO:tasks.workunit.client.0.smithi110.stdout:2/887: dwrite d2/de7/d116/da3/fb1 [0,4194304] 0 2024-09-21T04:15:29.014 INFO:tasks.workunit.client.0.smithi110.stdout:2/888: truncate d2/de7/d116/d8b/d10b/d29/d19/fb7 1628070 0 2024-09-21T04:15:29.016 INFO:tasks.workunit.client.0.smithi110.stdout:2/889: unlink d2/de7/d116/da3/fb1 0 2024-09-21T04:15:29.018 INFO:tasks.workunit.client.0.smithi110.stdout:2/890: creat d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9b/f124 x:0 0 0 2024-09-21T04:15:29.034 INFO:tasks.workunit.client.0.smithi110.stdout:2/891: dwrite d2/de7/d116/d8b/d10b/d65/fe8 [0,4194304] 0 2024-09-21T04:15:29.034 INFO:tasks.workunit.client.0.smithi110.stdout:2/892: chown d2/de7/d116/lf3 835393 1 2024-09-21T04:15:29.051 INFO:tasks.workunit.client.0.smithi110.stdout:2/893: dwrite d2/d50/d56/f7a [0,4194304] 0 2024-09-21T04:15:29.054 INFO:tasks.workunit.client.0.smithi110.stdout:2/894: mkdir d2/de7/d116/da3/db9/d10f/d110/d125 0 2024-09-21T04:15:29.055 INFO:tasks.workunit.client.0.smithi110.stdout:2/895: mkdir d2/d50/d126 0 2024-09-21T04:15:29.057 INFO:tasks.workunit.client.0.smithi110.stdout:2/896: mkdir d2/de7/d127 0 2024-09-21T04:15:29.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/903: sync 2024-09-21T04:15:29.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/904: dread - d7/d10/d88/d8f/da0/fe6 zero size 2024-09-21T04:15:29.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/905: chown d7/d26/l30 7784692 1 2024-09-21T04:15:29.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/906: truncate d7/d10/d12a/d14/f134 180241 0 2024-09-21T04:15:29.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/907: chown d7/d10/d12a/d14/f134 94585 1 2024-09-21T04:15:29.177 INFO:tasks.workunit.client.0.smithi110.stdout:9/908: dread d7/d10/d12a/d11/f8b [0,4194304] 0 2024-09-21T04:15:29.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/909: getdents d7/d10/d88/d8f/da0 0 2024-09-21T04:15:29.196 INFO:tasks.workunit.client.0.smithi110.stdout:9/910: dwrite d7/d10/d12a/d14/d31/f6a [0,4194304] 0 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/901: sync 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/902: fdatasync dd/d4d/fa1 0 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/903: dread - dd/d4d/dee/dbc/fed zero size 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/904: readlink dd/d1c/dad/db2/d4a/l79 0 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/905: read dd/d4d/dee/fd3 [242927,27949] 0 2024-09-21T04:15:29.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/906: symlink dd/l12a 0 2024-09-21T04:15:29.451 INFO:tasks.workunit.client.0.smithi110.stdout:7/907: mknod dd/d1c/dad/db2/d56/d66/da6/c12b 0 2024-09-21T04:15:29.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/908: mkdir dd/d1c/dad/db2/d4a/da5/d111/d12c 0 2024-09-21T04:15:29.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/909: chown dd/d1c/dad/db2/d56/d66/da6/d7a/l11b 259 1 2024-09-21T04:15:29.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/910: link dd/d1c/lb0 dd/l12d 0 2024-09-21T04:15:29.456 INFO:tasks.workunit.client.0.smithi110.stdout:7/911: creat dd/d48/d9f/de9/f12e x:0 0 0 2024-09-21T04:15:29.456 INFO:tasks.workunit.client.0.smithi110.stdout:7/912: fdatasync dd/d14/d17/d8a/dd2/fe7 0 2024-09-21T04:15:29.466 INFO:tasks.workunit.client.0.smithi110.stdout:7/913: write dd/db1/ff1 [3272933,71703] 0 2024-09-21T04:15:29.466 INFO:tasks.workunit.client.0.smithi110.stdout:7/914: creat dd/d4d/f12f x:0 0 0 2024-09-21T04:15:29.466 INFO:tasks.workunit.client.0.smithi110.stdout:7/915: write dd/d1c/dad/db2/d56/f94 [1687016,50077] 0 2024-09-21T04:15:29.466 INFO:tasks.workunit.client.0.smithi110.stdout:7/916: chown dd/d1c/dad/db2/d56/c112 193935 1 2024-09-21T04:15:29.494 INFO:tasks.workunit.client.0.smithi110.stdout:7/917: getdents dd/d48/d9f/de9 0 2024-09-21T04:15:29.494 INFO:tasks.workunit.client.0.smithi110.stdout:7/918: fdatasync dd/d14/d17/f54 0 2024-09-21T04:15:29.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/919: dread dd/fae [0,4194304] 0 2024-09-21T04:15:29.523 INFO:tasks.workunit.client.0.smithi110.stdout:7/920: dwrite dd/d1c/f113 [0,4194304] 0 2024-09-21T04:15:29.526 INFO:tasks.workunit.client.0.smithi110.stdout:7/921: symlink dd/d1c/l130 0 2024-09-21T04:15:29.526 INFO:tasks.workunit.client.0.smithi110.stdout:7/922: fdatasync dd/d1c/dad/db2/f61 0 2024-09-21T04:15:29.529 INFO:tasks.workunit.client.0.smithi110.stdout:7/923: creat dd/d1c/dad/db2/d4a/d129/f131 x:0 0 0 2024-09-21T04:15:29.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/924: dread dd/d1c/dad/db2/f3c [0,4194304] 0 2024-09-21T04:15:29.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/925: rmdir dd/d1c/dad/db2/d4a/d115 39 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/974: sync 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:9/911: sync 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:2/897: sync 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:3/898: sync 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/975: truncate d0/d30/d34/f12f 1023466 0 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:5/994: sync 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/976: truncate d0/d30/d54/f98 4402578 0 2024-09-21T04:15:29.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/977: chown d0/d7/d6f/f14d 11279 1 2024-09-21T04:15:29.748 INFO:tasks.workunit.client.0.smithi110.stdout:3/899: write d0/d12/d3c/dce/fe1 [538684,74725] 0 2024-09-21T04:15:29.748 INFO:tasks.workunit.client.0.smithi110.stdout:3/900: chown d0/d12/d3c/d5a/lea 1747418 1 2024-09-21T04:15:29.753 INFO:tasks.workunit.client.0.smithi110.stdout:9/912: symlink d7/d96/d99/l13a 0 2024-09-21T04:15:29.757 INFO:tasks.workunit.client.0.smithi110.stdout:8/978: rename d0/d30/d34/d3e/d5d/dd7/fe0 to d0/d30/d54/d6b/d133/f161 0 2024-09-21T04:15:29.762 INFO:tasks.workunit.client.0.smithi110.stdout:3/901: rmdir d0/d12/d2a/d80/da2 39 2024-09-21T04:15:29.762 INFO:tasks.workunit.client.0.smithi110.stdout:3/902: fsync d0/d12/d16/d1a/d5c/faa 0 2024-09-21T04:15:29.762 INFO:tasks.workunit.client.0.smithi110.stdout:5/995: dwrite d1/d22/fb9 [0,4194304] 0 2024-09-21T04:15:29.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/996: dread d1/d7/d18/ff8 [0,4194304] 0 2024-09-21T04:15:29.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/997: write d1/d22/f97 [374905,62627] 0 2024-09-21T04:15:29.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/898: dwrite d2/de7/d116/d8b/d10b/d5f/f95 [0,4194304] 0 2024-09-21T04:15:29.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/913: rename d7/d96/d99/l2b to d7/d10/d12a/d35/d109/l13b 0 2024-09-21T04:15:29.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/914: chown d7/d96/fc3 7 1 2024-09-21T04:15:29.777 INFO:tasks.workunit.client.0.smithi110.stdout:8/979: creat d0/d7/d13/d46/df8/dbe/f162 x:0 0 0 2024-09-21T04:15:29.780 INFO:tasks.workunit.client.0.smithi110.stdout:3/903: mkdir d0/d12/d3c/d5a/d143 0 2024-09-21T04:15:29.782 INFO:tasks.workunit.client.0.smithi110.stdout:5/998: getdents d1/d7/d58/d72/dcb 0 2024-09-21T04:15:29.784 INFO:tasks.workunit.client.0.smithi110.stdout:2/899: creat d2/de7/d116/d8b/d10b/d29/d19/f128 x:0 0 0 2024-09-21T04:15:29.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/904: rename d0/d12/d114/c11b to d0/d12/d2a/d80/c144 0 2024-09-21T04:15:29.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/905: truncate d0/d12/d3c/fe8 28519 0 2024-09-21T04:15:29.786 INFO:tasks.workunit.client.0.smithi110.stdout:2/900: symlink d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df7/l129 0 2024-09-21T04:15:29.786 INFO:tasks.workunit.client.0.smithi110.stdout:2/901: chown d2/de7/d116/d8b/d10b/d5f/f2b 3046647 1 2024-09-21T04:15:29.787 INFO:tasks.workunit.client.0.smithi110.stdout:8/980: getdents d0/d30/d105 0 2024-09-21T04:15:29.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/981: creat d0/d7/d23/d6c/d12e/f163 x:0 0 0 2024-09-21T04:15:29.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/982: stat d0/d7/d13/d4b/d108 0 2024-09-21T04:15:29.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/983: write d0/d30/d54/d10c/f118 [692495,94056] 0 2024-09-21T04:15:29.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/984: dread - d0/d30/d34/d3e/d5d/dd7/dda/f142 zero size 2024-09-21T04:15:29.790 INFO:tasks.workunit.client.0.smithi110.stdout:8/985: rename d0/d7/d13/dc5/ffe to d0/d7/d13/f164 0 2024-09-21T04:15:29.792 INFO:tasks.workunit.client.0.smithi110.stdout:5/999: write d1/dc/fd6 [4129205,1150] 0 2024-09-21T04:15:29.792 INFO:tasks.workunit.client.0.smithi110.stdout:8/986: creat d0/d30/d34/d3e/d60/dec/f165 x:0 0 0 2024-09-21T04:15:29.793 INFO:tasks.workunit.client.0.smithi110.stdout:3/906: write d0/d12/d2a/f3e [4145856,66058] 0 2024-09-21T04:15:29.793 INFO:tasks.workunit.client.0.smithi110.stdout:3/907: chown d0/d12/d2a/d80/d9c/f103 33228121 1 2024-09-21T04:15:29.793 INFO:tasks.workunit.client.0.smithi110.stdout:2/902: write d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df7/f107 [716906,73772] 0 2024-09-21T04:15:29.794 INFO:tasks.workunit.client.0.smithi110.stdout:8/987: symlink d0/d7/d89/d10f/d14c/d15f/l166 0 2024-09-21T04:15:29.794 INFO:tasks.workunit.client.0.smithi110.stdout:8/988: write d0/d7/d13/d46/f4c [5037713,55327] 0 2024-09-21T04:15:29.797 INFO:tasks.workunit.client.0.smithi110.stdout:9/915: dwrite d7/d10/d12a/d11/f2e [0,4194304] 0 2024-09-21T04:15:29.797 INFO:tasks.workunit.client.0.smithi110.stdout:8/989: symlink d0/d7/d23/d82/d85/l167 0 2024-09-21T04:15:29.797 INFO:tasks.workunit.client.0.smithi110.stdout:8/990: write d0/d30/d34/d3e/f8c [1035313,86261] 0 2024-09-21T04:15:29.800 INFO:tasks.workunit.client.0.smithi110.stdout:9/916: symlink d7/d10/d12a/d1f/d43/db5/l13c 0 2024-09-21T04:15:29.802 INFO:tasks.workunit.client.0.smithi110.stdout:2/903: write d2/d50/d56/fc4 [4470959,129529] 0 2024-09-21T04:15:29.803 INFO:tasks.workunit.client.0.smithi110.stdout:9/917: rename d7/d10/d88/c102 to d7/d10/d12a/d4f/d9f/d6f/c13d 0 2024-09-21T04:15:29.805 INFO:tasks.workunit.client.0.smithi110.stdout:2/904: truncate d2/de7/d116/d8b/d10b/d65/fe8 972815 0 2024-09-21T04:15:29.805 INFO:tasks.workunit.client.0.smithi110.stdout:2/905: dread - d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/f11f zero size 2024-09-21T04:15:29.805 INFO:tasks.workunit.client.0.smithi110.stdout:2/906: stat d2/lc5 0 2024-09-21T04:15:29.809 INFO:tasks.workunit.client.0.smithi110.stdout:2/907: write d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/dc3/fee [1288124,5231] 0 2024-09-21T04:15:29.809 INFO:tasks.workunit.client.0.smithi110.stdout:2/908: chown d2/d50/d7b 16 1 2024-09-21T04:15:29.809 INFO:tasks.workunit.client.0.smithi110.stdout:2/909: chown d2/de7/d116/d8b/d10b/d65/fe8 191561277 1 2024-09-21T04:15:29.810 INFO:tasks.workunit.client.0.smithi110.stdout:2/910: mkdir d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/d12a 0 2024-09-21T04:15:29.810 INFO:tasks.workunit.client.0.smithi110.stdout:8/991: dread d0/d7/d13/d46/f4c [0,4194304] 0 2024-09-21T04:15:29.810 INFO:tasks.workunit.client.0.smithi110.stdout:8/992: dread - d0/d7/d23/d52/fd0 zero size 2024-09-21T04:15:29.811 INFO:tasks.workunit.client.0.smithi110.stdout:2/911: mkdir d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/d12a/d12b 0 2024-09-21T04:15:29.811 INFO:tasks.workunit.client.0.smithi110.stdout:3/908: dwrite d0/f7 [4194304,4194304] 0 2024-09-21T04:15:29.813 INFO:tasks.workunit.client.0.smithi110.stdout:8/993: symlink d0/d7/l168 0 2024-09-21T04:15:29.813 INFO:tasks.workunit.client.0.smithi110.stdout:9/918: dread d7/d10/d12a/d4f/d9f/d6f/d78/fb9 [0,4194304] 0 2024-09-21T04:15:29.815 INFO:tasks.workunit.client.0.smithi110.stdout:2/912: symlink d2/de7/d116/d8b/d10b/d5f/d9d/d86/dad/l12c 0 2024-09-21T04:15:29.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/909: rename d0/d12/d3c/dce/d123 to d0/d145 0 2024-09-21T04:15:29.818 INFO:tasks.workunit.client.0.smithi110.stdout:8/994: creat d0/d30/d34/d3e/d32/da3/f169 x:0 0 0 2024-09-21T04:15:29.819 INFO:tasks.workunit.client.0.smithi110.stdout:9/919: mknod d7/c13e 0 2024-09-21T04:15:29.822 INFO:tasks.workunit.client.0.smithi110.stdout:2/913: creat d2/de7/d116/d8b/d10b/d2a/da1/f12d x:0 0 0 2024-09-21T04:15:29.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/910: creat d0/d145/f146 x:0 0 0 2024-09-21T04:15:29.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/911: stat d0/d22/d29/d76/fb1 0 2024-09-21T04:15:29.827 INFO:tasks.workunit.client.0.smithi110.stdout:3/912: creat d0/d22/d29/d76/f147 x:0 0 0 2024-09-21T04:15:29.827 INFO:tasks.workunit.client.0.smithi110.stdout:9/920: rmdir d7/d10/d12a/d1f/d43/db5/dca 39 2024-09-21T04:15:29.827 INFO:tasks.workunit.client.0.smithi110.stdout:8/995: dread d0/d30/d54/d6b/f75 [0,4194304] 0 2024-09-21T04:15:29.828 INFO:tasks.workunit.client.0.smithi110.stdout:8/996: chown d0/d30/d34/f12f 51081719 1 2024-09-21T04:15:29.831 INFO:tasks.workunit.client.0.smithi110.stdout:2/914: dread d2/de7/d116/d8b/d10b/d5f/d9d/d86/dad/fed [0,4194304] 0 2024-09-21T04:15:29.832 INFO:tasks.workunit.client.0.smithi110.stdout:8/997: mkdir d0/d7/d13/d4b/d108/d12d/d16a 0 2024-09-21T04:15:29.832 INFO:tasks.workunit.client.0.smithi110.stdout:8/998: read - d0/d30/d34/d3e/d5d/f125 zero size 2024-09-21T04:15:29.833 INFO:tasks.workunit.client.0.smithi110.stdout:8/999: unlink d0/d30/d54/l131 0 2024-09-21T04:15:29.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/915: write d2/d50/f7f [2430141,35466] 0 2024-09-21T04:15:29.839 INFO:tasks.workunit.client.0.smithi110.stdout:2/916: write d2/de7/d116/d8b/d10b/d5f/f58 [1886568,25879] 0 2024-09-21T04:15:29.840 INFO:tasks.workunit.client.0.smithi110.stdout:2/917: write d2/d50/d56/f102 [5140520,85662] 0 2024-09-21T04:15:29.841 INFO:tasks.workunit.client.0.smithi110.stdout:3/913: write d0/f77 [1844628,15896] 0 2024-09-21T04:15:29.841 INFO:tasks.workunit.client.0.smithi110.stdout:3/914: fdatasync d0/d12/d16/d1a/dc6/fe0 0 2024-09-21T04:15:29.841 INFO:tasks.workunit.client.0.smithi110.stdout:3/915: fsync d0/d12/fe4 0 2024-09-21T04:15:29.842 INFO:tasks.workunit.client.0.smithi110.stdout:3/916: fdatasync d0/d22/d29/f8b 0 2024-09-21T04:15:29.842 INFO:tasks.workunit.client.0.smithi110.stdout:3/917: stat d0/d22/d29 0 2024-09-21T04:15:29.843 INFO:tasks.workunit.client.0.smithi110.stdout:2/918: getdents d2/de7/d116/d60/dfd/d111 0 2024-09-21T04:15:29.844 INFO:tasks.workunit.client.0.smithi110.stdout:3/918: dread d0/d12/d2a/fde [0,4194304] 0 2024-09-21T04:15:29.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/919: mknod d2/de7/d116/d60/dfd/d111/c12e 0 2024-09-21T04:15:29.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/920: write d2/d50/ff6 [1237748,41776] 0 2024-09-21T04:15:29.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/921: chown d2/de7/d116/d8b/d10b/d5f 7768 1 2024-09-21T04:15:29.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/922: fdatasync d2/d81/dfa/f11e 0 2024-09-21T04:15:29.847 INFO:tasks.workunit.client.0.smithi110.stdout:2/923: mknod d2/de7/d116/d8b/d10b/d29/c12f 0 2024-09-21T04:15:29.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/924: rename d2/de7/d116/d8b/da5/da9/ccf to d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df7/dff/c130 0 2024-09-21T04:15:29.850 INFO:tasks.workunit.client.0.smithi110.stdout:9/921: dwrite d7/d10/d12a/dae/f131 [0,4194304] 0 2024-09-21T04:15:29.850 INFO:tasks.workunit.client.0.smithi110.stdout:9/922: creat d7/d10/d12a/dae/f13f x:0 0 0 2024-09-21T04:15:29.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/925: link d2/de7/d116/d8b/d10b/d29/c12f d2/de7/d116/d8b/d10b/d5f/c131 0 2024-09-21T04:15:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/919: dwrite d0/d12/d3c/d5a/d65/feb [0,4194304] 0 2024-09-21T04:15:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/920: write d0/f7 [3160093,73980] 0 2024-09-21T04:15:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/921: dread - d0/d51/d7b/f8d zero size 2024-09-21T04:15:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/922: read d0/d12/d2a/fde [1242290,95878] 0 2024-09-21T04:15:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/923: truncate d0/d12/d16/d1a/d5c/faa 903011 0 2024-09-21T04:15:29.873 INFO:tasks.workunit.client.0.smithi110.stdout:9/923: mknod d7/d26/d10c/c140 0 2024-09-21T04:15:29.874 INFO:tasks.workunit.client.0.smithi110.stdout:2/926: creat d2/de7/d116/d60/dfd/f132 x:0 0 0 2024-09-21T04:15:29.875 INFO:tasks.workunit.client.0.smithi110.stdout:3/924: dwrite d0/d22/f46 [0,4194304] 0 2024-09-21T04:15:29.877 INFO:tasks.workunit.client.0.smithi110.stdout:9/924: creat d7/d10/d12a/d35/f141 x:0 0 0 2024-09-21T04:15:29.877 INFO:tasks.workunit.client.0.smithi110.stdout:9/925: readlink d7/d96/l115 0 2024-09-21T04:15:29.879 INFO:tasks.workunit.client.0.smithi110.stdout:2/927: creat d2/de7/d116/d60/f133 x:0 0 0 2024-09-21T04:15:29.882 INFO:tasks.workunit.client.0.smithi110.stdout:2/928: dread d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df7/f107 [0,4194304] 0 2024-09-21T04:15:29.889 INFO:tasks.workunit.client.0.smithi110.stdout:7/926: sync 2024-09-21T04:15:29.890 INFO:tasks.workunit.client.0.smithi110.stdout:7/927: creat dd/f132 x:0 0 0 2024-09-21T04:15:29.892 INFO:tasks.workunit.client.0.smithi110.stdout:9/926: dread d7/d10/d12a/fdf [0,4194304] 0 2024-09-21T04:15:29.892 INFO:tasks.workunit.client.0.smithi110.stdout:9/927: write d7/d10/d12a/d14/d31/f105 [769117,20443] 0 2024-09-21T04:15:29.893 INFO:tasks.workunit.client.0.smithi110.stdout:2/929: write d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9b/fb4 [847942,105627] 0 2024-09-21T04:15:29.894 INFO:tasks.workunit.client.0.smithi110.stdout:9/928: truncate d7/d10/d12a/d1f/d43/f6c 3686296 0 2024-09-21T04:15:29.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/930: mkdir d2/de7/d116/d8b/d134 0 2024-09-21T04:15:29.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/931: dread - d2/de7/d116/d60/f10e zero size 2024-09-21T04:15:29.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/932: unlink d2/d50/d7b/f91 0 2024-09-21T04:15:29.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/933: chown d2/de7/d116/d8b/d10b/l96 1401964 1 2024-09-21T04:15:29.901 INFO:tasks.workunit.client.0.smithi110.stdout:2/934: rmdir d2/de7/d116/d8b/d10b/d2a 39 2024-09-21T04:15:29.903 INFO:tasks.workunit.client.0.smithi110.stdout:3/925: dwrite d0/d12/d2a/d80/da2/db0/fd8 [0,4194304] 0 2024-09-21T04:15:29.903 INFO:tasks.workunit.client.0.smithi110.stdout:3/926: fdatasync d0/d12/d3c/f48 0 2024-09-21T04:15:29.904 INFO:tasks.workunit.client.0.smithi110.stdout:2/935: creat d2/de7/d116/d8b/d10b/d65/f135 x:0 0 0 2024-09-21T04:15:29.907 INFO:tasks.workunit.client.0.smithi110.stdout:3/927: creat d0/d12/d2a/d80/da2/f148 x:0 0 0 2024-09-21T04:15:29.913 INFO:tasks.workunit.client.0.smithi110.stdout:9/929: dwrite d7/d10/d12a/d1f/d43/db5/fba [0,4194304] 0 2024-09-21T04:15:29.913 INFO:tasks.workunit.client.0.smithi110.stdout:7/928: dwrite dd/d1c/dad/db2/d56/d66/fe1 [0,4194304] 0 2024-09-21T04:15:29.917 INFO:tasks.workunit.client.0.smithi110.stdout:9/930: link d7/d10/d12a/d1f/d43/f6c d7/d10/d88/d8f/d12e/f142 0 2024-09-21T04:15:29.917 INFO:tasks.workunit.client.0.smithi110.stdout:9/931: chown d7/d10/d12a/d11/c80 319626389 1 2024-09-21T04:15:29.917 INFO:tasks.workunit.client.0.smithi110.stdout:2/936: dread d2/d50/f7f [0,4194304] 0 2024-09-21T04:15:29.919 INFO:tasks.workunit.client.0.smithi110.stdout:7/929: symlink dd/d1c/dad/db2/d56/d66/da6/da7/d11c/l133 0 2024-09-21T04:15:29.919 INFO:tasks.workunit.client.0.smithi110.stdout:3/928: write d0/f6 [693643,9425] 0 2024-09-21T04:15:29.921 INFO:tasks.workunit.client.0.smithi110.stdout:7/930: symlink dd/d14/d17/l134 0 2024-09-21T04:15:29.923 INFO:tasks.workunit.client.0.smithi110.stdout:2/937: mkdir d2/de7/d116/d8b/d10b/d29/d19/d136 0 2024-09-21T04:15:29.926 INFO:tasks.workunit.client.0.smithi110.stdout:9/932: read d7/d10/d12a/d1f/d68/f7c [3803397,116468] 0 2024-09-21T04:15:29.927 INFO:tasks.workunit.client.0.smithi110.stdout:9/933: unlink d7/d10/d12a/d35/f75 0 2024-09-21T04:15:29.928 INFO:tasks.workunit.client.0.smithi110.stdout:9/934: dread - d7/d96/ff2 zero size 2024-09-21T04:15:29.929 INFO:tasks.workunit.client.0.smithi110.stdout:9/935: truncate d7/d10/d12a/d35/d71/fc5 1679751 0 2024-09-21T04:15:29.929 INFO:tasks.workunit.client.0.smithi110.stdout:9/936: write d7/d10/d12a/dae/fb6 [412174,124930] 0 2024-09-21T04:15:29.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/937: creat d7/d10/d12a/d1f/d72/d59/f143 x:0 0 0 2024-09-21T04:15:29.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/938: chown d7/d10/d12a/dae 243212826 1 2024-09-21T04:15:29.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/939: rename d7/d10/d88/d8f to d7/d10/d88/d8f/d12c/d144 22 2024-09-21T04:15:29.935 INFO:tasks.workunit.client.0.smithi110.stdout:7/931: dread dd/f1a [0,4194304] 0 2024-09-21T04:15:29.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/932: mknod dd/d1c/dad/db2/d56/d66/da6/d7a/c135 0 2024-09-21T04:15:29.938 INFO:tasks.workunit.client.0.smithi110.stdout:7/933: write dd/d14/d17/d8a/f8b [1326271,4776] 0 2024-09-21T04:15:29.942 INFO:tasks.workunit.client.0.smithi110.stdout:3/929: dwrite d0/d12/d16/d1a/d5c/f66 [0,4194304] 0 2024-09-21T04:15:29.946 INFO:tasks.workunit.client.0.smithi110.stdout:7/934: dread dd/d1c/dad/db2/d4a/d52/f58 [0,4194304] 0 2024-09-21T04:15:29.946 INFO:tasks.workunit.client.0.smithi110.stdout:7/935: creat dd/d1c/dad/d10d/f136 x:0 0 0 2024-09-21T04:15:29.946 INFO:tasks.workunit.client.0.smithi110.stdout:7/936: write dd/d4d/dee/fd3 [883758,48630] 0 2024-09-21T04:15:29.950 INFO:tasks.workunit.client.0.smithi110.stdout:7/937: mknod dd/d1c/dad/db2/d4a/d52/c137 0 2024-09-21T04:15:29.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/938: dwrite d2/de7/d116/d60/f10e [0,4194304] 0 2024-09-21T04:15:29.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/939: readlink d2/de7/d116/d8b/d10b/d29/d19/l7c 0 2024-09-21T04:15:29.951 INFO:tasks.workunit.client.0.smithi110.stdout:9/940: dwrite d7/d10/d12a/ff4 [0,4194304] 0 2024-09-21T04:15:29.953 INFO:tasks.workunit.client.0.smithi110.stdout:7/938: link dd/d14/c27 dd/d1c/dad/db2/d4a/d129/c138 0 2024-09-21T04:15:29.955 INFO:tasks.workunit.client.0.smithi110.stdout:3/930: dread d0/d12/d16/d1a/d5c/f60 [0,4194304] 0 2024-09-21T04:15:29.958 INFO:tasks.workunit.client.0.smithi110.stdout:2/940: truncate d2/de7/d116/d8b/d10b/f5d 2047974 0 2024-09-21T04:15:29.960 INFO:tasks.workunit.client.0.smithi110.stdout:7/939: dread dd/d1c/dad/db2/d56/d66/da6/f87 [0,4194304] 0 2024-09-21T04:15:29.960 INFO:tasks.workunit.client.0.smithi110.stdout:7/940: fdatasync dd/d1c/dad/db2/f4f 0 2024-09-21T04:15:29.961 INFO:tasks.workunit.client.0.smithi110.stdout:9/941: mkdir d7/d10/d88/d8f/d12e/d145 0 2024-09-21T04:15:29.962 INFO:tasks.workunit.client.0.smithi110.stdout:3/931: link d0/d12/d16/l2b d0/d12/d3c/d5a/d70/l149 0 2024-09-21T04:15:29.963 INFO:tasks.workunit.client.0.smithi110.stdout:3/932: dread d0/d12/fd5 [0,4194304] 0 2024-09-21T04:15:29.965 INFO:tasks.workunit.client.0.smithi110.stdout:7/941: unlink dd/f3e 0 2024-09-21T04:15:29.965 INFO:tasks.workunit.client.0.smithi110.stdout:7/942: readlink dd/l12d 0 2024-09-21T04:15:29.966 INFO:tasks.workunit.client.0.smithi110.stdout:9/942: mkdir d7/d10/d88/d8f/d12c/d146 0 2024-09-21T04:15:29.967 INFO:tasks.workunit.client.0.smithi110.stdout:2/941: getdents d2/d81 0 2024-09-21T04:15:29.967 INFO:tasks.workunit.client.0.smithi110.stdout:2/942: read - d2/de7/d116/d8b/d10b/d2a/ff0 zero size 2024-09-21T04:15:29.968 INFO:tasks.workunit.client.0.smithi110.stdout:2/943: dread d2/de7/d116/d8b/d10b/d29/d19/fdb [0,4194304] 0 2024-09-21T04:15:29.969 INFO:tasks.workunit.client.0.smithi110.stdout:3/933: mknod d0/d22/dd2/c14a 0 2024-09-21T04:15:29.969 INFO:tasks.workunit.client.0.smithi110.stdout:3/934: write d0/f6 [5037757,74514] 0 2024-09-21T04:15:29.970 INFO:tasks.workunit.client.0.smithi110.stdout:7/943: getdents dd/d4d/dee/ddc 0 2024-09-21T04:15:29.970 INFO:tasks.workunit.client.0.smithi110.stdout:7/944: chown dd 1 1 2024-09-21T04:15:29.971 INFO:tasks.workunit.client.0.smithi110.stdout:9/943: unlink d7/d10/d12a/d4f/c11c 0 2024-09-21T04:15:29.971 INFO:tasks.workunit.client.0.smithi110.stdout:9/944: chown d7/d10/d12a/d1f/d25/l58 207326317 1 2024-09-21T04:15:29.973 INFO:tasks.workunit.client.0.smithi110.stdout:2/944: fsync d2/de7/d116/f105 0 2024-09-21T04:15:29.974 INFO:tasks.workunit.client.0.smithi110.stdout:3/935: symlink d0/l14b 0 2024-09-21T04:15:29.974 INFO:tasks.workunit.client.0.smithi110.stdout:3/936: fdatasync d0/f56 0 2024-09-21T04:15:29.976 INFO:tasks.workunit.client.0.smithi110.stdout:7/945: unlink dd/d1c/dad/db2/d4a/d52/f9c 0 2024-09-21T04:15:29.976 INFO:tasks.workunit.client.0.smithi110.stdout:9/945: truncate d7/d10/d88/d8f/d12e/f142 1408645 0 2024-09-21T04:15:29.979 INFO:tasks.workunit.client.0.smithi110.stdout:2/945: symlink d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/d12a/d12b/l137 0 2024-09-21T04:15:29.980 INFO:tasks.workunit.client.0.smithi110.stdout:3/937: unlink d0/d12/d16/d1a/d6f/dd9/c134 0 2024-09-21T04:15:29.982 INFO:tasks.workunit.client.0.smithi110.stdout:7/946: creat dd/d4d/dee/ddc/f139 x:0 0 0 2024-09-21T04:15:29.983 INFO:tasks.workunit.client.0.smithi110.stdout:9/946: mknod d7/d10/d12a/d14/c147 0 2024-09-21T04:15:29.984 INFO:tasks.workunit.client.0.smithi110.stdout:9/947: dread d7/d10/d12a/d11/fc6 [0,4194304] 0 2024-09-21T04:15:29.987 INFO:tasks.workunit.client.0.smithi110.stdout:3/938: link d0/d12/d16/d1a/cf7 d0/c14c 0 2024-09-21T04:15:29.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/946: write d2/de7/d116/f105 [2753935,97943] 0 2024-09-21T04:15:29.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/948: creat d7/d10/d12a/d35/d109/f148 x:0 0 0 2024-09-21T04:15:29.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/949: dread - d7/d10/d12a/d35/fa7 zero size 2024-09-21T04:15:29.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/950: fsync d7/d10/d12a/d4f/d9f/d6f/d78/d7d/ffd 0 2024-09-21T04:15:29.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/951: read d7/d10/d88/d8f/db7/f10f [668386,126459] 0 2024-09-21T04:15:29.991 INFO:tasks.workunit.client.0.smithi110.stdout:7/947: write dd/d14/d17/f85 [3222642,73830] 0 2024-09-21T04:15:29.997 INFO:tasks.workunit.client.0.smithi110.stdout:7/948: dread dd/d1c/dad/db2/f3c [0,4194304] 0 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/949: write dd/d4d/dee/f7c [1050889,80295] 0 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/950: chown dd/d1c/dad/c9e 4241332 1 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/951: chown dd/d1c/dad/db2/d56/l78 6 1 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/952: creat dd/d48/f13a x:0 0 0 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/953: dread - dd/d14/d17/fb9 zero size 2024-09-21T04:15:29.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/954: readlink dd/d1c/dad/db2/d56/d66/da6/da7/la9 0 2024-09-21T04:15:30.012 INFO:tasks.workunit.client.0.smithi110.stdout:3/939: dwrite d0/d145/f146 [0,4194304] 0 2024-09-21T04:15:30.013 INFO:tasks.workunit.client.0.smithi110.stdout:7/955: dwrite dd/d14/f36 [0,4194304] 0 2024-09-21T04:15:30.016 INFO:tasks.workunit.client.0.smithi110.stdout:3/940: dread d0/d12/d16/d1a/d5c/f60 [0,4194304] 0 2024-09-21T04:15:30.033 INFO:tasks.workunit.client.0.smithi110.stdout:3/941: dwrite d0/d12/d16/d1a/d5c/ff5 [0,4194304] 0 2024-09-21T04:15:30.033 INFO:tasks.workunit.client.0.smithi110.stdout:3/942: creat d0/f14d x:0 0 0 2024-09-21T04:15:30.042 INFO:tasks.workunit.client.0.smithi110.stdout:3/943: dread d0/d22/dd2/f12a [0,4194304] 0 2024-09-21T04:15:30.048 INFO:tasks.workunit.client.0.smithi110.stdout:2/947: stat d2/d81/c85 0 2024-09-21T04:15:30.048 INFO:tasks.workunit.client.0.smithi110.stdout:2/948: fsync d2/d50/d56/f104 0 2024-09-21T04:15:30.050 INFO:tasks.workunit.client.0.smithi110.stdout:9/952: dread d7/d10/d12a/d1f/d43/f92 [0,4194304] 0 2024-09-21T04:15:30.071 INFO:tasks.workunit.client.0.smithi110.stdout:2/949: dwrite d2/de7/d116/d8b/d10b/d29/f48 [0,4194304] 0 2024-09-21T04:15:30.071 INFO:tasks.workunit.client.0.smithi110.stdout:9/953: dwrite d7/d10/d12a/d1f/d43/f6e [4194304,4194304] 0 2024-09-21T04:15:30.072 INFO:tasks.workunit.client.0.smithi110.stdout:9/954: fsync d7/d10/d88/d8f/d12c/f126 0 2024-09-21T04:15:30.072 INFO:tasks.workunit.client.0.smithi110.stdout:9/955: stat d7/d10/d12a/d35 0 2024-09-21T04:15:30.097 INFO:tasks.workunit.client.0.smithi110.stdout:9/956: dwrite d7/d10/d12a/d14/ff0 [0,4194304] 0 2024-09-21T04:15:30.098 INFO:tasks.workunit.client.0.smithi110.stdout:9/957: write d7/d10/d88/d8f/da0/fc8 [1031904,24156] 0 2024-09-21T04:15:30.098 INFO:tasks.workunit.client.0.smithi110.stdout:2/950: dwrite d2/d50/f7f [0,4194304] 0 2024-09-21T04:15:30.176 INFO:tasks.workunit.client.0.smithi110.stdout:3/944: truncate d0/d12/d16/d1a/f35 3382443 0 2024-09-21T04:15:30.193 INFO:tasks.workunit.client.0.smithi110.stdout:9/958: symlink d7/d10/d12a/d4f/d101/l149 0 2024-09-21T04:15:30.198 INFO:tasks.workunit.client.0.smithi110.stdout:2/951: creat d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/d12a/d12b/f138 x:0 0 0 2024-09-21T04:15:30.203 INFO:tasks.workunit.client.0.smithi110.stdout:3/945: mkdir d0/d12/d16/d1a/d6f/dd9/d14e 0 2024-09-21T04:15:30.210 INFO:tasks.workunit.client.0.smithi110.stdout:7/956: unlink dd/d1c/dad/db2/d4a/c6c 0 2024-09-21T04:15:30.222 INFO:tasks.workunit.client.0.smithi110.stdout:3/946: mkdir d0/d12/d3c/dce/d14f 0 2024-09-21T04:15:30.232 INFO:tasks.workunit.client.0.smithi110.stdout:7/957: link dd/d1c/dad/db2/d56/d66/da6/d7a/f97 dd/db1/f13b 0 2024-09-21T04:15:30.232 INFO:tasks.workunit.client.0.smithi110.stdout:7/958: write dd/d1c/f10f [18509,66160] 0 2024-09-21T04:15:30.234 INFO:tasks.workunit.client.0.smithi110.stdout:7/959: creat dd/d1c/dad/db2/d56/d66/da6/da7/d11c/f13c x:0 0 0 2024-09-21T04:15:30.235 INFO:tasks.workunit.client.0.smithi110.stdout:7/960: truncate dd/d4d/dee/ddc/fec 1143605 0 2024-09-21T04:15:30.236 INFO:tasks.workunit.client.0.smithi110.stdout:7/961: mkdir dd/d1c/dad/db2/d4a/d52/d13d 0 2024-09-21T04:15:30.238 INFO:tasks.workunit.client.0.smithi110.stdout:3/947: dwrite d0/d12/d3c/d5a/d70/fbf [0,4194304] 0 2024-09-21T04:15:30.249 INFO:tasks.workunit.client.0.smithi110.stdout:3/948: read d0/d12/d2a/d80/da2/db0/fd8 [3189117,83648] 0 2024-09-21T04:15:30.249 INFO:tasks.workunit.client.0.smithi110.stdout:3/949: write d0/d22/d29/d76/f147 [793603,107597] 0 2024-09-21T04:15:30.253 INFO:tasks.workunit.client.0.smithi110.stdout:7/962: dwrite dd/d14/fcb [0,4194304] 0 2024-09-21T04:15:30.253 INFO:tasks.workunit.client.0.smithi110.stdout:7/963: chown dd/d1c/dad/db2 20 1 2024-09-21T04:15:30.264 INFO:tasks.workunit.client.0.smithi110.stdout:7/964: write dd/d1c/f113 [2572882,124758] 0 2024-09-21T04:15:30.264 INFO:tasks.workunit.client.0.smithi110.stdout:7/965: fdatasync dd/f57 0 2024-09-21T04:15:30.268 INFO:tasks.workunit.client.0.smithi110.stdout:3/950: dwrite d0/d12/f23 [4194304,4194304] 0 2024-09-21T04:15:30.270 INFO:tasks.workunit.client.0.smithi110.stdout:3/951: link d0/d12/d2a/d80/d9c/ff1 d0/d12/d3c/dad/f150 0 2024-09-21T04:15:30.270 INFO:tasks.workunit.client.0.smithi110.stdout:3/952: chown d0/fe 0 1 2024-09-21T04:15:30.272 INFO:tasks.workunit.client.0.smithi110.stdout:3/953: creat d0/d12/d2a/d80/dc4/f151 x:0 0 0 2024-09-21T04:15:30.276 INFO:tasks.workunit.client.0.smithi110.stdout:3/954: dread d0/d22/d29/fc5 [0,4194304] 0 2024-09-21T04:15:30.285 INFO:tasks.workunit.client.0.smithi110.stdout:7/966: dwrite dd/d1c/dad/db2/d4a/f10a [0,4194304] 0 2024-09-21T04:15:30.285 INFO:tasks.workunit.client.0.smithi110.stdout:7/967: fsync dd/d1c/dad/db2/d56/d66/fcd 0 2024-09-21T04:15:30.286 INFO:tasks.workunit.client.0.smithi110.stdout:7/968: write dd/d1c/dad/db2/d4a/d129/fe4 [709819,112555] 0 2024-09-21T04:15:30.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/969: symlink dd/d4d/dee/ddc/l13e 0 2024-09-21T04:15:30.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/970: truncate dd/d48/d9f/fdb 479261 0 2024-09-21T04:15:30.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/971: truncate dd/d4d/f5e 2188635 0 2024-09-21T04:15:30.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/972: fdatasync dd/f1a 0 2024-09-21T04:15:30.290 INFO:tasks.workunit.client.0.smithi110.stdout:7/973: creat dd/d1c/dad/db2/f13f x:0 0 0 2024-09-21T04:15:30.294 INFO:tasks.workunit.client.0.smithi110.stdout:7/974: unlink dd/d1c/c43 0 2024-09-21T04:15:30.299 INFO:tasks.workunit.client.0.smithi110.stdout:3/955: dwrite d0/d12/d3c/f4f [0,4194304] 0 2024-09-21T04:15:30.301 INFO:tasks.workunit.client.0.smithi110.stdout:3/956: link d0/d12/d16/d1a/d13c/fa8 d0/d51/d7b/f152 0 2024-09-21T04:15:30.308 INFO:tasks.workunit.client.0.smithi110.stdout:7/975: dwrite dd/d14/fcb [4194304,4194304] 0 2024-09-21T04:15:30.314 INFO:tasks.workunit.client.0.smithi110.stdout:7/976: write dd/f1a [1857771,120985] 0 2024-09-21T04:15:30.314 INFO:tasks.workunit.client.0.smithi110.stdout:3/957: dwrite d0/f77 [0,4194304] 0 2024-09-21T04:15:30.318 INFO:tasks.workunit.client.0.smithi110.stdout:3/958: mkdir d0/d12/d3c/d5a/dc8/d153 0 2024-09-21T04:15:30.318 INFO:tasks.workunit.client.0.smithi110.stdout:3/959: chown d0/d12/d16/d1a/fc1 575056 1 2024-09-21T04:15:30.323 INFO:tasks.workunit.client.0.smithi110.stdout:3/960: symlink d0/d12/d3c/d5a/d65/l154 0 2024-09-21T04:15:30.325 INFO:tasks.workunit.client.0.smithi110.stdout:3/961: symlink d0/d12/d2a/d80/l155 0 2024-09-21T04:15:30.342 INFO:tasks.workunit.client.0.smithi110.stdout:3/962: dwrite d0/d12/d16/d1a/f35 [0,4194304] 0 2024-09-21T04:15:30.342 INFO:tasks.workunit.client.0.smithi110.stdout:3/963: write d0/d12/d3c/f5e [3647645,22265] 0 2024-09-21T04:15:30.344 INFO:tasks.workunit.client.0.smithi110.stdout:3/964: creat d0/d12/d16/d1a/dc6/d137/f156 x:0 0 0 2024-09-21T04:15:30.353 INFO:tasks.workunit.client.0.smithi110.stdout:3/965: write d0/d12/f23 [1000079,3755] 0 2024-09-21T04:15:30.353 INFO:tasks.workunit.client.0.smithi110.stdout:3/966: dread - d0/d12/d16/d1a/d13c/fef zero size 2024-09-21T04:15:30.359 INFO:tasks.workunit.client.0.smithi110.stdout:3/967: mknod d0/d12/d114/d131/c157 0 2024-09-21T04:15:30.359 INFO:tasks.workunit.client.0.smithi110.stdout:3/968: write d0/d22/dd2/f12a [2331578,66014] 0 2024-09-21T04:15:30.360 INFO:tasks.workunit.client.0.smithi110.stdout:3/969: write d0/d12/d3c/d5a/d65/f75 [608893,12146] 0 2024-09-21T04:15:30.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/970: rename d0/d12/d3c/d5a/d65/df0/f109 to d0/d12/d3c/d5a/d70/dab/f158 0 2024-09-21T04:15:30.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/971: fsync d0/d12/d3c/dad/d11f/f12c 0 2024-09-21T04:15:30.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/972: chown d0/d12/d16 67893 1 2024-09-21T04:15:30.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/973: fsync d0/d12/d2a/d80/d9c/fba 0 2024-09-21T04:15:30.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/974: symlink d0/d12/d3c/d5a/d70/l159 0 2024-09-21T04:15:30.365 INFO:tasks.workunit.client.0.smithi110.stdout:3/975: dread - d0/d12/d16/d1a/dc6/d137/f156 zero size 2024-09-21T04:15:30.367 INFO:tasks.workunit.client.0.smithi110.stdout:3/976: rename d0/d12/fb3 to d0/d12/d2a/d80/da2/db0/d116/f15a 0 2024-09-21T04:15:30.373 INFO:tasks.workunit.client.0.smithi110.stdout:3/977: dread d0/f77 [0,4194304] 0 2024-09-21T04:15:30.375 INFO:tasks.workunit.client.0.smithi110.stdout:3/978: rename d0/d22/l59 to d0/d12/d16/d1a/dac/l15b 0 2024-09-21T04:15:30.376 INFO:tasks.workunit.client.0.smithi110.stdout:3/979: fdatasync d0/f56 0 2024-09-21T04:15:30.376 INFO:tasks.workunit.client.0.smithi110.stdout:3/980: fdatasync d0/d12/d16/f62 0 2024-09-21T04:15:30.378 INFO:tasks.workunit.client.0.smithi110.stdout:3/981: creat d0/d12/d3c/d5a/d143/f15c x:0 0 0 2024-09-21T04:15:30.378 INFO:tasks.workunit.client.0.smithi110.stdout:3/982: creat d0/d12/d3c/d5a/d70/dab/f15d x:0 0 0 2024-09-21T04:15:30.378 INFO:tasks.workunit.client.0.smithi110.stdout:3/983: chown d0/f56 150 1 2024-09-21T04:15:30.387 INFO:tasks.workunit.client.0.smithi110.stdout:3/984: dread d0/d12/d2a/f31 [0,4194304] 0 2024-09-21T04:15:30.840 INFO:tasks.workunit.client.0.smithi110.stdout:9/959: sync 2024-09-21T04:15:30.841 INFO:tasks.workunit.client.0.smithi110.stdout:9/960: rename d7/d10/d12a/d4f/l50 to d7/d10/dfb/l14a 0 2024-09-21T04:15:30.841 INFO:tasks.workunit.client.0.smithi110.stdout:9/961: chown d7/d10/d12a/d35/f3a 76439 1 2024-09-21T04:15:30.841 INFO:tasks.workunit.client.0.smithi110.stdout:9/962: write d7/d26/fb1 [626067,58121] 0 2024-09-21T04:15:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:30 smithi110 ceph-mon[27723]: pgmap v12: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 352 MiB/s rd, 441 MiB/s wr, 634 op/s 2024-09-21T04:15:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:30 smithi148 ceph-mon[34957]: pgmap v12: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 352 MiB/s rd, 441 MiB/s wr, 634 op/s 2024-09-21T04:15:30.951 INFO:tasks.workunit.client.0.smithi110.stdout:7/977: sync 2024-09-21T04:15:30.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/952: sync 2024-09-21T04:15:30.951 INFO:tasks.workunit.client.0.smithi110.stdout:3/985: sync 2024-09-21T04:15:30.955 INFO:tasks.workunit.client.0.smithi110.stdout:2/953: link d2/de7/d116/d8b/d10b/d5f/d9d/d86/dad/fed d2/de7/d116/da3/db9/d10f/d110/d125/f139 0 2024-09-21T04:15:30.958 INFO:tasks.workunit.client.0.smithi110.stdout:7/978: write dd/d1c/dad/db2/d56/fac [2370970,91742] 0 2024-09-21T04:15:30.960 INFO:tasks.workunit.client.0.smithi110.stdout:7/979: mknod dd/d1c/dad/db2/d4a/d115/c140 0 2024-09-21T04:15:30.960 INFO:tasks.workunit.client.0.smithi110.stdout:3/986: dread d0/fcb [0,4194304] 0 2024-09-21T04:15:30.962 INFO:tasks.workunit.client.0.smithi110.stdout:7/980: rename dd/d4d/l108 to dd/db1/l141 0 2024-09-21T04:15:30.962 INFO:tasks.workunit.client.0.smithi110.stdout:7/981: chown dd/d14/f41 2977762 1 2024-09-21T04:15:30.962 INFO:tasks.workunit.client.0.smithi110.stdout:7/982: chown dd/d14/d17/fb9 22412 1 2024-09-21T04:15:30.963 INFO:tasks.workunit.client.0.smithi110.stdout:3/987: symlink d0/d12/d2a/d80/dc4/l15e 0 2024-09-21T04:15:30.963 INFO:tasks.workunit.client.0.smithi110.stdout:3/988: chown d0/d12/d16/d1a 3217 1 2024-09-21T04:15:30.964 INFO:tasks.workunit.client.0.smithi110.stdout:3/989: creat d0/d12/d114/d131/f15f x:0 0 0 2024-09-21T04:15:30.965 INFO:tasks.workunit.client.0.smithi110.stdout:2/954: read d2/de7/d116/d8b/d10b/d5f/d9d/d86/f103 [183868,119321] 0 2024-09-21T04:15:30.965 INFO:tasks.workunit.client.0.smithi110.stdout:2/955: write d2/de7/d116/d8b/d10b/d5f/d9d/f109 [4574571,45341] 0 2024-09-21T04:15:30.966 INFO:tasks.workunit.client.0.smithi110.stdout:2/956: chown d2/de7/d116/lf3 162244734 1 2024-09-21T04:15:30.972 INFO:tasks.workunit.client.0.smithi110.stdout:2/957: write d2/de7/d116/d8b/d10b/d29/d19/fdb [291777,130717] 0 2024-09-21T04:15:30.973 INFO:tasks.workunit.client.0.smithi110.stdout:2/958: symlink d2/d50/d7b/l13a 0 2024-09-21T04:15:30.982 INFO:tasks.workunit.client.0.smithi110.stdout:2/959: dread d2/de7/d116/d8b/d10b/d29/f4f [0,4194304] 0 2024-09-21T04:15:30.984 INFO:tasks.workunit.client.0.smithi110.stdout:2/960: rename d2/de7/d116/d8b/d10b/fe4 to d2/d50/d9c/f13b 0 2024-09-21T04:15:30.984 INFO:tasks.workunit.client.0.smithi110.stdout:2/961: truncate d2/de7/d116/d8b/d10b/d29/f71 1154539 0 2024-09-21T04:15:30.985 INFO:tasks.workunit.client.0.smithi110.stdout:2/962: symlink d2/de7/d116/d8b/d10b/d29/l13c 0 2024-09-21T04:15:30.986 INFO:tasks.workunit.client.0.smithi110.stdout:2/963: creat d2/de7/d116/da3/db9/d10f/f13d x:0 0 0 2024-09-21T04:15:30.986 INFO:tasks.workunit.client.0.smithi110.stdout:2/964: chown d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/c115 35 1 2024-09-21T04:15:30.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/965: symlink d2/de7/l13e 0 2024-09-21T04:15:30.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/966: mknod d2/de7/d116/da3/db9/d10f/d110/d125/c13f 0 2024-09-21T04:15:30.990 INFO:tasks.workunit.client.0.smithi110.stdout:2/967: mkdir d2/d50/d140 0 2024-09-21T04:15:30.991 INFO:tasks.workunit.client.0.smithi110.stdout:7/983: dwrite dd/d1c/dad/db2/d56/d66/da6/d98/fb5 [0,4194304] 0 2024-09-21T04:15:30.991 INFO:tasks.workunit.client.0.smithi110.stdout:3/990: dwrite d0/fe [0,4194304] 0 2024-09-21T04:15:30.991 INFO:tasks.workunit.client.0.smithi110.stdout:3/991: chown d0/d12/d3c/d5a/d70/cca 4 1 2024-09-21T04:15:30.993 INFO:tasks.workunit.client.0.smithi110.stdout:7/984: link dd/d1c/dad/db2/cc9 dd/d1c/dad/db2/d56/c142 0 2024-09-21T04:15:30.994 INFO:tasks.workunit.client.0.smithi110.stdout:3/992: creat d0/d12/d3c/d5a/dc8/d153/f160 x:0 0 0 2024-09-21T04:15:30.995 INFO:tasks.workunit.client.0.smithi110.stdout:7/985: mkdir dd/d14/d17/d143 0 2024-09-21T04:15:30.996 INFO:tasks.workunit.client.0.smithi110.stdout:2/968: dread d2/de7/d116/d8b/d10b/d65/f75 [4194304,4194304] 0 2024-09-21T04:15:30.997 INFO:tasks.workunit.client.0.smithi110.stdout:3/993: unlink d0/l14b 0 2024-09-21T04:15:30.997 INFO:tasks.workunit.client.0.smithi110.stdout:3/994: chown d0/d12/d16/d1a 94 1 2024-09-21T04:15:30.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/986: write dd/d1c/f113 [339611,6289] 0 2024-09-21T04:15:31.006 INFO:tasks.workunit.client.0.smithi110.stdout:2/969: rmdir d2/d50/d126 0 2024-09-21T04:15:31.007 INFO:tasks.workunit.client.0.smithi110.stdout:2/970: write d2/d50/ff6 [1108238,118080] 0 2024-09-21T04:15:31.009 INFO:tasks.workunit.client.0.smithi110.stdout:7/987: rename dd/d1c/dad/db2/d4a/d129/fe4 to dd/d1c/dad/db2/d56/d124/f144 0 2024-09-21T04:15:31.016 INFO:tasks.workunit.client.0.smithi110.stdout:7/988: write dd/d1c/dad/db2/d4a/d52/faf [1471764,79737] 0 2024-09-21T04:15:31.016 INFO:tasks.workunit.client.0.smithi110.stdout:7/989: fsync dd/db1/ff1 0 2024-09-21T04:15:31.018 INFO:tasks.workunit.client.0.smithi110.stdout:7/990: truncate dd/d1c/f34 76351 0 2024-09-21T04:15:31.018 INFO:tasks.workunit.client.0.smithi110.stdout:3/995: dwrite d0/d12/d16/d1a/d5c/f79 [0,4194304] 0 2024-09-21T04:15:31.019 INFO:tasks.workunit.client.0.smithi110.stdout:3/996: readlink d0/d12/d16/d1a/dac/l15b 0 2024-09-21T04:15:31.019 INFO:tasks.workunit.client.0.smithi110.stdout:7/991: rename dd/d1c/dad/db2/d4a/da5/c10e to dd/d4d/dee/ddc/c145 0 2024-09-21T04:15:31.021 INFO:tasks.workunit.client.0.smithi110.stdout:3/997: stat d0/d12/d16/d1a/dc6/d137/c141 0 2024-09-21T04:15:31.021 INFO:tasks.workunit.client.0.smithi110.stdout:3/998: creat d0/f161 x:0 0 0 2024-09-21T04:15:31.021 INFO:tasks.workunit.client.0.smithi110.stdout:3/999: write d0/d12/d16/d1a/d95/fc9 [403976,98498] 0 2024-09-21T04:15:31.022 INFO:tasks.workunit.client.0.smithi110.stdout:7/992: unlink dd/d1c/dad/db2/d56/d66/fb6 0 2024-09-21T04:15:31.023 INFO:tasks.workunit.client.0.smithi110.stdout:2/971: dread d2/d50/ff6 [0,4194304] 0 2024-09-21T04:15:31.025 INFO:tasks.workunit.client.0.smithi110.stdout:7/993: rmdir dd/d14/d17/d143 0 2024-09-21T04:15:31.026 INFO:tasks.workunit.client.0.smithi110.stdout:2/972: symlink d2/d50/d7b/l141 0 2024-09-21T04:15:31.027 INFO:tasks.workunit.client.0.smithi110.stdout:7/994: mkdir dd/d1c/dad/db2/d56/d66/da6/d98/d146 0 2024-09-21T04:15:31.027 INFO:tasks.workunit.client.0.smithi110.stdout:2/973: mknod d2/de7/d116/d8b/d134/c142 0 2024-09-21T04:15:31.028 INFO:tasks.workunit.client.0.smithi110.stdout:2/974: fdatasync d2/d50/d9c/f13b 0 2024-09-21T04:15:31.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/995: symlink dd/d1c/dad/db2/d4a/da5/d111/l147 0 2024-09-21T04:15:31.029 INFO:tasks.workunit.client.0.smithi110.stdout:2/975: symlink d2/de7/d116/d60/l143 0 2024-09-21T04:15:31.030 INFO:tasks.workunit.client.0.smithi110.stdout:7/996: symlink dd/d1c/dad/db2/d56/d66/da6/d98/l148 0 2024-09-21T04:15:31.031 INFO:tasks.workunit.client.0.smithi110.stdout:2/976: chown d2/de7/d116/d90/cce 1467409626 1 2024-09-21T04:15:31.032 INFO:tasks.workunit.client.0.smithi110.stdout:7/997: creat dd/d1c/dad/db2/d4a/d52/f149 x:0 0 0 2024-09-21T04:15:31.032 INFO:tasks.workunit.client.0.smithi110.stdout:7/998: creat dd/d1c/dad/db2/d4a/da5/f14a x:0 0 0 2024-09-21T04:15:31.032 INFO:tasks.workunit.client.0.smithi110.stdout:7/999: write dd/d14/d17/fc3 [1148241,119679] 0 2024-09-21T04:15:31.034 INFO:tasks.workunit.client.0.smithi110.stdout:9/963: sync 2024-09-21T04:15:31.039 INFO:tasks.workunit.client.0.smithi110.stdout:2/977: dread d2/de7/d116/d8b/d10b/d5f/f1d [0,4194304] 0 2024-09-21T04:15:31.039 INFO:tasks.workunit.client.0.smithi110.stdout:2/978: write d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/f118 [472882,122434] 0 2024-09-21T04:15:31.040 INFO:tasks.workunit.client.0.smithi110.stdout:2/979: mkdir d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/df4/d12a/d144 0 2024-09-21T04:15:31.040 INFO:tasks.workunit.client.0.smithi110.stdout:2/980: dread - d2/de7/d116/d8b/d10b/d5f/d9d/d86/d9f/f11f zero size 2024-09-21T04:15:31.040 INFO:tasks.workunit.client.0.smithi110.stdout:2/981: chown d2/de7/d116/d8b/d10b/d5f/d9d/d86/c106 1 1 2024-09-21T04:15:31.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/982: symlink d2/de7/d116/d8b/da5/l145 0 2024-09-21T04:15:31.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/983: stat d2/d50/f52 0 2024-09-21T04:15:31.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/984: chown d2/de7/d127 151559 1 2024-09-21T04:15:31.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/964: dwrite d7/d10/d12a/d35/f137 [0,4194304] 0 2024-09-21T04:15:31.054 INFO:tasks.workunit.client.0.smithi110.stdout:2/985: dread d2/de7/d116/d8b/d10b/d5f/d9d/f78 [0,4194304] 0 2024-09-21T04:15:31.060 INFO:tasks.workunit.client.0.smithi110.stdout:9/965: dread d7/d10/d12a/d11/f8c [0,4194304] 0 2024-09-21T04:15:31.062 INFO:tasks.workunit.client.0.smithi110.stdout:9/966: symlink d7/d10/d12a/d1f/d25/l14b 0 2024-09-21T04:15:31.062 INFO:tasks.workunit.client.0.smithi110.stdout:9/967: dread - d7/d10/d12a/d4f/f112 zero size 2024-09-21T04:15:31.071 INFO:tasks.workunit.client.0.smithi110.stdout:2/986: dwrite d2/de7/d116/d8b/d10b/d29/f4c [0,4194304] 0 2024-09-21T04:15:31.073 INFO:tasks.workunit.client.0.smithi110.stdout:2/987: symlink d2/de7/d116/d8b/d10b/d29/d112/l146 0 2024-09-21T04:15:31.073 INFO:tasks.workunit.client.0.smithi110.stdout:2/988: rmdir d2/de7/d116/da3/db9/d10f/d110 39 2024-09-21T04:15:31.075 INFO:tasks.workunit.client.0.smithi110.stdout:9/968: dwrite d7/d10/d12a/d1f/d72/f4d [0,4194304] 0 2024-09-21T04:15:31.077 INFO:tasks.workunit.client.0.smithi110.stdout:9/969: truncate d7/d10/d12a/d11/f8b 2735927 0 2024-09-21T04:15:31.082 INFO:tasks.workunit.client.0.smithi110.stdout:9/970: write d7/d10/d12a/d1f/d43/f92 [1401479,99838] 0 2024-09-21T04:15:31.082 INFO:tasks.workunit.client.0.smithi110.stdout:9/971: getdents d7/d10/d88/d8f/d12c/d146 0 2024-09-21T04:15:31.082 INFO:tasks.workunit.client.0.smithi110.stdout:2/989: write d2/de7/d116/d8b/d10b/d65/f68 [1925602,1830] 0 2024-09-21T04:15:31.086 INFO:tasks.workunit.client.0.smithi110.stdout:2/990: mkdir d2/de7/d116/d8b/d10b/d29/d19/d147 0 2024-09-21T04:15:31.093 INFO:tasks.workunit.client.0.smithi110.stdout:9/972: dread d7/d10/d12a/d35/d71/ffc [0,4194304] 0 2024-09-21T04:15:31.093 INFO:tasks.workunit.client.0.smithi110.stdout:9/973: readlink d7/d10/d12a/d4f/d101/l149 0 2024-09-21T04:15:31.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/974: truncate d7/d10/d12a/d1f/f23 850405 0 2024-09-21T04:15:31.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/975: write d7/d10/d12a/d1f/d72/f4d [5108338,121219] 0 2024-09-21T04:15:31.099 INFO:tasks.workunit.client.0.smithi110.stdout:2/991: dwrite d2/de7/d116/d8b/d10b/d5f/f6f [0,4194304] 0 2024-09-21T04:15:31.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/992: creat d2/d50/f148 x:0 0 0 2024-09-21T04:15:31.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/993: chown d2/de7/d116/d8b/d10b/l96 134816570 1 2024-09-21T04:15:31.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/994: creat d2/de7/d116/d60/dfd/f149 x:0 0 0 2024-09-21T04:15:31.119 INFO:tasks.workunit.client.0.smithi110.stdout:9/976: dwrite d7/d10/d12a/d1f/d43/db5/ffe [0,4194304] 0 2024-09-21T04:15:31.119 INFO:tasks.workunit.client.0.smithi110.stdout:2/995: dwrite d2/d50/f52 [0,4194304] 0 2024-09-21T04:15:31.120 INFO:tasks.workunit.client.0.smithi110.stdout:2/996: stat d2/de7/d116/d8b/d10b/d2a/ff0 0 2024-09-21T04:15:31.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/977: dread d7/d10/d12a/d35/d71/fc5 [0,4194304] 0 2024-09-21T04:15:31.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/978: dread - d7/d10/d12a/d14/fc7 zero size 2024-09-21T04:15:31.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/979: chown d7/d10/d12a/d4f/d9f/d6f/d78/l125 1909930380 1 2024-09-21T04:15:31.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/980: chown d7/d10/d12a/d35/f141 8866113 1 2024-09-21T04:15:31.124 INFO:tasks.workunit.client.0.smithi110.stdout:2/997: symlink d2/de7/d116/d8b/d10b/d29/d19/d136/l14a 0 2024-09-21T04:15:31.127 INFO:tasks.workunit.client.0.smithi110.stdout:9/981: unlink d7/d10/d12a/d35/d109/l13b 0 2024-09-21T04:15:31.127 INFO:tasks.workunit.client.0.smithi110.stdout:9/982: write d7/d10/d12a/d4f/d9f/d6f/d78/f12d [1040104,92605] 0 2024-09-21T04:15:31.127 INFO:tasks.workunit.client.0.smithi110.stdout:2/998: rmdir d2/de7/d116/d8b/da5 39 2024-09-21T04:15:31.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/983: creat d7/d10/d12a/d1f/d72/d130/f14c x:0 0 0 2024-09-21T04:15:31.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/984: chown d7/d10/d12a/d4f/l117 1192018244 1 2024-09-21T04:15:31.131 INFO:tasks.workunit.client.0.smithi110.stdout:9/985: rmdir d7/d10/d88/d8f/da0 39 2024-09-21T04:15:31.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/999: dread d2/de7/d116/d8b/d10b/d5f/f1d [0,4194304] 0 2024-09-21T04:15:31.152 INFO:tasks.workunit.client.0.smithi110.stdout:9/986: dwrite d7/d10/d12a/fdf [4194304,4194304] 0 2024-09-21T04:15:31.154 INFO:tasks.workunit.client.0.smithi110.stdout:9/987: unlink d7/d96/d99/f38 0 2024-09-21T04:15:31.175 INFO:tasks.workunit.client.0.smithi110.stdout:9/988: dwrite d7/d10/d12a/d1f/d43/f6c [0,4194304] 0 2024-09-21T04:15:31.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/989: creat d7/d10/d12a/d14/f14d x:0 0 0 2024-09-21T04:15:31.192 INFO:tasks.workunit.client.0.smithi110.stdout:9/990: dwrite d7/d96/d99/ff9 [0,4194304] 0 2024-09-21T04:15:31.198 INFO:tasks.workunit.client.0.smithi110.stdout:9/991: mknod d7/d10/d88/d8f/d12e/d145/c14e 0 2024-09-21T04:15:31.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/992: sync 2024-09-21T04:15:31.672 INFO:tasks.workunit.client.0.smithi110.stdout:9/993: rename d7/d10/d12a/d14/f54 to d7/d10/d88/d8f/d12e/f14f 0 2024-09-21T04:15:31.693 INFO:tasks.workunit.client.0.smithi110.stdout:9/994: dwrite d7/d10/d12a/d14/fc2 [0,4194304] 0 2024-09-21T04:15:31.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/995: dread d7/d10/d12a/d11/f4e [0,4194304] 0 2024-09-21T04:15:31.705 INFO:tasks.workunit.client.0.smithi110.stdout:9/996: mkdir d7/d10/d12a/d1f/d72/d130/d150 0 2024-09-21T04:15:31.705 INFO:tasks.workunit.client.0.smithi110.stdout:9/997: chown d7/d10/d12a/d14 234350254 1 2024-09-21T04:15:31.705 INFO:tasks.workunit.client.0.smithi110.stdout:9/998: chown d7/d10/d12a/d1f/d43/db5/dca 844236041 1 2024-09-21T04:15:31.707 INFO:tasks.workunit.client.0.smithi110.stdout:9/999: creat d7/d10/d12a/d1f/d43/db5/dca/d120/f151 x:0 0 0 2024-09-21T04:15:31.712 INFO:tasks.workunit.client.0.smithi110.stderr:+ rm -rf -- ./tmp.zpC9udqq9t 2024-09-21T04:15:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:32 smithi110 ceph-mon[27723]: pgmap v13: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 234 MiB/s rd, 301 MiB/s wr, 408 op/s 2024-09-21T04:15:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:32 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:32 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:32 smithi148 ceph-mon[34957]: pgmap v13: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 234 MiB/s rd, 301 MiB/s wr, 408 op/s 2024-09-21T04:15:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:32 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:32 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: Adjusting osd_memory_target on smithi110 to 1216M 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:15:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:34.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:34 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: Adjusting osd_memory_target on smithi110 to 1216M 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:34.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:34 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: pgmap v14: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 279 MiB/s rd, 359 MiB/s wr, 539 op/s 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Reconfiguring ceph-exporter.smithi110 (dependencies changed)... 2024-09-21T04:15:35.406 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:35 smithi110 ceph-mon[27723]: Deploying daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:15:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:15:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:15:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: pgmap v14: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 279 MiB/s rd, 359 MiB/s wr, 539 op/s 2024-09-21T04:15:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:15:35.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:15:35.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:15:35.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:15:35.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Reconfiguring ceph-exporter.smithi110 (dependencies changed)... 2024-09-21T04:15:35.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:35 smithi148 ceph-mon[34957]: Deploying daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:15:37.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:37 smithi110 ceph-mon[27723]: pgmap v15: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 344 op/s 2024-09-21T04:15:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:37 smithi148 ceph-mon[34957]: pgmap v15: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 344 op/s 2024-09-21T04:15:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:38 smithi110 ceph-mon[27723]: pgmap v16: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 344 op/s 2024-09-21T04:15:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:38 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:38 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:38 smithi110 ceph-mon[27723]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:15:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:38 smithi110 ceph-mon[27723]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:15:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:38 smithi148 ceph-mon[34957]: pgmap v16: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 344 op/s 2024-09-21T04:15:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:38 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:38 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:38 smithi148 ceph-mon[34957]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:15:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:38 smithi148 ceph-mon[34957]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:15:41.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:40 smithi110 ceph-mon[27723]: pgmap v17: 65 pgs: 65 active+clean; 2.5 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 446 op/s 2024-09-21T04:15:41.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:40 smithi148 ceph-mon[34957]: pgmap v17: 65 pgs: 65 active+clean; 2.5 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 167 MiB/s rd, 218 MiB/s wr, 446 op/s 2024-09-21T04:15:41.378 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T04:15:41.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-21T04:15:42.826 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:42 smithi110 ceph-mon[27723]: pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 59 MiB/s wr, 233 op/s 2024-09-21T04:15:42.826 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:42 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:43.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:42 smithi148 ceph-mon[34957]: pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 59 MiB/s wr, 233 op/s 2024-09-21T04:15:43.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:42 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: Reconfiguring ceph-exporter.smithi148 (dependencies changed)... 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:15:44.629 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:44 smithi148 ceph-mon[34957]: pgmap v19: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 45 MiB/s rd, 59 MiB/s wr, 321 op/s 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: Reconfiguring ceph-exporter.smithi148 (dependencies changed)... 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:15:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:44 smithi110 ceph-mon[27723]: pgmap v19: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 45 MiB/s rd, 59 MiB/s wr, 321 op/s 2024-09-21T04:15:45.408 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T04:15:45.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T04:15:46.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:46 smithi110 ceph-mon[27723]: pgmap v20: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-21T04:15:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:46 smithi148 ceph-mon[34957]: pgmap v20: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: pgmap v21: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: Upgrade: Need to upgrade myself (mgr.smithi148.zjrahi) 2024-09-21T04:15:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:48 smithi110 ceph-mon[27723]: Upgrade: Need to upgrade myself (mgr.smithi148.zjrahi) 2024-09-21T04:15:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: pgmap v21: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-21T04:15:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: Upgrade: Need to upgrade myself (mgr.smithi148.zjrahi) 2024-09-21T04:15:48.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:48 smithi148 ceph-mon[34957]: Upgrade: Need to upgrade myself (mgr.smithi148.zjrahi) 2024-09-21T04:15:49.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 -- 172.21.15.110:0/766604981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b01074b0 msgr2=0x7f88b0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 --2- 172.21.15.110:0/766604981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b01074b0 0x7f88b0107930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009880 tx=0x7f88a002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 -- 172.21.15.110:0/766604981 shutdown_connections 2024-09-21T04:15:49.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 --2- 172.21.15.110:0/766604981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b01074b0 0x7f88b0107930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 --2- 172.21.15.110:0/766604981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b0071430 0x7f88b0071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.003+0000 7f88b595f640 1 -- 172.21.15.110:0/766604981 >> 172.21.15.110:0/766604981 conn(0x7f88b006cc80 msgr2=0x7f88b006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:49.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.004+0000 7f88b595f640 1 -- 172.21.15.110:0/766604981 shutdown_connections 2024-09-21T04:15:49.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.004+0000 7f88b595f640 1 -- 172.21.15.110:0/766604981 wait complete. 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 Processor -- start 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 -- start start 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 0x7f88b010efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 0x7f88b010f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b010fac0 con 0x7f88b01074b0 2024-09-21T04:15:49.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.005+0000 7f88b595f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b010fc30 con 0x7f88b0071430 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 0x7f88b010f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 0x7f88b010f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35670/0 (socket says 172.21.15.110:35670) 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88a7fff640 1 -- 172.21.15.110:0/2477362231 learned_addr learned my addr 172.21.15.110:0/2477362231 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88aeffd640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 0x7f88b010efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88aeffd640 1 -- 172.21.15.110:0/2477362231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 msgr2=0x7f88b010f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88aeffd640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 0x7f88b010f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.006+0000 7f88aeffd640 1 -- 172.21.15.110:0/2477362231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a0009560 con 0x7f88b0071430 2024-09-21T04:15:49.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.007+0000 7f88aeffd640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 0x7f88b010efb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f889800b340 tx=0x7f889800b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.007+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8898011800 con 0x7f88b0071430 2024-09-21T04:15:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.007+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8898011e20 con 0x7f88b0071430 2024-09-21T04:15:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.007+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889800f2e0 con 0x7f88b0071430 2024-09-21T04:15:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.008+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b0114700 con 0x7f88b0071430 2024-09-21T04:15:49.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.008+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0114ca0 con 0x7f88b0071430 2024-09-21T04:15:49.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.011+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7f8898010380 con 0x7f88b0071430 2024-09-21T04:15:49.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.012+0000 7f88b495d640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 0x7f889c0764a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.012+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f88980949b0 con 0x7f88b0071430 2024-09-21T04:15:49.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.013+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0117fa0 con 0x7f88b0071430 2024-09-21T04:15:49.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.013+0000 7f88a7fff640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 0x7f889c0764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.014+0000 7f88a7fff640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 0x7f889c0764a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88b01104d0 tx=0x7f88a00314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.024+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f8898057f40 con 0x7f88b0071430 2024-09-21T04:15:49.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.370+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88b010be20 con 0x7f889c073fe0 2024-09-21T04:15:49.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.371+0000 7f88b495d640 1 -- 172.21.15.110:0/2477362231 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f88b010be20 con 0x7f889c073fe0 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.373+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 msgr2=0x7f889c0764a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.373+0000 7f88b595f640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 0x7f889c0764a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88b01104d0 tx=0x7f88a00314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.373+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 msgr2=0x7f88b010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.373+0000 7f88b595f640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 0x7f88b010efb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f889800b340 tx=0x7f889800b800 comp rx=0 tx=0).stop 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 shutdown_connections 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f889c073fe0 0x7f889c0764a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b01074b0 0x7f88b010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 --2- 172.21.15.110:0/2477362231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88b0071430 0x7f88b010efb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 >> 172.21.15.110:0/2477362231 conn(0x7f88b006cc80 msgr2=0x7f88b010a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 shutdown_connections 2024-09-21T04:15:49.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.374+0000 7f88b595f640 1 -- 172.21.15.110:0/2477362231 wait complete. 2024-09-21T04:15:49.385 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:15:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 -- 172.21.15.110:0/1030809063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44102ef0 msgr2=0x7fea441052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 --2- 172.21.15.110:0/1030809063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44102ef0 0x7fea441052e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fea380098b0 tx=0x7fea3802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 -- 172.21.15.110:0/1030809063 shutdown_connections 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 --2- 172.21.15.110:0/1030809063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44102ef0 0x7fea441052e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 --2- 172.21.15.110:0/1030809063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea441005c0 0x7fea441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 -- 172.21.15.110:0/1030809063 >> 172.21.15.110:0/1030809063 conn(0x7fea440fa1c0 msgr2=0x7fea440fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 -- 172.21.15.110:0/1030809063 shutdown_connections 2024-09-21T04:15:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.475+0000 7fea48ade640 1 -- 172.21.15.110:0/1030809063 wait complete. 2024-09-21T04:15:49.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.476+0000 7fea48ade640 1 Processor -- start 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea48ade640 1 -- start start 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea48ade640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea48ade640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 0x7fea440fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea48ade640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440feab0 con 0x7fea441005c0 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea48ade640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440fec20 con 0x7fea44102ef0 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35682/0 (socket says 172.21.15.110:35682) 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 -- 172.21.15.110:0/1001295015 learned_addr learned my addr 172.21.15.110:0/1001295015 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea41d9b640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 0x7fea440fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 -- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 msgr2=0x7fea440fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 0x7fea440fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea4259c640 1 -- 172.21.15.110:0/1001295015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38009560 con 0x7fea441005c0 2024-09-21T04:15:49.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.477+0000 7fea41d9b640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 0x7fea440fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.478+0000 7fea4259c640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d8d0 tx=0x7fea3400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.478+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3400bea0 con 0x7fea441005c0 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.478+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea34009c00 con 0x7fea441005c0 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.478+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3400f710 con 0x7fea441005c0 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.478+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea440fef00 con 0x7fea441005c0 2024-09-21T04:15:49.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.479+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4419d6a0 con 0x7fea441005c0 2024-09-21T04:15:49.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.480+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea441009a0 con 0x7fea441005c0 2024-09-21T04:15:49.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.480+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7fea34010460 con 0x7fea441005c0 2024-09-21T04:15:49.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.481+0000 7fea337fe640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 0x7fea2c0763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.481+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fea34094860 con 0x7fea441005c0 2024-09-21T04:15:49.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.481+0000 7fea41d9b640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 0x7fea2c0763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.482+0000 7fea41d9b640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 0x7fea2c0763a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a9f0 tx=0x7fea38005de0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.486+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fea34057da0 con 0x7fea441005c0 2024-09-21T04:15:49.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.718+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea44103910 con 0x7fea2c073ee0 2024-09-21T04:15:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.719+0000 7fea337fe640 1 -- 172.21.15.110:0/1001295015 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fea44103910 con 0x7fea2c073ee0 2024-09-21T04:15:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 msgr2=0x7fea2c0763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 0x7fea2c0763a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a9f0 tx=0x7fea38005de0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 msgr2=0x7fea440fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d8d0 tx=0x7fea3400dd90 comp rx=0 tx=0).stop 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 shutdown_connections 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fea2c073ee0 0x7fea2c0763a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea44102ef0 0x7fea440fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 --2- 172.21.15.110:0/1001295015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441005c0 0x7fea440fff20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 >> 172.21.15.110:0/1001295015 conn(0x7fea440fa1c0 msgr2=0x7fea440fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 shutdown_connections 2024-09-21T04:15:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.722+0000 7fea48ade640 1 -- 172.21.15.110:0/1001295015 wait complete. 2024-09-21T04:15:49.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 -- 172.21.15.110:0/2036219223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0ffc30 msgr2=0x7f66ec1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 --2- 172.21.15.110:0/2036219223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0ffc30 0x7f66ec1000b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009880 tx=0x7f66d802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 -- 172.21.15.110:0/2036219223 shutdown_connections 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 --2- 172.21.15.110:0/2036219223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0ffc30 0x7f66ec1000b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 --2- 172.21.15.110:0/2036219223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0fea30 0x7f66ec0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 -- 172.21.15.110:0/2036219223 >> 172.21.15.110:0/2036219223 conn(0x7f66ec0fa1c0 msgr2=0x7f66ec0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 -- 172.21.15.110:0/2036219223 shutdown_connections 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 -- 172.21.15.110:0/2036219223 wait complete. 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.821+0000 7f66f30b5640 1 Processor -- start 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f30b5640 1 -- start start 2024-09-21T04:15:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f30b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f30b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0ffc30 0x7f66ec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f30b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec19aa70 con 0x7f66ec0fea30 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f30b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec19abb0 con 0x7f66ec0ffc30 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35706/0 (socket says 172.21.15.110:35706) 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 -- 172.21.15.110:0/1595530255 learned_addr learned my addr 172.21.15.110:0/1595530255 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66ebfff640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0ffc30 0x7f66ec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 -- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0ffc30 msgr2=0x7f66ec19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0ffc30 0x7f66ec19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 -- 172.21.15.110:0/1595530255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8009560 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.822+0000 7f66f0e51640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f66e000b3d0 tx=0x7f66e000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.823+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e0011800 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.823+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66e0011e20 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.823+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec19ae60 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.823+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e000f510 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.823+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec19b3a0 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.825+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec0feeb0 con 0x7f66ec0fea30 2024-09-21T04:15:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.825+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7f66e0011960 con 0x7f66ec0fea30 2024-09-21T04:15:49.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.826+0000 7f66e9ffb640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 0x7f66dc076180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:49.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.826+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f66e0094a00 con 0x7f66ec0fea30 2024-09-21T04:15:49.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.826+0000 7f66ebfff640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 0x7f66dc076180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:49.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.826+0000 7f66ebfff640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 0x7f66dc076180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d80314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:49.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:49.833+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f66e0057fb0 con 0x7f66ec0fea30 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: Upgrade: Updating mgr.smithi110.axmqzb 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:50.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:49 smithi110 ceph-mon[27723]: Deploying daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:15:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.146+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66ec104580 con 0x7f66dc073cc0 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.164+0000 7f66e9ffb640 1 -- 172.21.15.110:0/1595530255 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f66ec104580 con 0x7f66dc073cc0 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (7m) 17s ago 9m 25.3M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 starting - - - - 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 starting - - - - 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (9m) 17s ago 9m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (8m) 30s ago 8m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (7m) 17s ago 8m 85.4M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (3m) 17s ago 3m 205M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (3m) 17s ago 3m 13.4M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:15:50.167 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (2m) 30s ago 2m 15.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (3m) 30s ago 3m 16.6M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:9283,8765,8443 running (10m) 17s ago 10m 257M - 18.2.4-871-g6a1bdf85 0adab276c97e dfdcdc429f9f 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (61s) 30s ago 8m 496M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0d777a7727d1 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (10m) 17s ago 10m 52.6M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (8m) 30s ago 8m 39.3M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (9m) 17s ago 9m 16.5M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (8m) 30s ago 8m 15.3M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 17s ago 6m 219M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (6m) 17s ago 6m 239M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 17s ago 5m 201M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (5m) 30s ago 5m 271M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (5m) 30s ago 5m 224M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (4m) 30s ago 4m 274M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:15:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 starting - - - - 2024-09-21T04:15:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 msgr2=0x7f66dc076180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 0x7f66dc076180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d80314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 msgr2=0x7f66ec068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f66e000b3d0 tx=0x7f66e000b890 comp rx=0 tx=0).stop 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 shutdown_connections 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f66dc073cc0 0x7f66dc076180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66ec0ffc30 0x7f66ec19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 --2- 172.21.15.110:0/1595530255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66ec0fea30 0x7f66ec068740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 >> 172.21.15.110:0/1595530255 conn(0x7f66ec0fa1c0 msgr2=0x7f66ec102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 shutdown_connections 2024-09-21T04:15:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.168+0000 7f66f30b5640 1 -- 172.21.15.110:0/1595530255 wait complete. 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: Upgrade: Updating mgr.smithi110.axmqzb 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:15:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:49 smithi148 ceph-mon[34957]: Deploying daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:15:50.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 -- 172.21.15.110:0/1912938869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780fea80 msgr2=0x7f06780fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 --2- 172.21.15.110:0/1912938869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780fea80 0x7f06780fee80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0668009880 tx=0x7f066802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 -- 172.21.15.110:0/1912938869 shutdown_connections 2024-09-21T04:15:50.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 --2- 172.21.15.110:0/1912938869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780ffc80 0x7f0678100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 --2- 172.21.15.110:0/1912938869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780fea80 0x7f06780fee80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 -- 172.21.15.110:0/1912938869 >> 172.21.15.110:0/1912938869 conn(0x7f06780fa230 msgr2=0x7f06780fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:50.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 -- 172.21.15.110:0/1912938869 shutdown_connections 2024-09-21T04:15:50.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.283+0000 7f067e043640 1 -- 172.21.15.110:0/1912938869 wait complete. 2024-09-21T04:15:50.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.284+0000 7f067e043640 1 Processor -- start 2024-09-21T04:15:50.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.284+0000 7f067e043640 1 -- start start 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f067e043640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780fea80 0x7f06781955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f067e043640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f067e043640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06781960f0 con 0x7f06780ffc80 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f067e043640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678196260 con 0x7f06780fea80 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35722/0 (socket says 172.21.15.110:35722) 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 -- 172.21.15.110:0/4163835109 learned_addr learned my addr 172.21.15.110:0/4163835109 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f06777fe640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780fea80 0x7f06781955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 -- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780fea80 msgr2=0x7f06781955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780fea80 0x7f06781955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 -- 172.21.15.110:0/4163835109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0668009560 con 0x7f06780ffc80 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.285+0000 7f0676ffd640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c840 tx=0x7f066c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.286+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066c0041f0 con 0x7f06780ffc80 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.286+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066c007e70 con 0x7f06780ffc80 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.286+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067819ad00 con 0x7f06780ffc80 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.286+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066c0049d0 con 0x7f06780ffc80 2024-09-21T04:15:50.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.286+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067819b2d0 con 0x7f06780ffc80 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.287+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067818f730 con 0x7f06780ffc80 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.287+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7f066c0079b0 con 0x7f06780ffc80 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.288+0000 7f0674ff9640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 0x7f06600763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.288+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f066c094f90 con 0x7f06780ffc80 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.288+0000 7f06777fe640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 0x7f06600763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.289+0000 7f06777fe640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 0x7f06600763a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0668009230 tx=0x7f06680314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:50.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.293+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f066c0cbce0 con 0x7f06780ffc80 2024-09-21T04:15:50.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.612+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0678068f20 con 0x7f06780ffc80 2024-09-21T04:15:50.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.612+0000 7f0674ff9640 1 -- 172.21.15.110:0/4163835109 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f066c058480 con 0x7f06780ffc80 2024-09-21T04:15:50.615 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 1, 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:50.616 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 13, 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:15:50.617 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 msgr2=0x7f06600763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 0x7f06600763a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0668009230 tx=0x7f06680314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 msgr2=0x7f0678195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c840 tx=0x7f066c00cd00 comp rx=0 tx=0).stop 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 shutdown_connections 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f0660073ee0 0x7f06600763a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06780ffc80 0x7f0678195b20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 --2- 172.21.15.110:0/4163835109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06780fea80 0x7f06781955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 >> 172.21.15.110:0/4163835109 conn(0x7f06780fa230 msgr2=0x7f06781032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:50.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 shutdown_connections 2024-09-21T04:15:50.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.616+0000 7f067e043640 1 -- 172.21.15.110:0/4163835109 wait complete. 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.732+0000 7f7a5af68640 1 -- 172.21.15.110:0/3506801962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a540713b0 msgr2=0x7f7a540717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.732+0000 7f7a5af68640 1 --2- 172.21.15.110:0/3506801962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a540713b0 0x7f7a540717b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009880 tx=0x7f7a4802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 -- 172.21.15.110:0/3506801962 shutdown_connections 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 --2- 172.21.15.110:0/3506801962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 0x7f7a54108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 --2- 172.21.15.110:0/3506801962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a540713b0 0x7f7a540717b0 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009880 tx=0x7f7a4802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 -- 172.21.15.110:0/3506801962 >> 172.21.15.110:0/3506801962 conn(0x7f7a5406cc80 msgr2=0x7f7a5406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:50.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 -- 172.21.15.110:0/3506801962 shutdown_connections 2024-09-21T04:15:50.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.733+0000 7f7a5af68640 1 -- 172.21.15.110:0/3506801962 wait complete. 2024-09-21T04:15:50.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.734+0000 7f7a5af68640 1 Processor -- start 2024-09-21T04:15:50.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.734+0000 7f7a5af68640 1 -- start start 2024-09-21T04:15:50.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a5af68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 0x7f7a5419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a5af68640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a5af68640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5419d8a0 con 0x7f7a5419d420 2024-09-21T04:15:50.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a5af68640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5419da10 con 0x7f7a541085c0 2024-09-21T04:15:50.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a58d04640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 0x7f7a5419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35736/0 (socket says 172.21.15.110:35736) 2024-09-21T04:15:50.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a53fff640 1 -- 172.21.15.110:0/2275315038 learned_addr learned my addr 172.21.15.110:0/2275315038 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:50.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a53fff640 1 -- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 msgr2=0x7f7a5419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:50.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.735+0000 7f7a53fff640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 0x7f7a5419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a53fff640 1 -- 172.21.15.110:0/2275315038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009560 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a53fff640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c8a0 tx=0x7f7a4000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a40007b80 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a40007ce0 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a541074b0 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a400179a0 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.736+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a541079d0 con 0x7f7a5419d420 2024-09-21T04:15:50.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.738+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5410fe90 con 0x7f7a5419d420 2024-09-21T04:15:50.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.739+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7f7a4000f550 con 0x7f7a5419d420 2024-09-21T04:15:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.739+0000 7f7a51ffb640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 0x7f7a440763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.739+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7a40096f10 con 0x7f7a5419d420 2024-09-21T04:15:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.740+0000 7f7a58d04640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 0x7f7a440763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:50.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.740+0000 7f7a58d04640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 0x7f7a440763e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006010 tx=0x7f7a480314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:50.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:50.744+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f7a4005dcd0 con 0x7f7a5419d420 2024-09-21T04:15:51.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:50 smithi110 ceph-mon[27723]: pgmap v22: 65 pgs: 65 active+clean; 296 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-09-21T04:15:51.020 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:50 smithi110 ceph-mon[27723]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:51.020 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:50 smithi110 ceph-mon[27723]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:51.020 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:50 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4163835109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:15:51.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.068+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a54107c40 con 0x7f7a5419d420 2024-09-21T04:15:51.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.070+0000 7f7a51ffb640 1 -- 172.21.15.110:0/2275315038 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f7a4005d670 con 0x7f7a5419d420 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:15:51.073 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:15:51.074 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:51.075 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:51.076 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:15:51.076 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:15:51.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.074+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 msgr2=0x7f7a440763e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.074+0000 7f7a5af68640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 0x7f7a440763e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006010 tx=0x7f7a480314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.074+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 msgr2=0x7f7a541a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.074+0000 7f7a5af68640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c8a0 tx=0x7f7a4000cd60 comp rx=0 tx=0).stop 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 shutdown_connections 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f7a44073f20 0x7f7a440763e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a5419d420 0x7f7a541a2490 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 --2- 172.21.15.110:0/2275315038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a541085c0 0x7f7a5419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 >> 172.21.15.110:0/2275315038 conn(0x7f7a5406cc80 msgr2=0x7f7a5410b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 shutdown_connections 2024-09-21T04:15:51.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.075+0000 7f7a5af68640 1 -- 172.21.15.110:0/2275315038 wait complete. 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 -- 172.21.15.110:0/681263994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0fea30 msgr2=0x7fd28c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 --2- 172.21.15.110:0/681263994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0fea30 0x7fd28c0fee30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd280009880 tx=0x7fd28002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 -- 172.21.15.110:0/681263994 shutdown_connections 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 --2- 172.21.15.110:0/681263994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0ffc30 0x7fd28c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 --2- 172.21.15.110:0/681263994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0fea30 0x7fd28c0fee30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.166+0000 7fd293206640 1 -- 172.21.15.110:0/681263994 >> 172.21.15.110:0/681263994 conn(0x7fd28c0fa1c0 msgr2=0x7fd28c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 -- 172.21.15.110:0/681263994 shutdown_connections 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 -- 172.21.15.110:0/681263994 wait complete. 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 Processor -- start 2024-09-21T04:15:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 -- start start 2024-09-21T04:15:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0fea30 0x7fd28c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c196130 con 0x7fd28c0ffc30 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd293206640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c1962a0 con 0x7fd28c0fea30 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd28bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.167+0000 7fd28bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35764/0 (socket says 172.21.15.110:35764) 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd28bfff640 1 -- 172.21.15.110:0/1902235192 learned_addr learned my addr 172.21.15.110:0/1902235192 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd290fa2640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0fea30 0x7fd28c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd28bfff640 1 -- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0fea30 msgr2=0x7fd28c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd28bfff640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0fea30 0x7fd28c195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd28bfff640 1 -- 172.21.15.110:0/1902235192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009560 con 0x7fd28c0ffc30 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd28bfff640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9a0 tx=0x7fd27800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd27800cd50 con 0x7fd28c0ffc30 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd27800ceb0 con 0x7fd28c0ffc30 2024-09-21T04:15:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c19ad40 con 0x7fd28c0ffc30 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd278010660 con 0x7fd28c0ffc30 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.168+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c1010a0 con 0x7fd28c0ffc30 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.170+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c107640 con 0x7fd28c0ffc30 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.170+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7fd278004ef0 con 0x7fd28c0ffc30 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.170+0000 7fd289ffb640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 0x7fd27c07a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.171+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd278014070 con 0x7fd28c0ffc30 2024-09-21T04:15:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.171+0000 7fd290fa2640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 0x7fd27c07a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.171+0000 7fd290fa2640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 0x7fd27c07a7a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd280006010 tx=0x7fd2800314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:51.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.175+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fd278058b70 con 0x7fd28c0ffc30 2024-09-21T04:15:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:50 smithi148 ceph-mon[34957]: pgmap v22: 65 pgs: 65 active+clean; 296 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 353 op/s 2024-09-21T04:15:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:50 smithi148 ceph-mon[34957]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:50 smithi148 ceph-mon[34957]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:50 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4163835109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:15:51.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.432+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd28c0fc3d0 con 0x7fd27c0782e0 2024-09-21T04:15:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.433+0000 7fd289ffb640 1 -- 172.21.15.110:0/1902235192 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fd28c0fc3d0 con 0x7fd27c0782e0 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "1/23 daemons upgraded", 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mgr daemons", 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:15:51.436 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 msgr2=0x7fd27c07a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 0x7fd27c07a7a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd280006010 tx=0x7fd2800314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 msgr2=0x7fd28c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9a0 tx=0x7fd27800eda0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 shutdown_connections 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd27c0782e0 0x7fd27c07a7a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd28c0ffc30 0x7fd28c195b60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 --2- 172.21.15.110:0/1902235192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd28c0fea30 0x7fd28c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 >> 172.21.15.110:0/1902235192 conn(0x7fd28c0fa1c0 msgr2=0x7fd28c0fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 shutdown_connections 2024-09-21T04:15:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.436+0000 7fd293206640 1 -- 172.21.15.110:0/1902235192 wait complete. 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- 172.21.15.110:0/2668579554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe328102ef0 msgr2=0x7fe3281052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 --2- 172.21.15.110:0/2668579554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe328102ef0 0x7fe3281052e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3180098b0 tx=0x7fe31802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- 172.21.15.110:0/2668579554 shutdown_connections 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 --2- 172.21.15.110:0/2668579554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe328102ef0 0x7fe3281052e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 --2- 172.21.15.110:0/2668579554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3281005c0 0x7fe3281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- 172.21.15.110:0/2668579554 >> 172.21.15.110:0/2668579554 conn(0x7fe3280fa1c0 msgr2=0x7fe3280fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- 172.21.15.110:0/2668579554 shutdown_connections 2024-09-21T04:15:51.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- 172.21.15.110:0/2668579554 wait complete. 2024-09-21T04:15:51.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 Processor -- start 2024-09-21T04:15:51.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.545+0000 7fe33070a640 1 -- start start 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe33070a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe33070a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 0x7fe3280fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe33070a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280feab0 con 0x7fe3281005c0 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe33070a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280fec20 con 0x7fe328102ef0 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35786/0 (socket says 172.21.15.110:35786) 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 -- 172.21.15.110:0/316098535 learned_addr learned my addr 172.21.15.110:0/316098535 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32dca5640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 0x7fe3280fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 -- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 msgr2=0x7fe3280fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 0x7fe3280fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 -- 172.21.15.110:0/316098535 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318009560 con 0x7fe3281005c0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32dca5640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 0x7fe3280fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe32e4a6640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e970 tx=0x7fe32400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.546+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32400cc90 con 0x7fe3281005c0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.547+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3280fef00 con 0x7fe3281005c0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.547+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe32400cdf0 con 0x7fe3281005c0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.547+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe324010430 con 0x7fe3281005c0 2024-09-21T04:15:51.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.547+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32819d6c0 con 0x7fe3281005c0 2024-09-21T04:15:51.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.548+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3281009a0 con 0x7fe3281005c0 2024-09-21T04:15:51.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.548+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) v1 ==== 97116+0+0 (secure 0 0 0) 0x7fe324005020 con 0x7fe3281005c0 2024-09-21T04:15:51.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.549+0000 7fe31f7fe640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 0x7fe3140763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:15:51.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.549+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe324014070 con 0x7fe3281005c0 2024-09-21T04:15:51.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.549+0000 7fe32dca5640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 0x7fe3140763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:15:51.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.550+0000 7fe32dca5640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 0x7fe3140763a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe318009880 tx=0x7fe3180314e0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:15:51.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.554+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fe324058c90 con 0x7fe3281005c0 2024-09-21T04:15:51.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.882+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3281089a0 con 0x7fe3281005c0 2024-09-21T04:15:51.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.883+0000 7fe31f7fe640 1 -- 172.21.15.110:0/316098535 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe32405bdb0 con 0x7fe3281005c0 2024-09-21T04:15:51.886 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 msgr2=0x7fe3140763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 0x7fe3140763a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe318009880 tx=0x7fe3180314e0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 msgr2=0x7fe3280fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e970 tx=0x7fe32400ed70 comp rx=0 tx=0).stop 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 shutdown_connections 2024-09-21T04:15:51.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe314073ee0 0x7fe3140763a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe328102ef0 0x7fe3280fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 --2- 172.21.15.110:0/316098535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281005c0 0x7fe3280fff20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:15:51.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 >> 172.21.15.110:0/316098535 conn(0x7fe3280fa1c0 msgr2=0x7fe3280fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:15:51.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 shutdown_connections 2024-09-21T04:15:51.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:15:51.885+0000 7fe33070a640 1 -- 172.21.15.110:0/316098535 wait complete. 2024-09-21T04:15:52.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:51 smithi110 ceph-mon[27723]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:52.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:51 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2275315038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:15:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:51 smithi148 ceph-mon[34957]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:51 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2275315038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:15:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:52 smithi110 ceph-mon[27723]: pgmap v23: 65 pgs: 65 active+clean; 296 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-21T04:15:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:52 smithi110 ceph-mon[27723]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:52 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/316098535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:15:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:52 smithi148 ceph-mon[34957]: pgmap v23: 65 pgs: 65 active+clean; 296 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-21T04:15:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:52 smithi148 ceph-mon[34957]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:15:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:52 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/316098535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:15:55.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:54 smithi110 ceph-mon[27723]: pgmap v24: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.4 MiB/s wr, 424 op/s 2024-09-21T04:15:55.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:54 smithi148 ceph-mon[34957]: pgmap v24: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.4 MiB/s wr, 424 op/s 2024-09-21T04:15:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:56 smithi148 ceph-mon[34957]: pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 335 op/s 2024-09-21T04:15:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:56 smithi110 ceph-mon[27723]: pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 335 op/s 2024-09-21T04:15:58.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:57 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:57 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: pgmap v26: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 335 op/s 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:15:58 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: pgmap v26: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 335 op/s 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:15:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:15:58 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:01.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:00 smithi110 ceph-mon[27723]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 490 op/s 2024-09-21T04:16:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:00 smithi148 ceph-mon[34957]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 490 op/s 2024-09-21T04:16:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:01 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:01 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:02.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:01 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:02.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:01 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:03.340 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:02 smithi110 ceph-mon[27723]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 328 op/s 2024-09-21T04:16:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:02 smithi148 ceph-mon[34957]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 328 op/s 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: Standby manager daemon smithi110.axmqzb restarted 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: Standby manager daemon smithi110.axmqzb started 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/crt"}]: dispatch 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/key"}]: dispatch 2024-09-21T04:16:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:03 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: Standby manager daemon smithi110.axmqzb restarted 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: Standby manager daemon smithi110.axmqzb started 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/crt"}]: dispatch 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.axmqzb/key"}]: dispatch 2024-09-21T04:16:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:03 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:16:05.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:05 smithi110 ceph-mon[27723]: pgmap v29: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 478 op/s 2024-09-21T04:16:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:05 smithi110 ceph-mon[27723]: mgrmap e27: smithi148.zjrahi(active, since 52s), standbys: smithi110.axmqzb 2024-09-21T04:16:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:05 smithi148 ceph-mon[34957]: pgmap v29: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 478 op/s 2024-09-21T04:16:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:05 smithi148 ceph-mon[34957]: mgrmap e27: smithi148.zjrahi(active, since 52s), standbys: smithi110.axmqzb 2024-09-21T04:16:07.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:07 smithi110 ceph-mon[27723]: pgmap v30: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 305 op/s 2024-09-21T04:16:07.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:07 smithi148 ceph-mon[34957]: pgmap v30: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 305 op/s 2024-09-21T04:16:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:09 smithi110 ceph-mon[27723]: pgmap v31: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 305 op/s 2024-09-21T04:16:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:09 smithi148 ceph-mon[34957]: pgmap v31: 65 pgs: 65 active+clean; 308 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 305 op/s 2024-09-21T04:16:11.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:11 smithi110 ceph-mon[27723]: pgmap v32: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.7 MiB/s wr, 412 op/s 2024-09-21T04:16:11.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:11 smithi148 ceph-mon[34957]: pgmap v32: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.7 MiB/s wr, 412 op/s 2024-09-21T04:16:13.166 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:13 smithi110 ceph-mon[27723]: pgmap v33: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-21T04:16:13.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:13 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:13 smithi148 ceph-mon[34957]: pgmap v33: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-21T04:16:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:13 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:15.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:15 smithi110 ceph-mon[27723]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 3.1 MiB/s wr, 652 op/s 2024-09-21T04:16:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:15 smithi148 ceph-mon[34957]: pgmap v34: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 3.1 MiB/s wr, 652 op/s 2024-09-21T04:16:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:16 smithi110 ceph-mon[27723]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 501 op/s 2024-09-21T04:16:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:16 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:16 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:16 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:16 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:16 smithi148 ceph-mon[34957]: pgmap v35: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 501 op/s 2024-09-21T04:16:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:16 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:16 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:16 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:16 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:18.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:18 smithi110 ceph-mon[27723]: pgmap v36: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 501 op/s 2024-09-21T04:16:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:18 smithi148 ceph-mon[34957]: pgmap v36: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 501 op/s 2024-09-21T04:16:20.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:20 smithi110 ceph-mon[27723]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 3.4 MiB/s wr, 946 op/s 2024-09-21T04:16:20.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:20 smithi148 ceph-mon[34957]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 3.4 MiB/s wr, 946 op/s 2024-09-21T04:16:21.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.989+0000 7fd1443e7640 1 -- 172.21.15.110:0/964282508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 msgr2=0x7fd13c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:21.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.989+0000 7fd1443e7640 1 --2- 172.21.15.110:0/964282508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c105380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009880 tx=0x7fd12c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:21.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- 172.21.15.110:0/964282508 shutdown_connections 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 --2- 172.21.15.110:0/964282508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c105380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 --2- 172.21.15.110:0/964282508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 0x7fd13c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- 172.21.15.110:0/964282508 >> 172.21.15.110:0/964282508 conn(0x7fd13c0fa1c0 msgr2=0x7fd13c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- 172.21.15.110:0/964282508 shutdown_connections 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- 172.21.15.110:0/964282508 wait complete. 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 Processor -- start 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- start start 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 0x7fd13c195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c196090 con 0x7fd13c102f90 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.990+0000 7fd1443e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c196200 con 0x7fd13c100660 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:21.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50254/0 (socket says 172.21.15.110:50254) 2024-09-21T04:16:21.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 -- 172.21.15.110:0/1884543055 learned_addr learned my addr 172.21.15.110:0/1884543055 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd142183640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 0x7fd13c195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 -- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 msgr2=0x7fd13c195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 0x7fd13c195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 -- 172.21.15.110:0/1884543055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c009560 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd141982640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd12c02a9c0 tx=0x7fd12c005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12c004b20 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd12c004c80 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c19ac10 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.991+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12c003b80 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.992+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c19b130 con 0x7fd13c102f90 2024-09-21T04:16:21.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.993+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c108730 con 0x7fd13c102f90 2024-09-21T04:16:21.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.994+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7fd12c031640 con 0x7fd13c102f90 2024-09-21T04:16:21.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.995+0000 7fd1337fe640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 0x7fd12807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:21.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.995+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd12c0bf2b0 con 0x7fd13c102f90 2024-09-21T04:16:21.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.995+0000 7fd142183640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 0x7fd12807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:21.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:21.996+0000 7fd142183640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 0x7fd12807d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd138005e10 tx=0x7fd138005da0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:22.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.000+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fd12c0826d0 con 0x7fd13c102f90 2024-09-21T04:16:22.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.206+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd13c19b320 con 0x7fd12807b020 2024-09-21T04:16:22.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.208+0000 7fd1337fe640 1 -- 172.21.15.110:0/1884543055 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fd13c19b320 con 0x7fd12807b020 2024-09-21T04:16:22.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 msgr2=0x7fd12807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 0x7fd12807d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd138005e10 tx=0x7fd138005da0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 msgr2=0x7fd13c195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd12c02a9c0 tx=0x7fd12c005610 comp rx=0 tx=0).stop 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 shutdown_connections 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fd12807b020 0x7fd12807d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd13c102f90 0x7fd13c195b50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 --2- 172.21.15.110:0/1884543055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd13c100660 0x7fd13c195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 >> 172.21.15.110:0/1884543055 conn(0x7fd13c0fa1c0 msgr2=0x7fd13c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 shutdown_connections 2024-09-21T04:16:22.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.210+0000 7fd1443e7640 1 -- 172.21.15.110:0/1884543055 wait complete. 2024-09-21T04:16:22.222 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:16:22.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 -- 172.21.15.110:0/1706908461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 msgr2=0x7fe6341000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 --2- 172.21.15.110:0/1706908461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe6341000b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009880 tx=0x7fe61c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 -- 172.21.15.110:0/1706908461 shutdown_connections 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 --2- 172.21.15.110:0/1706908461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe6341000b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 --2- 172.21.15.110:0/1706908461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 0x7fe6340fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 -- 172.21.15.110:0/1706908461 >> 172.21.15.110:0/1706908461 conn(0x7fe6340fa1c0 msgr2=0x7fe6340fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 -- 172.21.15.110:0/1706908461 shutdown_connections 2024-09-21T04:16:22.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.322+0000 7fe63a314640 1 -- 172.21.15.110:0/1706908461 wait complete. 2024-09-21T04:16:22.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 Processor -- start 2024-09-21T04:16:22.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 -- start start 2024-09-21T04:16:22.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 0x7fe634195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341960d0 con 0x7fe6340ffc30 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.323+0000 7fe63a314640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634196240 con 0x7fe6340fea30 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40012/0 (socket says 172.21.15.110:40012) 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 -- 172.21.15.110:0/686496333 learned_addr learned my addr 172.21.15.110:0/686496333 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe633fff640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 0x7fe634195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 -- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 msgr2=0x7fe634195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 0x7fe634195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 -- 172.21.15.110:0/686496333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009560 con 0x7fe6340ffc30 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe62bfff640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005610 tx=0x7fe61c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c004ba0 con 0x7fe6340ffc30 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe61c004d00 con 0x7fe6340ffc30 2024-09-21T04:16:22.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63419acc0 con 0x7fe6340ffc30 2024-09-21T04:16:22.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.324+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c003bd0 con 0x7fe6340ffc30 2024-09-21T04:16:22.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.325+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63419b1e0 con 0x7fe6340ffc30 2024-09-21T04:16:22.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.326+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634107620 con 0x7fe6340ffc30 2024-09-21T04:16:22.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.327+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7fe61c031640 con 0x7fe6340ffc30 2024-09-21T04:16:22.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.328+0000 7fe631ffb640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 0x7fe62007d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.328+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe61c0bf350 con 0x7fe6340ffc30 2024-09-21T04:16:22.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.328+0000 7fe633fff640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 0x7fe62007d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.328+0000 7fe633fff640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 0x7fe62007d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6240058c0 tx=0x7fe624005850 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:22.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.332+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fe61c0826d0 con 0x7fe6340ffc30 2024-09-21T04:16:22.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.563+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6340fc3e0 con 0x7fe62007b070 2024-09-21T04:16:22.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.565+0000 7fe631ffb640 1 -- 172.21.15.110:0/686496333 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe6340fc3e0 con 0x7fe62007b070 2024-09-21T04:16:22.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.567+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 msgr2=0x7fe62007d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.567+0000 7fe63a314640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 0x7fe62007d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6240058c0 tx=0x7fe624005850 comp rx=0 tx=0).stop 2024-09-21T04:16:22.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.567+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 msgr2=0x7fe634195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.567+0000 7fe63a314640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005610 tx=0x7fe61c005640 comp rx=0 tx=0).stop 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 shutdown_connections 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fe62007b070 0x7fe62007d530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6340ffc30 0x7fe634195b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 --2- 172.21.15.110:0/686496333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6340fea30 0x7fe634195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 >> 172.21.15.110:0/686496333 conn(0x7fe6340fa1c0 msgr2=0x7fe6340fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 shutdown_connections 2024-09-21T04:16:22.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.568+0000 7fe63a314640 1 -- 172.21.15.110:0/686496333 wait complete. 2024-09-21T04:16:22.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 -- 172.21.15.110:0/3032286024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0fea30 msgr2=0x7f5b0c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/3032286024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c0fee30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009880 tx=0x7f5b0002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 -- 172.21.15.110:0/3032286024 shutdown_connections 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/3032286024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/3032286024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c0fee30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 -- 172.21.15.110:0/3032286024 >> 172.21.15.110:0/3032286024 conn(0x7f5b0c0fa1c0 msgr2=0x7f5b0c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 -- 172.21.15.110:0/3032286024 shutdown_connections 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.672+0000 7f5b12c9e640 1 -- 172.21.15.110:0/3032286024 wait complete. 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 Processor -- start 2024-09-21T04:16:22.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 -- start start 2024-09-21T04:16:22.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c196210 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b12c9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c196380 con 0x7f5b0c0fea30 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40042/0 (socket says 172.21.15.110:40042) 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.673+0000 7f5b0bfff640 1 -- 172.21.15.110:0/4000287448 learned_addr learned my addr 172.21.15.110:0/4000287448 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b10a3a640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b0bfff640 1 -- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0fea30 msgr2=0x7f5b0c195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b0bfff640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b0bfff640 1 -- 172.21.15.110:0/4000287448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00009560 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b0bfff640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5af800b450 tx=0x7f5af800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af8011860 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c19ae20 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5af8011e80 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af800f570 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.674+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c19b3c0 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.675+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0feeb0 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.676+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f5af80103e0 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.677+0000 7f5b09ffb640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 0x7f5afc07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:22.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.677+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5af809dd20 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.677+0000 7f5b10a3a640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 0x7f5afc07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:22.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.678+0000 7f5b10a3a640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 0x7f5afc07d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009230 tx=0x7f5b00005b50 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:22.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.682+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f5af8061140 con 0x7f5b0c0ffc30 2024-09-21T04:16:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.897+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b0c0682f0 con 0x7f5afc07b020 2024-09-21T04:16:22.900 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:22 smithi110 ceph-mon[27723]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 839 op/s 2024-09-21T04:16:22.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.913+0000 7f5b09ffb640 1 -- 172.21.15.110:0/4000287448 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5b0c0682f0 con 0x7f5afc07b020 2024-09-21T04:16:22.916 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:16:22.916 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (7m) 7s ago 9m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (45s) 7s ago 9m 7507k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (36s) 21s ago 8m 7172k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (9m) 7s ago 9m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (8m) 21s ago 8m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (7m) 7s ago 9m 87.5M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (3m) 7s ago 3m 229M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (3m) 7s ago 3m 13.8M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (3m) 21s ago 3m 15.5M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (3m) 21s ago 3m 17.0M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (28s) 7s ago 10m 143M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (94s) 21s ago 8m 509M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0d777a7727d1 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (10m) 7s ago 11m 53.6M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (8m) 21s ago 8m 41.9M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (9m) 7s ago 9m 16.6M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:16:22.917 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (8m) 21s ago 8m 15.0M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 7s ago 7m 219M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (6m) 7s ago 6m 226M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 7s ago 6m 188M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (5m) 21s ago 5m 264M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (5m) 21s ago 5m 210M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (5m) 21s ago 5m 210M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:16:22.918 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (40s) 7s ago 9m 44.8M - 2.43.0 a07b618ecd1d 6c1297c97077 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 msgr2=0x7f5afc07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 0x7f5afc07d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009230 tx=0x7f5b00005b50 comp rx=0 tx=0).stop 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 msgr2=0x7f5b0c195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5af800b450 tx=0x7f5af800b910 comp rx=0 tx=0).stop 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 shutdown_connections 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f5afc07b020 0x7f5afc07d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0c0ffc30 0x7f5b0c195c40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 --2- 172.21.15.110:0/4000287448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0c0fea30 0x7f5b0c195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:22.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 >> 172.21.15.110:0/4000287448 conn(0x7f5b0c0fa1c0 msgr2=0x7f5b0c067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:22.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 shutdown_connections 2024-09-21T04:16:22.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:22.918+0000 7f5b12c9e640 1 -- 172.21.15.110:0/4000287448 wait complete. 2024-09-21T04:16:22.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:22 smithi148 ceph-mon[34957]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 839 op/s 2024-09-21T04:16:23.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.059+0000 7f6f30cbd640 1 -- 172.21.15.110:0/1679922661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 msgr2=0x7f6f2c071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.059+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/1679922661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c071640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f2002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.060+0000 7f6f30cbd640 1 -- 172.21.15.110:0/1679922661 shutdown_connections 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.060+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/1679922661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c071640 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.060+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/1679922661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.060+0000 7f6f30cbd640 1 -- 172.21.15.110:0/1679922661 >> 172.21.15.110:0/1679922661 conn(0x7f6f2c06cc80 msgr2=0x7f6f2c06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.061+0000 7f6f30cbd640 1 -- 172.21.15.110:0/1679922661 shutdown_connections 2024-09-21T04:16:23.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.061+0000 7f6f30cbd640 1 -- 172.21.15.110:0/1679922661 wait complete. 2024-09-21T04:16:23.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.062+0000 7f6f30cbd640 1 Processor -- start 2024-09-21T04:16:23.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.062+0000 7f6f30cbd640 1 -- start start 2024-09-21T04:16:23.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.062+0000 7f6f30cbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f30cbd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c1174b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f30cbd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c117a80 con 0x7f6f2c1080e0 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f2a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f2a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:33380/0 (socket says 172.21.15.110:33380) 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f2a59c640 1 -- 172.21.15.110:0/2182770962 learned_addr learned my addr 172.21.15.110:0/2182770962 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c117bf0 con 0x7f6f2c107710 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.063+0000 7f6f29d9b640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c1174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.064+0000 7f6f2a59c640 1 -- 172.21.15.110:0/2182770962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 msgr2=0x7f6f2c1174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.064+0000 7f6f2a59c640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c1174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.064+0000 7f6f2a59c640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20009560 con 0x7f6f2c107710 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.064+0000 7f6f2a59c640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b3f0 tx=0x7f6f1c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.065+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f1c011800 con 0x7f6f2c107710 2024-09-21T04:16:23.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.065+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f1c011e20 con 0x7f6f2c107710 2024-09-21T04:16:23.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.065+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f1c00f2e0 con 0x7f6f2c107710 2024-09-21T04:16:23.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.065+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c11a6b0 con 0x7f6f2c107710 2024-09-21T04:16:23.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.066+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c11ac80 con 0x7f6f2c107710 2024-09-21T04:16:23.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.068+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f6f1c010380 con 0x7f6f2c107710 2024-09-21T04:16:23.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.069+0000 7f6f1b7fe640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 0x7f6f1407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.069+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6f1c09dad0 con 0x7f6f2c107710 2024-09-21T04:16:23.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.070+0000 7f6f29d9b640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 0x7f6f1407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.070+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c10a330 con 0x7f6f2c107710 2024-09-21T04:16:23.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.070+0000 7f6f29d9b640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 0x7f6f1407d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c118490 tx=0x7f6f20005b50 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.079+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f6f1c060ed0 con 0x7f6f2c107710 2024-09-21T04:16:23.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.403+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6f2c10a4c0 con 0x7f6f2c107710 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.404+0000 7f6f1b7fe640 1 -- 172.21.15.110:0/2182770962 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f6f1c0152f0 con 0x7f6f2c107710 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:16:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 12, 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:16:23.408 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:16:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 msgr2=0x7f6f1407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 0x7f6f1407d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c118490 tx=0x7f6f20005b50 comp rx=0 tx=0).stop 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 msgr2=0x7f6f2c116f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b3f0 tx=0x7f6f1c00b8b0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 shutdown_connections 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f6f1407b070 0x7f6f1407d530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f2c1080e0 0x7f6f2c1174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 --2- 172.21.15.110:0/2182770962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f2c107710 0x7f6f2c116f70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 >> 172.21.15.110:0/2182770962 conn(0x7f6f2c06cc80 msgr2=0x7f6f2c10cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 shutdown_connections 2024-09-21T04:16:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.407+0000 7f6f30cbd640 1 -- 172.21.15.110:0/2182770962 wait complete. 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 -- 172.21.15.110:0/3783673042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 msgr2=0x7f061c0ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 --2- 172.21.15.110:0/3783673042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c0ffc40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0610009880 tx=0x7f061002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 -- 172.21.15.110:0/3783673042 shutdown_connections 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 --2- 172.21.15.110:0/3783673042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c0ffc40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 --2- 172.21.15.110:0/3783673042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 0x7f061c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 -- 172.21.15.110:0/3783673042 >> 172.21.15.110:0/3783673042 conn(0x7f061c0fb330 msgr2=0x7f061c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 -- 172.21.15.110:0/3783673042 shutdown_connections 2024-09-21T04:16:23.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.489+0000 7f0621841640 1 -- 172.21.15.110:0/3783673042 wait complete. 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 Processor -- start 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 -- start start 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 0x7f061c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c196170 con 0x7f061c0ff7c0 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f0621841640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1962e0 con 0x7f061c101280 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40078/0 (socket says 172.21.15.110:40078) 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 -- 172.21.15.110:0/3536887661 learned_addr learned my addr 172.21.15.110:0/3536887661 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 -- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 msgr2=0x7f061c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061a7fc640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 0x7f061c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 0x7f061c195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 -- 172.21.15.110:0/3536887661 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610009560 con 0x7f061c0ff7c0 2024-09-21T04:16:23.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.490+0000 7f061affd640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c8f0 tx=0x7f060c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.491+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c0041f0 con 0x7f061c0ff7c0 2024-09-21T04:16:23.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.491+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c19adc0 con 0x7f061c0ff7c0 2024-09-21T04:16:23.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.491+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060c007e10 con 0x7f061c0ff7c0 2024-09-21T04:16:23.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.491+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c00f450 con 0x7f061c0ff7c0 2024-09-21T04:16:23.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.491+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c19b2e0 con 0x7f061c0ff7c0 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.492+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c0ffc40 con 0x7f061c0ff7c0 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.492+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f060c007950 con 0x7f061c0ff7c0 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.493+0000 7f062083f640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 0x7f060407d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.493+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f060c013070 con 0x7f061c0ff7c0 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.493+0000 7f061a7fc640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 0x7f060407d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.494+0000 7f061a7fc640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 0x7f060407d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0610031540 tx=0x7f0610000bc0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.498+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f060c061440 con 0x7f061c0ff7c0 2024-09-21T04:16:23.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:23 smithi110 ceph-mon[27723]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:23.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.767+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f061c18f810 con 0x7f061c0ff7c0 2024-09-21T04:16:23.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.767+0000 7f062083f640 1 -- 172.21.15.110:0/3536887661 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f060c064560 con 0x7f061c0ff7c0 2024-09-21T04:16:23.770 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:16:23.770 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:16:23.771 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:16:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 msgr2=0x7f060407d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 0x7f060407d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0610031540 tx=0x7f0610000bc0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 msgr2=0x7f061c1956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c8f0 tx=0x7f060c00cdb0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 shutdown_connections 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f060407b020 0x7f060407d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f061c101280 0x7f061c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 --2- 172.21.15.110:0/3536887661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f061c0ff7c0 0x7f061c1956f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.769+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 >> 172.21.15.110:0/3536887661 conn(0x7f061c0fb330 msgr2=0x7f061c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.770+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 shutdown_connections 2024-09-21T04:16:23.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.770+0000 7f0621841640 1 -- 172.21.15.110:0/3536887661 wait complete. 2024-09-21T04:16:23.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.901+0000 7f71c2e45640 1 -- 172.21.15.110:0/1265156839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc101d60 msgr2=0x7f71bc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.901+0000 7f71c2e45640 1 --2- 172.21.15.110:0/1265156839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc101d60 0x7f71bc102160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71b00097e0 tx=0x7f71b002a410 comp rx=0 tx=0).stop 2024-09-21T04:16:23.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.903+0000 7f71c2e45640 1 -- 172.21.15.110:0/1265156839 shutdown_connections 2024-09-21T04:16:23.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.903+0000 7f71c2e45640 1 --2- 172.21.15.110:0/1265156839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc102f60 0x7f71bc1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.903+0000 7f71c2e45640 1 --2- 172.21.15.110:0/1265156839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc101d60 0x7f71bc102160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.903+0000 7f71c2e45640 1 -- 172.21.15.110:0/1265156839 >> 172.21.15.110:0/1265156839 conn(0x7f71bc0fd550 msgr2=0x7f71bc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:23.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.904+0000 7f71c2e45640 1 -- 172.21.15.110:0/1265156839 shutdown_connections 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.906+0000 7f71c2e45640 1 -- 172.21.15.110:0/1265156839 wait complete. 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 Processor -- start 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 -- start start 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 0x7f71bc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc06db30 con 0x7f71bc102f60 2024-09-21T04:16:23.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c2e45640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc06dca0 con 0x7f71bc101d60 2024-09-21T04:16:23.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c0be1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c0be1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:33404/0 (socket says 172.21.15.110:33404) 2024-09-21T04:16:23.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.907+0000 7f71c0be1640 1 -- 172.21.15.110:0/3635751691 learned_addr learned my addr 172.21.15.110:0/3635751691 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:23.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.908+0000 7f71bbfff640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 0x7f71bc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.908+0000 7f71bbfff640 1 -- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 msgr2=0x7f71bc06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:23.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.908+0000 7f71bbfff640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:23.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.908+0000 7f71bbfff640 1 -- 172.21.15.110:0/3635751691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0009490 con 0x7f71bc102f60 2024-09-21T04:16:23.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.908+0000 7f71c0be1640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:16:23.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.909+0000 7f71bbfff640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 0x7f71bc06d5f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e710 tx=0x7f71a800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.912+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a800ca80 con 0x7f71bc102f60 2024-09-21T04:16:23.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.912+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71a800cbe0 con 0x7f71bc102f60 2024-09-21T04:16:23.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.912+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a8018700 con 0x7f71bc102f60 2024-09-21T04:16:23.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.912+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc1b23b0 con 0x7f71bc102f60 2024-09-21T04:16:23.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.912+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc1b2880 con 0x7f71bc102f60 2024-09-21T04:16:23.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.915+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f71a800cd50 con 0x7f71bc102f60 2024-09-21T04:16:23.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.916+0000 7f71b9ffb640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 0x7f71ac07d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:23.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.916+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f71a8014070 con 0x7f71bc102f60 2024-09-21T04:16:23.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.916+0000 7f71c0be1640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 0x7f71ac07d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:23.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.917+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc102160 con 0x7f71bc102f60 2024-09-21T04:16:23.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.917+0000 7f71c0be1640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 0x7f71ac07d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71b002aa00 tx=0x7f71b0030870 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:23.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:23.926+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f71a80615d0 con 0x7f71bc102f60 2024-09-21T04:16:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:23 smithi148 ceph-mon[34957]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:23 smithi148 ceph-mon[34957]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:23 smithi148 ceph-mon[34957]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:23 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2182770962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:16:24.120 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:23 smithi110 ceph-mon[27723]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:24.120 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:23 smithi110 ceph-mon[27723]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:24.120 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:23 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/2182770962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:16:24.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.183+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 --> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71bc0608f0 con 0x7f71ac07b0b0 2024-09-21T04:16:24.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.185+0000 7f71b9ffb640 1 -- 172.21.15.110:0/3635751691 <== mgr.24415 v2:172.21.15.148:6828/2999382373 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f71bc0608f0 con 0x7f71ac07b0b0 2024-09-21T04:16:24.187 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "2/23 daemons upgraded", 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mgr daemons", 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:16:24.188 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:16:24.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 msgr2=0x7f71ac07d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 0x7f71ac07d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71b002aa00 tx=0x7f71b0030870 comp rx=0 tx=0).stop 2024-09-21T04:16:24.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 msgr2=0x7f71bc06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 0x7f71bc06d5f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e710 tx=0x7f71a800ebd0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 shutdown_connections 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7f71ac07b0b0 0x7f71ac07d570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71bc102f60 0x7f71bc06d5f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 --2- 172.21.15.110:0/3635751691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71bc101d60 0x7f71bc06d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 >> 172.21.15.110:0/3635751691 conn(0x7f71bc0fd550 msgr2=0x7f71bc1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 shutdown_connections 2024-09-21T04:16:24.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.187+0000 7f71c2e45640 1 -- 172.21.15.110:0/3635751691 wait complete. 2024-09-21T04:16:24.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 -- 172.21.15.110:0/820318265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 msgr2=0x7fbde80fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 --2- 172.21.15.110:0/820318265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde80fee30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009880 tx=0x7fbdd802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 -- 172.21.15.110:0/820318265 shutdown_connections 2024-09-21T04:16:24.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 --2- 172.21.15.110:0/820318265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 0x7fbde81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 --2- 172.21.15.110:0/820318265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde80fee30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 -- 172.21.15.110:0/820318265 >> 172.21.15.110:0/820318265 conn(0x7fbde80fa1c0 msgr2=0x7fbde80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:24.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 -- 172.21.15.110:0/820318265 shutdown_connections 2024-09-21T04:16:24.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.279+0000 7fbdee911640 1 -- 172.21.15.110:0/820318265 wait complete. 2024-09-21T04:16:24.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.280+0000 7fbdee911640 1 Processor -- start 2024-09-21T04:16:24.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.280+0000 7fbdee911640 1 -- start start 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbdee911640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbdee911640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 0x7fbde819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbdee911640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde819aa00 con 0x7fbde80fea30 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbdee911640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde819ab70 con 0x7fbde80ffc30 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40112/0 (socket says 172.21.15.110:40112) 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 -- 172.21.15.110:0/798310788 learned_addr learned my addr 172.21.15.110:0/798310788 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde77fe640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 0x7fbde819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 -- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 msgr2=0x7fbde819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 0x7fbde819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 -- 172.21.15.110:0/798310788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8009560 con 0x7fbde80fea30 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde7fff640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802a9c0 tx=0x7fbdd8005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.281+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8004b20 con 0x7fbde80fea30 2024-09-21T04:16:24.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.282+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdd8004c80 con 0x7fbde80fea30 2024-09-21T04:16:24.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.282+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde819adf0 con 0x7fbde80fea30 2024-09-21T04:16:24.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.282+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde819b330 con 0x7fbde80fea30 2024-09-21T04:16:24.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.282+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8003b10 con 0x7fbde80fea30 2024-09-21T04:16:24.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.283+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde80feeb0 con 0x7fbde80fea30 2024-09-21T04:16:24.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.284+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 103285+0+0 (secure 0 0 0) 0x7fbdd8003d20 con 0x7fbde80fea30 2024-09-21T04:16:24.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.284+0000 7fbde57fa640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 0x7fbdd007d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:24.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.284+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbdd80bf190 con 0x7fbde80fea30 2024-09-21T04:16:24.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.285+0000 7fbde77fe640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 0x7fbdd007d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:24.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.285+0000 7fbde77fe640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 0x7fbdd007d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbde8068500 tx=0x7fbddc009110 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.291+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fbdd8082610 con 0x7fbde80fea30 2024-09-21T04:16:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.633+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbde81078f0 con 0x7fbde80fea30 2024-09-21T04:16:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.633+0000 7fbde57fa640 1 -- 172.21.15.110:0/798310788 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbdd8085730 con 0x7fbde80fea30 2024-09-21T04:16:24.636 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:16:24.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 msgr2=0x7fbdd007d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 0x7fbdd007d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbde8068500 tx=0x7fbddc009110 comp rx=0 tx=0).stop 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 msgr2=0x7fbde8068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802a9c0 tx=0x7fbdd8005610 comp rx=0 tx=0).stop 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 shutdown_connections 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:6828/2999382373,v1:172.21.15.148:6829/2999382373] conn(0x7fbdd007b020 0x7fbdd007d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde80ffc30 0x7fbde819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 --2- 172.21.15.110:0/798310788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde80fea30 0x7fbde8068780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 >> 172.21.15.110:0/798310788 conn(0x7fbde80fa1c0 msgr2=0x7fbde8102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 shutdown_connections 2024-09-21T04:16:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:24.636+0000 7fbdee911640 1 -- 172.21.15.110:0/798310788 wait complete. 2024-09-21T04:16:24.831 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:24 smithi110 ceph-mon[27723]: pgmap v39: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 3.0 MiB/s wr, 1.25k op/s 2024-09-21T04:16:24.831 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/3536887661' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:16:24.831 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:24 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/798310788' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:16:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:24 smithi148 ceph-mon[34957]: pgmap v39: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 3.0 MiB/s wr, 1.25k op/s 2024-09-21T04:16:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:24 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/3536887661' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:16:24.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:24 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/798310788' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:16:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:25 smithi110 ceph-mon[27723]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:25.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:25 smithi148 ceph-mon[34957]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:26.913 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:26 smithi110 ceph-mon[27723]: pgmap v40: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 857 op/s 2024-09-21T04:16:26.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:26 smithi148 ceph-mon[34957]: pgmap v40: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 857 op/s 2024-09-21T04:16:27.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:27 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:27.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:27 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:28 smithi148 ceph-mon[34957]: pgmap v41: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 857 op/s 2024-09-21T04:16:28.953 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:28 smithi110 ceph-mon[27723]: pgmap v41: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 857 op/s 2024-09-21T04:16:30.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: pgmap v42: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.5 MiB/s wr, 1.21k op/s 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: osdmap e48: 6 total, 6 up, 6 in 2024-09-21T04:16:30.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd='[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]': finished 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: mgrmap e28: smithi110.axmqzb(active, starting, since 0.0118617s) 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:16:30.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:16:30.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:16:30.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:16:30.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:30 smithi148 ceph-mon[34957]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:16:31.054 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: pgmap v42: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.5 MiB/s wr, 1.21k op/s 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 172.21.15.148:0/4214916765' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:31.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: osdmap e48: 6 total, 6 up, 6 in 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.24415 ' entity='mgr.smithi148.zjrahi' cmd='[{"prefix": "mgr fail", "who": "smithi148.zjrahi"}]': finished 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: mgrmap e28: smithi110.axmqzb(active, starting, since 0.0118617s) 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:16:31.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:16:31.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:30 smithi110 ceph-mon[27723]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:16:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:31 smithi110 ceph-mon[27723]: mgrmap e29: smithi110.axmqzb(active, since 1.01062s) 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"}]: dispatch 2024-09-21T04:16:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:31 smithi148 ceph-mon[34957]: mgrmap e29: smithi110.axmqzb(active, since 1.01062s) 2024-09-21T04:16:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:32 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:16:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:32 smithi110 ceph-mon[27723]: pgmap v3: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:32.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:32 smithi110 ceph-mon[27723]: mgrmap e30: smithi110.axmqzb(active, since 2s) 2024-09-21T04:16:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:32 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:16:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:32 smithi148 ceph-mon[34957]: pgmap v3: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:32.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:32 smithi148 ceph-mon[34957]: mgrmap e30: smithi110.axmqzb(active, since 2s) 2024-09-21T04:16:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: [21/Sep/2024:04:16:32] ENGINE Bus STARTING 2024-09-21T04:16:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: pgmap v4: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: [21/Sep/2024:04:16:32] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:16:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: [21/Sep/2024:04:16:32] ENGINE Error in HTTPServer.serve 2024-09-21T04:16:33.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:16:33.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-09-21T04:16:33.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:16:33.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-09-21T04:16:33.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:16:33.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: [21/Sep/2024:04:16:32] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:16:33.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:33 smithi148 ceph-mon[34957]: [21/Sep/2024:04:16:32] ENGINE Bus STARTED 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: [21/Sep/2024:04:16:32] ENGINE Bus STARTING 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: pgmap v4: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: [21/Sep/2024:04:16:32] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: [21/Sep/2024:04:16:32] ENGINE Error in HTTPServer.serve 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:16:34.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: [21/Sep/2024:04:16:32] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:16:34.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:33 smithi110 ceph-mon[27723]: [21/Sep/2024:04:16:32] ENGINE Bus STARTED 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: pgmap v5: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: mgrmap e31: smithi110.axmqzb(active, since 4s) 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:35 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: pgmap v5: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: mgrmap e31: smithi110.axmqzb(active, since 4s) 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:35 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:36.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:36 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:16:36.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:36 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:16:36.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:36 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:16:36.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:36 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:16:36.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:36 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:16:36.673 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:36 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:16:36.674 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:36 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:16:36.674 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:36 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:16:36.674 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:36 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:16:36.674 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:36 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/693591209' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:16:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:37 smithi110 ceph-mon[27723]: pgmap v6: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 176 op/s 2024-09-21T04:16:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:37 smithi110 ceph-mon[27723]: mgrmap e32: smithi110.axmqzb(active, since 6s), standbys: smithi148.zjrahi 2024-09-21T04:16:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:37 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:37 smithi148 ceph-mon[34957]: pgmap v6: 65 pgs: 65 active+clean; 263 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 176 op/s 2024-09-21T04:16:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:37 smithi148 ceph-mon[34957]: mgrmap e32: smithi110.axmqzb(active, since 6s), standbys: smithi148.zjrahi 2024-09-21T04:16:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:37 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"}]: dispatch 2024-09-21T04:16:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:39 smithi110 ceph-mon[27723]: pgmap v7: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 382 op/s 2024-09-21T04:16:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:39 smithi148 ceph-mon[34957]: pgmap v7: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 382 op/s 2024-09-21T04:16:41.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:41 smithi110 ceph-mon[27723]: pgmap v8: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-09-21T04:16:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:41 smithi148 ceph-mon[34957]: pgmap v8: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-09-21T04:16:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:43 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:43 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:43 smithi110 ceph-mon[27723]: pgmap v9: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.7 MiB/s wr, 744 op/s 2024-09-21T04:16:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:43 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:43 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:43 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:43 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:43 smithi148 ceph-mon[34957]: pgmap v9: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.7 MiB/s wr, 744 op/s 2024-09-21T04:16:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:43 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:43 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:45.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:45 smithi110 ceph-mon[27723]: pgmap v10: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 682 op/s 2024-09-21T04:16:45.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:45 smithi148 ceph-mon[34957]: pgmap v10: 65 pgs: 65 active+clean; 252 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 682 op/s 2024-09-21T04:16:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:46 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:46.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:46 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:16:47.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:47 smithi110 ceph-mon[27723]: pgmap v11: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.8 MiB/s wr, 887 op/s 2024-09-21T04:16:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:47 smithi148 ceph-mon[34957]: pgmap v11: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.8 MiB/s wr, 887 op/s 2024-09-21T04:16:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:49 smithi110 ceph-mon[27723]: pgmap v12: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.3 MiB/s wr, 1.15k op/s 2024-09-21T04:16:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:49 smithi148 ceph-mon[34957]: pgmap v12: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.3 MiB/s wr, 1.15k op/s 2024-09-21T04:16:51.362 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-21T04:16:51.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-21T04:16:51.653 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:51 smithi148 ceph-mon[34957]: pgmap v13: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 1.00k op/s 2024-09-21T04:16:51.654 DEBUG:teuthology.parallel:result is None 2024-09-21T04:16:51.666 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:51 smithi110 ceph-mon[27723]: pgmap v13: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 1.00k op/s 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:52 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:53.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:52 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: pgmap v14: 65 pgs: 65 active+clean; 224 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.3 MiB/s wr, 1.28k op/s 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:16:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:53 smithi110 ceph-mon[27723]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: pgmap v14: 65 pgs: 65 active+clean; 224 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.3 MiB/s wr, 1.28k op/s 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:16:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:53 smithi148 ceph-mon[34957]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 -- 172.21.15.110:0/2568781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500ffc30 msgr2=0x7f2c501000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 --2- 172.21.15.110:0/2568781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500ffc30 0x7f2c501000b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009880 tx=0x7f2c4002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 -- 172.21.15.110:0/2568781029 shutdown_connections 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 --2- 172.21.15.110:0/2568781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500ffc30 0x7f2c501000b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 --2- 172.21.15.110:0/2568781029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500fea30 0x7f2c500fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 -- 172.21.15.110:0/2568781029 >> 172.21.15.110:0/2568781029 conn(0x7f2c500fa1c0 msgr2=0x7f2c500fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 -- 172.21.15.110:0/2568781029 shutdown_connections 2024-09-21T04:16:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.733+0000 7f2c57439640 1 -- 172.21.15.110:0/2568781029 wait complete. 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 Processor -- start 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 -- start start 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500ffc30 0x7f2c50101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50101550 con 0x7f2c500fea30 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c57439640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c501016c0 con 0x7f2c500ffc30 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45856/0 (socket says 172.21.15.110:45856) 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 -- 172.21.15.110:0/1047026022 learned_addr learned my addr 172.21.15.110:0/1047026022 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c549d4640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500ffc30 0x7f2c50101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 -- 172.21.15.110:0/1047026022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500ffc30 msgr2=0x7f2c50101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:54.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500ffc30 0x7f2c50101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.734+0000 7f2c551d5640 1 -- 172.21.15.110:0/1047026022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009560 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c551d5640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e940 tx=0x7f2c4c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c4c0096e0 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c4c0044f0 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c501019a0 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c4c010430 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.735+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5019d6b0 con 0x7f2c500fea30 2024-09-21T04:16:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.736+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c500fee30 con 0x7f2c500fea30 2024-09-21T04:16:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.737+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f2c4c00cc90 con 0x7f2c500fea30 2024-09-21T04:16:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.738+0000 7f2c467fc640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 0x7f2c3c07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:54.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.738+0000 7f2c549d4640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 0x7f2c3c07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:54.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.738+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2c4c014070 con 0x7f2c500fea30 2024-09-21T04:16:54.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.739+0000 7f2c549d4640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 0x7f2c3c07d4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c40000c00 tx=0x7f2c400314e0 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:54.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:54.743+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f2c4c061760 con 0x7f2c500fea30 2024-09-21T04:16:54.862 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-21T04:16:54.863 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T04:16:54.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:54 smithi148 ceph-mon[34957]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:16:55.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:54 smithi110 ceph-mon[27723]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-09-21T04:16:55.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.028+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 --> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c500fc3b0 con 0x7f2c3c07b020 2024-09-21T04:16:55.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.033+0000 7f2c467fc640 1 -- 172.21.15.110:0/1047026022 <== mgr.14790 v2:172.21.15.110:6800/1516382105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2c500fc3b0 con 0x7f2c3c07b020 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 msgr2=0x7f2c3c07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 0x7f2c3c07d4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c40000c00 tx=0x7f2c400314e0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 msgr2=0x7f2c501029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e940 tx=0x7f2c4c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 shutdown_connections 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f2c3c07b020 0x7f2c3c07d4e0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c40000c00 tx=0x7f2c400314e0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c500ffc30 0x7f2c50101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 --2- 172.21.15.110:0/1047026022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c500fea30 0x7f2c501029c0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e940 tx=0x7f2c4c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.037+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 >> 172.21.15.110:0/1047026022 conn(0x7f2c500fa1c0 msgr2=0x7f2c500fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.039+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 shutdown_connections 2024-09-21T04:16:55.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.039+0000 7f2c57439640 1 -- 172.21.15.110:0/1047026022 wait complete. 2024-09-21T04:16:55.057 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:16:55.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2676827802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0ffc30 msgr2=0x7fb9ec1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2676827802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec1000b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009880 tx=0x7fb9e002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2676827802 shutdown_connections 2024-09-21T04:16:55.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2676827802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec1000b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2676827802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.145+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2676827802 >> 172.21.15.110:0/2676827802 conn(0x7fb9ec0fa1c0 msgr2=0x7fb9ec0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2676827802 shutdown_connections 2024-09-21T04:16:55.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2676827802 wait complete. 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 Processor -- start 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 -- start start 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec19a930 con 0x7fb9ec0fea30 2024-09-21T04:16:55.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.146+0000 7fb9f3f4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec19aa70 con 0x7fb9ec0ffc30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45866/0 (socket says 172.21.15.110:45866) 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 -- 172.21.15.110:0/2603519205 learned_addr learned my addr 172.21.15.110:0/2603519205 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f14e5640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 -- 172.21.15.110:0/2603519205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0ffc30 msgr2=0x7fb9ec19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 -- 172.21.15.110:0/2603519205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0009560 con 0x7fb9ec0fea30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f1ce6640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e940 tx=0x7fb9e800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e80096e0 con 0x7fb9ec0fea30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9e80044f0 con 0x7fb9ec0fea30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec19ad20 con 0x7fb9ec0fea30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e8010430 con 0x7fb9ec0fea30 2024-09-21T04:16:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.147+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec19b2c0 con 0x7fb9ec0fea30 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.149+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec0fee30 con 0x7fb9ec0fea30 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.149+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7fb9e800cc90 con 0x7fb9ec0fea30 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.150+0000 7fb9deffd640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 0x7fb9d807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.150+0000 7fb9f14e5640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 0x7fb9d807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.150+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb9e8014070 con 0x7fb9ec0fea30 2024-09-21T04:16:55.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.151+0000 7fb9f14e5640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 0x7fb9d807d4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000c00 tx=0x7fb9e00314e0 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.154+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7fb9e8061740 con 0x7fb9ec0fea30 2024-09-21T04:16:55.256 DEBUG:teuthology.parallel:result is None 2024-09-21T04:16:55.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:16:55.285 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:16:55.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-21T04:16:55.314 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-21T04:16:55.314 DEBUG:teuthology.parallel:result is None 2024-09-21T04:16:55.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.372+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 --> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9ec104580 con 0x7fb9d807b020 2024-09-21T04:16:55.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.374+0000 7fb9deffd640 1 -- 172.21.15.110:0/2603519205 <== mgr.14790 v2:172.21.15.110:6800/1516382105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb9ec104580 con 0x7fb9d807b020 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 msgr2=0x7fb9d807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 0x7fb9d807d4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000c00 tx=0x7fb9e00314e0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 msgr2=0x7fb9ec068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e940 tx=0x7fb9e800ee00 comp rx=0 tx=0).stop 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 shutdown_connections 2024-09-21T04:16:55.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7fb9d807b020 0x7fb9d807d4e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ec0ffc30 0x7fb9ec19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 --2- 172.21.15.110:0/2603519205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9ec0fea30 0x7fb9ec068740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 >> 172.21.15.110:0/2603519205 conn(0x7fb9ec0fa1c0 msgr2=0x7fb9ec102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 shutdown_connections 2024-09-21T04:16:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.376+0000 7fb9f3f4a640 1 -- 172.21.15.110:0/2603519205 wait complete. 2024-09-21T04:16:55.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.462+0000 7f664e257640 1 -- 172.21.15.110:0/2294191033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6648101280 msgr2=0x7f6648101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.462+0000 7f664e257640 1 --2- 172.21.15.110:0/2294191033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6648101280 0x7f6648101680 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6638009880 tx=0x7f663802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.462+0000 7f664e257640 1 -- 172.21.15.110:0/2294191033 shutdown_connections 2024-09-21T04:16:55.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.462+0000 7f664e257640 1 --2- 172.21.15.110:0/2294191033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66480ff7c0 0x7f66480ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 --2- 172.21.15.110:0/2294191033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6648101280 0x7f6648101680 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 -- 172.21.15.110:0/2294191033 >> 172.21.15.110:0/2294191033 conn(0x7f66480fb330 msgr2=0x7f66480fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 -- 172.21.15.110:0/2294191033 shutdown_connections 2024-09-21T04:16:55.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 -- 172.21.15.110:0/2294191033 wait complete. 2024-09-21T04:16:55.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 Processor -- start 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.463+0000 7f664e257640 1 -- start start 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f664e257640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f664e257640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6648101280 0x7f6648195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f664e257640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648196220 con 0x7f66480ff7c0 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f664e257640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648196390 con 0x7f6648101280 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45888/0 (socket says 172.21.15.110:45888) 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 -- 172.21.15.110:0/762920549 learned_addr learned my addr 172.21.15.110:0/762920549 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f6646ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6648101280 0x7f6648195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 -- 172.21.15.110:0/762920549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6648101280 msgr2=0x7f6648195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6648101280 0x7f6648195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 -- 172.21.15.110:0/762920549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638009560 con 0x7f66480ff7c0 2024-09-21T04:16:55.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f66477fe640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6638004650 tx=0x7f66380055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.464+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6638004aa0 con 0x7f66480ff7c0 2024-09-21T04:16:55.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.465+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6638005d50 con 0x7f66480ff7c0 2024-09-21T04:16:55.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.465+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66480726e0 con 0x7f66480ff7c0 2024-09-21T04:16:55.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.465+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66380039e0 con 0x7f66480ff7c0 2024-09-21T04:16:55.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.465+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648072c50 con 0x7f66480ff7c0 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.466+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66480ffc40 con 0x7f66480ff7c0 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.466+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f6638003c50 con 0x7f66480ff7c0 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.467+0000 7f6644ff9640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 0x7f663007d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.467+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f66380bf080 con 0x7f66480ff7c0 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.467+0000 7f6646ffd640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 0x7f663007d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.467+0000 7f6646ffd640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 0x7f663007d4e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6648196c30 tx=0x7f663c012040 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.471+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f6638082480 con 0x7f66480ff7c0 2024-09-21T04:16:55.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.664+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 --> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66480608f0 con 0x7f663007b020 2024-09-21T04:16:55.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.673+0000 7f6644ff9640 1 -- 172.21.15.110:0/762920549 <== mgr.14790 v2:172.21.15.110:6800/1516382105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f66480608f0 con 0x7f663007b020 2024-09-21T04:16:55.676 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:16:55.676 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (8m) 13s ago 10m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:16:55.676 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (78s) 13s ago 10m 7545k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:16:55.676 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (68s) 21s ago 9m 7214k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (10m) 13s ago 10m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (9m) 21s ago 9m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (8m) 13s ago 9m 87.5M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (4m) 13s ago 4m 205M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (4m) 13s ago 4m 14.2M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (4m) 21s ago 4m 15.9M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (4m) 21s ago 4m 17.5M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (61s) 13s ago 11m 490M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (2m) 21s ago 9m 122M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0d777a7727d1 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (11m) 13s ago 11m 57.2M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (9m) 21s ago 9m 45.4M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (10m) 13s ago 10m 16.8M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (9m) 21s ago 9m 14.9M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 13s ago 7m 261M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:16:55.677 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (7m) 13s ago 7m 255M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 13s ago 6m 206M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (6m) 21s ago 6m 303M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (6m) 21s ago 6m 241M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (5m) 21s ago 5m 232M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (72s) 13s ago 9m 47.2M - 2.43.0 a07b618ecd1d 6c1297c97077 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 msgr2=0x7f663007d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 0x7f663007d4e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6648196c30 tx=0x7f663c012040 comp rx=0 tx=0).stop 2024-09-21T04:16:55.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 msgr2=0x7f6648195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6638004650 tx=0x7f66380055a0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 shutdown_connections 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f663007b020 0x7f663007d4e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6648101280 0x7f6648195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 --2- 172.21.15.110:0/762920549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66480ff7c0 0x7f6648195710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.675+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 >> 172.21.15.110:0/762920549 conn(0x7f66480fb330 msgr2=0x7f6648104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.676+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 shutdown_connections 2024-09-21T04:16:55.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.676+0000 7f664e257640 1 -- 172.21.15.110:0/762920549 wait complete. 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 -- 172.21.15.110:0/1042031476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68701005e0 msgr2=0x7f68701029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 --2- 172.21.15.110:0/1042031476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68701005e0 0x7f68701029d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f68600098b0 tx=0x7f686002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 -- 172.21.15.110:0/1042031476 shutdown_connections 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 --2- 172.21.15.110:0/1042031476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6870102f10 0x7f6870105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 --2- 172.21.15.110:0/1042031476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68701005e0 0x7f68701029d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 -- 172.21.15.110:0/1042031476 >> 172.21.15.110:0/1042031476 conn(0x7f68700fa1c0 msgr2=0x7f68700fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 -- 172.21.15.110:0/1042031476 shutdown_connections 2024-09-21T04:16:55.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.755+0000 7f68769d7640 1 -- 172.21.15.110:0/1042031476 wait complete. 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 Processor -- start 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 -- start start 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68701005e0 0x7f6870195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870196160 con 0x7f6870102f10 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f68769d7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68701962d0 con 0x7f68701005e0 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f686f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f686f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45902/0 (socket says 172.21.15.110:45902) 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f686f7fe640 1 -- 172.21.15.110:0/1345858293 learned_addr learned my addr 172.21.15.110:0/1345858293 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.756+0000 7f686ffff640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68701005e0 0x7f6870195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686f7fe640 1 -- 172.21.15.110:0/1345858293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68701005e0 msgr2=0x7f6870195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686f7fe640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68701005e0 0x7f6870195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686f7fe640 1 -- 172.21.15.110:0/1345858293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009560 con 0x7f6870102f10 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686f7fe640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f686400d8d0 tx=0x7f686400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686400bea0 con 0x7f6870102f10 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6864009c00 con 0x7f6870102f10 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687019ad70 con 0x7f6870102f10 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6864010460 con 0x7f6870102f10 2024-09-21T04:16:55.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.757+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687019b2c0 con 0x7f6870102f10 2024-09-21T04:16:55.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.758+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6870108770 con 0x7f6870102f10 2024-09-21T04:16:55.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.759+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f6864003f10 con 0x7f6870102f10 2024-09-21T04:16:55.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.759+0000 7f686d7fa640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 0x7f685807d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:55.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.759+0000 7f686ffff640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 0x7f685807d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:55.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.759+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f686409de60 con 0x7f6870102f10 2024-09-21T04:16:55.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.760+0000 7f686ffff640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 0x7f685807d4e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6860006010 tx=0x7f6860005d50 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:55.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:55.764+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f6864061130 con 0x7f6870102f10 2024-09-21T04:16:56.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.015+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f687019b4b0 con 0x7f6870102f10 2024-09-21T04:16:56.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.016+0000 7f686d7fa640 1 -- 172.21.15.110:0/1345858293 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f6864064250 con 0x7f6870102f10 2024-09-21T04:16:56.018 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:16:56.018 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 12, 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:16:56.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:16:56.020 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:16:56.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 msgr2=0x7f685807d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 0x7f685807d4e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6860006010 tx=0x7f6860005d50 comp rx=0 tx=0).stop 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 msgr2=0x7f6870195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f686400d8d0 tx=0x7f686400dd90 comp rx=0 tx=0).stop 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 shutdown_connections 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f685807b020 0x7f685807d4e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6870102f10 0x7f6870195b90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 --2- 172.21.15.110:0/1345858293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68701005e0 0x7f6870195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 >> 172.21.15.110:0/1345858293 conn(0x7f68700fa1c0 msgr2=0x7f68700fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 shutdown_connections 2024-09-21T04:16:56.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.018+0000 7f68769d7640 1 -- 172.21.15.110:0/1345858293 wait complete. 2024-09-21T04:16:56.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 -- 172.21.15.110:0/709121349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0fea30 msgr2=0x7f714c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 --2- 172.21.15.110:0/709121349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0fea30 0x7f714c0fee30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7148009880 tx=0x7f714802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 -- 172.21.15.110:0/709121349 shutdown_connections 2024-09-21T04:16:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 --2- 172.21.15.110:0/709121349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0ffc30 0x7f714c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 --2- 172.21.15.110:0/709121349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0fea30 0x7f714c0fee30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.100+0000 7f715401a640 1 -- 172.21.15.110:0/709121349 >> 172.21.15.110:0/709121349 conn(0x7f714c0fa1c0 msgr2=0x7f714c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 -- 172.21.15.110:0/709121349 shutdown_connections 2024-09-21T04:16:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 -- 172.21.15.110:0/709121349 wait complete. 2024-09-21T04:16:56.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 Processor -- start 2024-09-21T04:16:56.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 -- start start 2024-09-21T04:16:56.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0fea30 0x7f714c194510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c194f90 con 0x7f714c0ffc30 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f715401a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c195100 con 0x7f714c0fea30 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f71515b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f71515b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45920/0 (socket says 172.21.15.110:45920) 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.101+0000 7f71515b5640 1 -- 172.21.15.110:0/4027199624 learned_addr learned my addr 172.21.15.110:0/4027199624 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f7151db6640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0fea30 0x7f714c194510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f71515b5640 1 -- 172.21.15.110:0/4027199624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0fea30 msgr2=0x7f714c194510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f71515b5640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0fea30 0x7f714c194510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f71515b5640 1 -- 172.21.15.110:0/4027199624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148009560 con 0x7f714c0ffc30 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f71515b5640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f714000c8a0 tx=0x7f714000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7140007b80 con 0x7f714c0ffc30 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c199be0 con 0x7f714c0ffc30 2024-09-21T04:16:56.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7140007ce0 con 0x7f714c0ffc30 2024-09-21T04:16:56.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7140018810 con 0x7f714c0ffc30 2024-09-21T04:16:56.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.102+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c067c60 con 0x7f714c0ffc30 2024-09-21T04:16:56.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.103+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c107620 con 0x7f714c0ffc30 2024-09-21T04:16:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.104+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f7140018a80 con 0x7f714c0ffc30 2024-09-21T04:16:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.104+0000 7f713effd640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 0x7f713807f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.104+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f714009e260 con 0x7f714c0ffc30 2024-09-21T04:16:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.105+0000 7f7151db6640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 0x7f713807f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.105+0000 7f7151db6640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 0x7f713807f6f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7148009230 tx=0x7f7148005b50 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.109+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f7140061530 con 0x7f714c0ffc30 2024-09-21T04:16:56.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:55 smithi110 ceph-mon[27723]: pgmap v15: 65 pgs: 65 active+clean; 224 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 818 op/s 2024-09-21T04:16:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:55 smithi148 ceph-mon[34957]: pgmap v15: 65 pgs: 65 active+clean; 224 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 818 op/s 2024-09-21T04:16:56.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.348+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f714c1077e0 con 0x7f714c0ffc30 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.349+0000 7f713effd640 1 -- 172.21.15.110:0/4027199624 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f7140086e60 con 0x7f714c0ffc30 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:16:56.352 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:16:56.353 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:56.354 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:56.355 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 msgr2=0x7f713807f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 0x7f713807f6f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7148009230 tx=0x7f7148005b50 comp rx=0 tx=0).stop 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 msgr2=0x7f714c194a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f714000c8a0 tx=0x7f714000cd60 comp rx=0 tx=0).stop 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 shutdown_connections 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f713807d230 0x7f713807f6f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f714c0ffc30 0x7f714c194a50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 --2- 172.21.15.110:0/4027199624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c0fea30 0x7f714c194510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 >> 172.21.15.110:0/4027199624 conn(0x7f714c0fa1c0 msgr2=0x7f714c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 shutdown_connections 2024-09-21T04:16:56.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.351+0000 7f715401a640 1 -- 172.21.15.110:0/4027199624 wait complete. 2024-09-21T04:16:56.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 -- 172.21.15.110:0/618388718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00fea30 msgr2=0x7f5af00fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 --2- 172.21.15.110:0/618388718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00fea30 0x7f5af00fee30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009880 tx=0x7f5ae002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 -- 172.21.15.110:0/618388718 shutdown_connections 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 --2- 172.21.15.110:0/618388718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00ffc30 0x7f5af01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 --2- 172.21.15.110:0/618388718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00fea30 0x7f5af00fee30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 -- 172.21.15.110:0/618388718 >> 172.21.15.110:0/618388718 conn(0x7f5af00fa1c0 msgr2=0x7f5af00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 -- 172.21.15.110:0/618388718 shutdown_connections 2024-09-21T04:16:56.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.436+0000 7f5af65ed640 1 -- 172.21.15.110:0/618388718 wait complete. 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 Processor -- start 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 -- start start 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 0x7f5af0194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0195040 con 0x7f5af00ffc30 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5af65ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01951b0 con 0x7f5af00fea30 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45942/0 (socket says 172.21.15.110:45942) 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.437+0000 7f5ae7fff640 1 -- 172.21.15.110:0/3662996459 learned_addr learned my addr 172.21.15.110:0/3662996459 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5ae7fff640 1 -- 172.21.15.110:0/3662996459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 msgr2=0x7f5af0194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5aeffff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 0x7f5af0194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5ae7fff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 0x7f5af0194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5ae7fff640 1 -- 172.21.15.110:0/3662996459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0009560 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5aeffff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 0x7f5af0194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5ae7fff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d720 tx=0x7f5ad800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad80051f0 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ad8009ce0 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af0199c50 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad8010460 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.438+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af019a220 con 0x7f5af00ffc30 2024-09-21T04:16:56.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.439+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af0107640 con 0x7f5af00ffc30 2024-09-21T04:16:56.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.440+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f5ad8005350 con 0x7f5af00ffc30 2024-09-21T04:16:56.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.440+0000 7f5aedffb640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 0x7f5adc07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.440+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5ad809db00 con 0x7f5af00ffc30 2024-09-21T04:16:56.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.440+0000 7f5aeffff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 0x7f5adc07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.441+0000 7f5aeffff640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 0x7f5adc07d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0031540 tx=0x7f5ae0005de0 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.445+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f5ad8060e50 con 0x7f5af00ffc30 2024-09-21T04:16:56.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.641+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 --> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af0104580 con 0x7f5adc07b020 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.642+0000 7f5aedffb640 1 -- 172.21.15.110:0/3662996459 <== mgr.14790 v2:172.21.15.110:6800/1516382105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f5af0104580 con 0x7f5adc07b020 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "2/23 daemons upgraded", 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-09-21T04:16:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:16:56.646 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 msgr2=0x7f5adc07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 0x7f5adc07d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0031540 tx=0x7f5ae0005de0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 msgr2=0x7f5af0194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d720 tx=0x7f5ad800dbe0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 shutdown_connections 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f5adc07b020 0x7f5adc07d4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af00ffc30 0x7f5af0194a70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 --2- 172.21.15.110:0/3662996459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af00fea30 0x7f5af0194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 >> 172.21.15.110:0/3662996459 conn(0x7f5af00fa1c0 msgr2=0x7f5af0102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 shutdown_connections 2024-09-21T04:16:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.644+0000 7f5af65ed640 1 -- 172.21.15.110:0/3662996459 wait complete. 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 -- 172.21.15.110:0/2946571660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 msgr2=0x7f7f64100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 --2- 172.21.15.110:0/2946571660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f64100120 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009880 tx=0x7f7f5802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 -- 172.21.15.110:0/2946571660 shutdown_connections 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 --2- 172.21.15.110:0/2946571660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f64100120 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 --2- 172.21.15.110:0/2946571660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 0x7f7f640feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 -- 172.21.15.110:0/2946571660 >> 172.21.15.110:0/2946571660 conn(0x7f7f640fa230 msgr2=0x7f7f640fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:56.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 -- 172.21.15.110:0/2946571660 shutdown_connections 2024-09-21T04:16:56.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 -- 172.21.15.110:0/2946571660 wait complete. 2024-09-21T04:16:56.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.726+0000 7f7f69378640 1 Processor -- start 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f69378640 1 -- start start 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f69378640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 0x7f7f64068f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f69378640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f69378640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419a8f0 con 0x7f7f640ffca0 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f69378640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419aa60 con 0x7f7f640feaa0 2024-09-21T04:16:56.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45946/0 (socket says 172.21.15.110:45946) 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 -- 172.21.15.110:0/554561753 learned_addr learned my addr 172.21.15.110:0/554561753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f62ffd640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 0x7f7f64068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 -- 172.21.15.110:0/554561753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 msgr2=0x7f7f64068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 0x7f7f64068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.727+0000 7f7f627fc640 1 -- 172.21.15.110:0/554561753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58009560 con 0x7f7f640ffca0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f627fc640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7f580054f0 tx=0x7f7f58005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f58004aa0 con 0x7f7f640ffca0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6419ace0 con 0x7f7f640ffca0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f58004c00 con 0x7f7f640ffca0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f58003ac0 con 0x7f7f640ffca0 2024-09-21T04:16:56.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.728+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6419b250 con 0x7f7f640ffca0 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.729+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6418e640 con 0x7f7f640ffca0 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.730+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 32) v1 ==== 103285+0+0 (secure 0 0 0) 0x7f7f58031c70 con 0x7f7f640ffca0 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.730+0000 7f7f4bfff640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 0x7f7f4c07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.730+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7f580bf2f0 con 0x7f7f640ffca0 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.730+0000 7f7f62ffd640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 0x7f7f4c07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:16:56.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.730+0000 7f7f62ffd640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 0x7f7f4c07d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f54005fd0 tx=0x7f7f5400b390 comp rx=0 tx=0).ready entity=mgr.14790 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:16:56.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:56.734+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+195789 (secure 0 0 0) 0x7f7f580825e0 con 0x7f7f640ffca0 2024-09-21T04:16:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:56 smithi110 ceph-mon[27723]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:56 smithi110 ceph-mon[27723]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:56 smithi110 ceph-mon[27723]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:56 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/1345858293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:16:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:56 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/4027199624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:16:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:56 smithi148 ceph-mon[34957]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:56 smithi148 ceph-mon[34957]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:56 smithi148 ceph-mon[34957]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:56 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/1345858293' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:16:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:56 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/4027199624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:16:57.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.041+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f641078f0 con 0x7f7f640ffca0 2024-09-21T04:16:57.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.041+0000 7f7f4bfff640 1 -- 172.21.15.110:0/554561753 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7f58085700 con 0x7f7f640ffca0 2024-09-21T04:16:57.044 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:16:57.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 msgr2=0x7f7f4c07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:57.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 0x7f7f4c07d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f54005fd0 tx=0x7f7f5400b390 comp rx=0 tx=0).stop 2024-09-21T04:16:57.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 msgr2=0x7f7f6419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:16:57.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7f580054f0 tx=0x7f7f58005520 comp rx=0 tx=0).stop 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 shutdown_connections 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:6800/1516382105,v1:172.21.15.110:6801/1516382105] conn(0x7f7f4c07b020 0x7f7f4c07d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640ffca0 0x7f7f6419a3b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 --2- 172.21.15.110:0/554561753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f640feaa0 0x7f7f64068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 >> 172.21.15.110:0/554561753 conn(0x7f7f640fa230 msgr2=0x7f7f64103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 shutdown_connections 2024-09-21T04:16:57.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:16:57.043+0000 7f7f69378640 1 -- 172.21.15.110:0/554561753 wait complete. 2024-09-21T04:16:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:57 smithi110 ceph-mon[27723]: pgmap v16: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 881 op/s 2024-09-21T04:16:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:57 smithi110 ceph-mon[27723]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:57 smithi110 ceph-mon[27723]: from='client.? 172.21.15.110:0/554561753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:16:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:57 smithi148 ceph-mon[34957]: pgmap v16: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 881 op/s 2024-09-21T04:16:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:57 smithi148 ceph-mon[34957]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:16:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:57 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/554561753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:17:00.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:59 smithi110 ceph-mon[27723]: pgmap v17: 65 pgs: 65 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 1.9 MiB/s wr, 757 op/s 2024-09-21T04:17:00.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:59 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:00.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:59 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:00.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:59 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:17:00.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:16:59 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:59 smithi148 ceph-mon[34957]: pgmap v17: 65 pgs: 65 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 1.9 MiB/s wr, 757 op/s 2024-09-21T04:17:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:59 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:59 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:59 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:17:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:16:59 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:01.124 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:00 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:17:01.124 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:00 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:17:01.125 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:00 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:17:01.125 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:00 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:01.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:00 smithi110 ceph-mon[27723]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:17:01.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:00 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:17:01.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:00 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:17:01.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:00 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:01 smithi110 ceph-mon[27723]: pgmap v18: 65 pgs: 65 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 418 op/s 2024-09-21T04:17:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:01 smithi110 ceph-mon[27723]: Upgrade: Updating mgr.smithi148.zjrahi 2024-09-21T04:17:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:01 smithi110 ceph-mon[27723]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:17:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:01 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:17:02.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:01 smithi148 ceph-mon[34957]: pgmap v18: 65 pgs: 65 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 418 op/s 2024-09-21T04:17:02.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:01 smithi148 ceph-mon[34957]: Upgrade: Updating mgr.smithi148.zjrahi 2024-09-21T04:17:02.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:01 smithi148 ceph-mon[34957]: Deploying daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:17:02.180 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:01 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:17:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:03 smithi110 ceph-mon[27723]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 439 op/s 2024-09-21T04:17:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:03 smithi148 ceph-mon[34957]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 439 op/s 2024-09-21T04:17:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:05 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:05 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:05 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:05 smithi148 ceph-mon[34957]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 563 KiB/s wr, 166 op/s 2024-09-21T04:17:05.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:05 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:05.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:05 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:05.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:05 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:05.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:05 smithi110 ceph-mon[27723]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 563 KiB/s wr, 166 op/s 2024-09-21T04:17:08.114 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:07 smithi148 ceph-mon[34957]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 563 KiB/s wr, 166 op/s 2024-09-21T04:17:08.114 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:07 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.114 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:07 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.114 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:07 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.114 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:07 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:07 smithi110 ceph-mon[27723]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 563 KiB/s wr, 166 op/s 2024-09-21T04:17:08.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:07 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:07 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:07 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:08.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:07 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 372 KiB/s wr, 103 op/s 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:17:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:17:10.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:17:10.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:17:10.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:09 smithi110 ceph-mon[27723]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:17:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 372 KiB/s wr, 103 op/s 2024-09-21T04:17:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:17:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:09 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1451233723' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: mgrmap e33: smithi110.axmqzb(active, since 39s), standbys: smithi148.zjrahi 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:17:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.axmqzb"}]: dispatch 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.axmqzb"}]': finished 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.zjrahi"}]: dispatch 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.zjrahi"}]': finished 2024-09-21T04:17:11.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:10 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: mgrmap e33: smithi110.axmqzb(active, since 39s), standbys: smithi148.zjrahi 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:17:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.axmqzb"}]: dispatch 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.axmqzb"}]': finished 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.zjrahi"}]: dispatch 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.zjrahi"}]': finished 2024-09-21T04:17:11.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:10 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:17:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:11 smithi110 ceph-mon[27723]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-09-21T04:17:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:11 smithi110 ceph-mon[27723]: Upgrade: Setting container_image for all mgr 2024-09-21T04:17:12.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:11 smithi148 ceph-mon[34957]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-09-21T04:17:12.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:11 smithi148 ceph-mon[34957]: Upgrade: Setting container_image for all mgr 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: Upgrade: Updating mon.smithi110 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: Deploying daemon mon.smithi110 on smithi110 2024-09-21T04:17:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 ceph-mon[27723]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: Upgrade: Updating mon.smithi110 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: Deploying daemon mon.smithi110 on smithi110 2024-09-21T04:17:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:13 smithi148 ceph-mon[34957]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-09-21T04:17:14.144 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:13 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:17:14.399 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:14 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[27719]: 2024-09-21T04:17:14.141+0000 7fed98ff2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:14.399 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:14 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[27719]: 2024-09-21T04:17:14.141+0000 7fed98ff2640 -1 mon.smithi110@0(leader) e2 *** Got Signal Terminated *** 2024-09-21T04:17:14.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:14 smithi110 podman[92274]: 2024-09-21 04:17:14.397350362 +0000 UTC m=+0.400258188 container died 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-6a1bdf8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-21T04:17:15.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:14 smithi110 podman[92274]: 2024-09-21 04:17:14.825262419 +0000 UTC m=+0.828170241 container cleanup 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, RELEASE=reef-6a1bdf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:17:15.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:14 smithi110 bash[92274]: ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110 2024-09-21T04:17:15.304 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 podman[92297]: 2024-09-21 04:17:15.043497831 +0000 UTC m=+0.643201291 container remove 199e013636ee3b51f92f5f34c55c64feb04c5c15522bc0bc8c574578b735e156 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=reef-6a1bdf8, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T04:17:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service: Deactivated successfully. 2024-09-21T04:17:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 systemd[1]: Stopped Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:17:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service: Consumed 11.414s CPU time. 2024-09-21T04:17:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 systemd[1]: Starting Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:17:16.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 podman[92434]: 2024-09-21 04:17:15.666163927 +0000 UTC m=+0.016589831 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:17:16.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:15 smithi110 podman[92434]: 2024-09-21 04:17:15.928492332 +0000 UTC m=+0.278918223 container create 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 podman[92434]: 2024-09-21 04:17:16.356532239 +0000 UTC m=+0.706958135 container init 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 podman[92434]: 2024-09-21 04:17:16.360123416 +0000 UTC m=+0.710549309 container start 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev), process ceph-mon, pid 2 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: pidfile_write: ignore empty --pid-file 2024-09-21T04:17:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: load: jerasure load: lrc 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Git sha 0 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Compile date 2024-09-20 23:32:13 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: DB SUMMARY 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: DB Session ID: OXJG2CJNBO1P5Z7RC77Z 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1507 Bytes 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi110/store.db dir, Total Num: 1, files: 000032.sst 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi110/store.db: 000030.log size: 458467 ; 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:17:16.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.env: 0x55f14f4d3120 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.info_log: 0x55f150f0b8e0 2024-09-21T04:17:16.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:17:16.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.statistics: (nil) 2024-09-21T04:17:16.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.use_fsync: 0 2024-09-21T04:17:16.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:17:16.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.db_log_dir: 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.wal_dir: 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:17:16.671 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.write_buffer_manager: 0x55f150f0fb80 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:17:16.672 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.unordered_write: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.row_cache: None 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.wal_filter: None 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.wal_compression: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:17:16.673 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:17:16.674 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_open_files: -1 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Compression algorithms supported: 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kZSTD supported: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:17:16.675 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000015 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.merge_operator: 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_filter: None 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:17:16.676 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f150f0b540) 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_type: 0 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_index_type: 0 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_shortening: 1 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: checksum: 4 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: no_block_cache: 0 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache: 0x55f150eff610 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:17:16.677 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_options: 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: capacity : 536870912 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_shard_bits : 4 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: strict_capacity_limit : 0 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_compressed: (nil) 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: persistent_cache: (nil) 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size: 4096 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size_deviation: 10 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_restart_interval: 16 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_block_restart_interval: 1 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_block_size: 4096 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: partition_filters: 0 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: use_delta_encoding: 1 2024-09-21T04:17:16.678 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: filter_policy: bloomfilter 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: whole_key_filtering: 1 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: verify_compression: 0 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: format_version: 5 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_index_compression: 1 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_align: 0 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_auto_readahead_size: 262144 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: prepopulate_block_cache: 0 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression: NoCompression 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:17:16.679 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.num_levels: 7 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:17:16.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:17:16.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:17:16.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:17:16.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.ttl: 2592000 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:17:16.684 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 11985, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 851bb4f2-0a44-4f16-8fba-304dcb9f1d37 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892236387912, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892236390094, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 444187, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11986, "largest_seqno": 12172, "table_properties": {"data_size": 442208, "index_size": 621, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2998, "raw_average_key_size": 28, "raw_value_size": 439645, "raw_average_value_size": 4108, "num_data_blocks": 26, "num_entries": 107, "num_filter_entries": 107, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726892236, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "851bb4f2-0a44-4f16-8fba-304dcb9f1d37", "db_session_id": "OXJG2CJNBO1P5Z7RC77Z", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892236390196, "job": 1, "event": "recovery_finished"} 2024-09-21T04:17:16.685 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi110/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f150f2ee00 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: DB pointer 0x55f151030000 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** DB Stats ** 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:17:16.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L0 1/0 433.78 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 318.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L6 1/0 7.02 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Sum 2/0 7.44 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 318.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 318.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:17:16.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 318.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative compaction: 0.00 GB write, 94.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval compaction: 0.00 GB write, 94.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache BinnedLRUCache@0x55f150eff610#2 capacity: 512.00 MB usage: 1.08 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T04:17:16.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.70 KB,0.00013411%) Misc(1,0.00 KB,0%) 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: starting mon.smithi110 rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi110 fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???) e2 preinit fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).mds e15 new map 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).mds e15 print_map 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: e15 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: legacy client fscid: 1 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Filesystem 'cephfs' (1) 2024-09-21T04:17:16.689 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: fs_name cephfs 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: epoch 15 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: tableserver 0 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: root 0 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: session_timeout 60 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: session_autoclose 300 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_file_size 1099511627776 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_xattr_size 65536 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: required_client_features {} 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: last_failure 0 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: last_failure_osd_epoch 46 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:16.690 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_mds 1 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: in 0 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: up {0=14562} 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: failed 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: damaged 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: stopped 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_pools [3] 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_pool 2 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: inline_data disabled 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: balancer 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: bal_rank_mask -1 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: standby_count_wanted 1 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: qdb_cluster leader: 0 members: 2024-09-21T04:17:16.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Standby daemons: 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 ceph-mon[92455]: mon.smithi110@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-09-21T04:17:16.692 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 bash[92434]: 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 2024-09-21T04:17:16.693 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:16 smithi110 systemd[1]: Started Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:17:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: mon.smithi110 calling monitor election 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: monmap epoch 2 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: last_changed 2024-09-21T04:07:43.822032+0000 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: created 2024-09-21T04:05:19.513898+0000 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: min_mon_release 18 (reef) 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: election_strategy: 1 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: osdmap e48: 6 total, 6 up, 6 in 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: mgrmap e33: smithi110.axmqzb(active, since 47s), standbys: smithi148.zjrahi 2024-09-21T04:17:18.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: overall HEALTH_OK 2024-09-21T04:17:18.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: from='mgr.14790 ' entity='' 2024-09-21T04:17:18.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:17 smithi110 ceph-mon[92455]: mgrmap e34: smithi110.axmqzb(active, since 47s), standbys: smithi148.zjrahi 2024-09-21T04:17:18.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mgr.14790 172.21.15.110:0/851518049' entity='mgr.smithi110.axmqzb' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: mon.smithi110 calling monitor election 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: monmap epoch 2 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: last_changed 2024-09-21T04:07:43.822032+0000 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: created 2024-09-21T04:05:19.513898+0000 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: min_mon_release 18 (reef) 2024-09-21T04:17:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: election_strategy: 1 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: osdmap e48: 6 total, 6 up, 6 in 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: mgrmap e33: smithi110.axmqzb(active, since 47s), standbys: smithi148.zjrahi 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: overall HEALTH_OK 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: from='mgr.14790 ' entity='' 2024-09-21T04:17:18.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:17 smithi148 ceph-mon[34957]: mgrmap e34: smithi110.axmqzb(active, since 47s), standbys: smithi148.zjrahi 2024-09-21T04:17:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:17:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:17:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: mgrmap e35: smithi110.axmqzb(active, starting, since 0.00527005s), standbys: smithi148.zjrahi 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi110"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi148"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"} : dispatch 2024-09-21T04:17:23.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T04:17:23.169 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:22 smithi110 ceph-mon[92455]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi restarted 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: Standby manager daemon smithi148.zjrahi started 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/crt"}]: dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.zjrahi/key"}]: dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.? 172.21.15.148:0/1591423937' entity='mgr.smithi148.zjrahi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: Active manager daemon smithi110.axmqzb restarted 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: Activating manager daemon smithi110.axmqzb 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: mgrmap e35: smithi110.axmqzb(active, starting, since 0.00527005s), standbys: smithi148.zjrahi 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi110"} : dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi148"} : dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:17:23.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr metadata", "who": "smithi110.axmqzb", "id": "smithi110.axmqzb"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr metadata", "who": "smithi148.zjrahi", "id": "smithi148.zjrahi"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T04:17:23.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:22 smithi148 ceph-mon[34957]: Manager daemon smithi110.axmqzb is now available 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"} : dispatch 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"} : dispatch 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:23 smithi110 ceph-mon[92455]: mgrmap e36: smithi110.axmqzb(active, since 1.0118s), standbys: smithi148.zjrahi 2024-09-21T04:17:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T04:17:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/mirror_snapshot_schedule"} : dispatch 2024-09-21T04:17:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.axmqzb/trash_purge_schedule"} : dispatch 2024-09-21T04:17:24.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:24.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:24.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:23 smithi148 ceph-mon[34957]: mgrmap e36: smithi110.axmqzb(active, since 1.0118s), standbys: smithi148.zjrahi 2024-09-21T04:17:25.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:24 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:25.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:24 smithi110 ceph-mon[92455]: pgmap v3: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:25.139 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:24 smithi148 ceph-mon[34957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:17:25.139 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:24 smithi148 ceph-mon[34957]: pgmap v3: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: [21/Sep/2024:04:17:23] ENGINE Bus STARTING 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: [21/Sep/2024:04:17:24] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: [21/Sep/2024:04:17:24] ENGINE Error in HTTPServer.serve 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Traceback (most recent call last): 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:17:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._run(expiration_interval) 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self.do_handshake() 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:17:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T04:17:25.972 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:17:25.972 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: [21/Sep/2024:04:17:24] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:17:25.972 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: [21/Sep/2024:04:17:24] ENGINE Bus STARTED 2024-09-21T04:17:25.972 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: pgmap v4: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:25.972 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:25 smithi110 ceph-mon[92455]: mgrmap e37: smithi110.axmqzb(active, since 2s), standbys: smithi148.zjrahi 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: [21/Sep/2024:04:17:23] ENGINE Bus STARTING 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: [21/Sep/2024:04:17:24] ENGINE Serving on https://172.21.15.110:7150 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: [21/Sep/2024:04:17:24] ENGINE Error in HTTPServer.serve 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Traceback (most recent call last): 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:17:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._run(expiration_interval) 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: s = self.context.wrap_socket( 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: return self.sslsocket_class._create( 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self.do_handshake() 2024-09-21T04:17:26.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: self._sslobj.do_handshake() 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: [21/Sep/2024:04:17:24] ENGINE Serving on http://172.21.15.110:8765 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: [21/Sep/2024:04:17:24] ENGINE Bus STARTED 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: pgmap v4: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:26.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:25 smithi148 ceph-mon[34957]: mgrmap e37: smithi110.axmqzb(active, since 2s), standbys: smithi148.zjrahi 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 -- 172.21.15.110:0/1305876351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 msgr2=0x7f64ac0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 --2- 172.21.15.110:0/1305876351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac0fee30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f649c009880 tx=0x7f649c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 -- 172.21.15.110:0/1305876351 shutdown_connections 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 --2- 172.21.15.110:0/1305876351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 --2- 172.21.15.110:0/1305876351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac0fee30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 -- 172.21.15.110:0/1305876351 >> 172.21.15.110:0/1305876351 conn(0x7f64ac0fa1c0 msgr2=0x7f64ac0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 -- 172.21.15.110:0/1305876351 shutdown_connections 2024-09-21T04:17:27.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.133+0000 7f64b29e4640 1 -- 172.21.15.110:0/1305876351 wait complete. 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 Processor -- start 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 -- start start 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac198350 con 0x7f64ac0fea30 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64b29e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1984c0 con 0x7f64ac0ffc30 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34390/0 (socket says 172.21.15.110:34390) 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 -- 172.21.15.110:0/356917568 learned_addr learned my addr 172.21.15.110:0/356917568 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64ab7fe640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 -- 172.21.15.110:0/356917568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 msgr2=0x7f64ac197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64abfff640 1 -- 172.21.15.110:0/356917568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c009560 con 0x7f64ac0fea30 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.134+0000 7f64ab7fe640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64abfff640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f649c0054f0 tx=0x7f649c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c004aa0 con 0x7f64ac0fea30 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f649c004c00 con 0x7f64ac0fea30 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac19cf00 con 0x7f64ac0fea30 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c003ac0 con 0x7f64ac0fea30 2024-09-21T04:17:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.135+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac19d370 con 0x7f64ac0fea30 2024-09-21T04:17:27.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.136+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac107620 con 0x7f64ac0fea30 2024-09-21T04:17:27.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.137+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 103368+0+0 (secure 0 0 0) 0x7f649c003d80 con 0x7f64ac0fea30 2024-09-21T04:17:27.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.137+0000 7f64a97fa640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 0x7f649807d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.137+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f649c0bf730 con 0x7f64ac0fea30 2024-09-21T04:17:27.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.137+0000 7f64ab7fe640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 0x7f649807d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.138+0000 7f64ab7fe640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 0x7f649807d7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6494005d00 tx=0x7f6494005ed0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.143+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f649c082600 con 0x7f64ac0fea30 2024-09-21T04:17:27.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.342+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64ac0fc3e0 con 0x7f649807b310 2024-09-21T04:17:27.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.343+0000 7f64a97fa640 1 -- 172.21.15.110:0/356917568 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f64ac0fc3e0 con 0x7f649807b310 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 msgr2=0x7f649807d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 0x7f649807d7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6494005d00 tx=0x7f6494005ed0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 msgr2=0x7f64ac197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f649c0054f0 tx=0x7f649c005520 comp rx=0 tx=0).stop 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 shutdown_connections 2024-09-21T04:17:27.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f649807b310 0x7f649807d7d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac0ffc30 0x7f64ac197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 --2- 172.21.15.110:0/356917568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64ac0fea30 0x7f64ac197840 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 >> 172.21.15.110:0/356917568 conn(0x7f64ac0fa1c0 msgr2=0x7f64ac0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 shutdown_connections 2024-09-21T04:17:27.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.345+0000 7f64b29e4640 1 -- 172.21.15.110:0/356917568 wait complete. 2024-09-21T04:17:27.357 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:17:27.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.446+0000 7f68ee7fc640 1 -- 172.21.15.110:0/1141027267 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e00051f0 con 0x7f68f00fde10 2024-09-21T04:17:27.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.446+0000 7f68f5971640 1 -- 172.21.15.110:0/1141027267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 msgr2=0x7f68f00fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.446+0000 7f68f5971640 1 --2- 172.21.15.110:0/1141027267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f00fe210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009880 tx=0x7f68e002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 -- 172.21.15.110:0/1141027267 shutdown_connections 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 --2- 172.21.15.110:0/1141027267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 0x7f68f0101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 --2- 172.21.15.110:0/1141027267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f00fe210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 -- 172.21.15.110:0/1141027267 >> 172.21.15.110:0/1141027267 conn(0x7f68f00f9940 msgr2=0x7f68f00fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 -- 172.21.15.110:0/1141027267 shutdown_connections 2024-09-21T04:17:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 -- 172.21.15.110:0/1141027267 wait complete. 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.447+0000 7f68f5971640 1 Processor -- start 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68f5971640 1 -- start start 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68f5971640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68f5971640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 0x7f68f01007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68f5971640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01a2000 con 0x7f68f00fde10 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68f5971640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01a2170 con 0x7f68f0100e90 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34414/0 (socket says 172.21.15.110:34414) 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 -- 172.21.15.110:0/4063004750 learned_addr learned my addr 172.21.15.110:0/4063004750 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68e7fff640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 0x7f68f01007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 -- 172.21.15.110:0/4063004750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 msgr2=0x7f68f01007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 0x7f68f01007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.448+0000 7f68eeffd640 1 -- 172.21.15.110:0/4063004750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0009560 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68eeffd640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006010 tx=0x7f68e0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e0004430 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68e0004e80 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00fec80 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e0031ae0 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.449+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f00ff1f0 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.450+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f006be10 con 0x7f68f00fde10 2024-09-21T04:17:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.451+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 103368+0+0 (secure 0 0 0) 0x7f68e0003cd0 con 0x7f68f00fde10 2024-09-21T04:17:27.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.451+0000 7f68f496f640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 0x7f68dc07d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.451+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68e00bf280 con 0x7f68f00fde10 2024-09-21T04:17:27.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.452+0000 7f68e7fff640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 0x7f68dc07d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.452+0000 7f68e7fff640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 0x7f68dc07d7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ffff0 tx=0x7f68d800b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.458+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f68e0082280 con 0x7f68f00fde10 2024-09-21T04:17:27.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.587+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f68e0081ea0 con 0x7f68f00fde10 2024-09-21T04:17:27.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.661+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68f00ff650 con 0x7f68dc07b310 2024-09-21T04:17:27.663 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.664 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.664 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:27 smithi110 ceph-mon[92455]: pgmap v5: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:27.664 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.664 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.662+0000 7f68f496f640 1 -- 172.21.15.110:0/4063004750 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f68f00ff650 con 0x7f68dc07b310 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 msgr2=0x7f68dc07d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 0x7f68dc07d7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ffff0 tx=0x7f68d800b390 comp rx=0 tx=0).stop 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 msgr2=0x7f68f0100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006010 tx=0x7f68e0004c60 comp rx=0 tx=0).stop 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 shutdown_connections 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f68dc07b310 0x7f68dc07d7d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68f0100e90 0x7f68f01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 --2- 172.21.15.110:0/4063004750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68f00fde10 0x7f68f0100270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 >> 172.21.15.110:0/4063004750 conn(0x7f68f00f9940 msgr2=0x7f68f01040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 shutdown_connections 2024-09-21T04:17:27.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.664+0000 7f68f5971640 1 -- 172.21.15.110:0/4063004750 wait complete. 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 -- 172.21.15.110:0/372488392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84102f40 msgr2=0x7f4e84105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 --2- 172.21.15.110:0/372488392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84102f40 0x7f4e84105330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e700098b0 tx=0x7f4e7002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 -- 172.21.15.110:0/372488392 shutdown_connections 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 --2- 172.21.15.110:0/372488392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84102f40 0x7f4e84105330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 --2- 172.21.15.110:0/372488392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84100610 0x7f4e84102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 -- 172.21.15.110:0/372488392 >> 172.21.15.110:0/372488392 conn(0x7f4e840fa230 msgr2=0x7f4e840fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 -- 172.21.15.110:0/372488392 shutdown_connections 2024-09-21T04:17:27.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 -- 172.21.15.110:0/372488392 wait complete. 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.756+0000 7f4e8affb640 1 Processor -- start 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 -- start start 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84102f40 0x7f4e84195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84196180 con 0x7f4e84100610 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841962f0 con 0x7f4e84102f40 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34432/0 (socket says 172.21.15.110:34432) 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 -- 172.21.15.110:0/2235903966 learned_addr learned my addr 172.21.15.110:0/2235903966 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e83fff640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84102f40 0x7f4e84195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 -- 172.21.15.110:0/2235903966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84102f40 msgr2=0x7f4e84195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84102f40 0x7f4e84195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 -- 172.21.15.110:0/2235903966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009560 con 0x7f4e84100610 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e88d97640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e9a0 tx=0x7f4e7800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7800cd50 con 0x7f4e84100610 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8419ad90 con 0x7f4e84100610 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.757+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e7800ceb0 con 0x7f4e84100610 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.758+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e78010660 con 0x7f4e84100610 2024-09-21T04:17:27.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.758+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8419b2d0 con 0x7f4e84100610 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.759+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e84108790 con 0x7f4e84100610 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.759+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f4e780048a0 con 0x7f4e84100610 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.760+0000 7f4e81ffb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 0x7f4e7407d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.760+0000 7f4e83fff640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 0x7f4e7407d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.760+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4e78014070 con 0x7f4e84100610 2024-09-21T04:17:27.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.760+0000 7f4e83fff640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 0x7f4e7407d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009880 tx=0x7f4e700314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:27.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.764+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4e78065010 con 0x7f4e84100610 2024-09-21T04:17:27.823 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:27 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.823 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:27 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.823 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:27 smithi148 ceph-mon[34957]: pgmap v5: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:27.823 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:27 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.823 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:27 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:27.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.953+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e84103960 con 0x7f4e7407b3a0 2024-09-21T04:17:27.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.963+0000 7f4e81ffb640 1 -- 172.21.15.110:0/2235903966 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4e84103960 con 0x7f4e7407b3a0 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (8m) 19s ago 10m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (110s) 19s ago 10m 7587k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (101s) 1s ago 9m 7302k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (10m) 19s ago 10m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (9m) 1s ago 9m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (8m) 19s ago 10m 87.5M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (4m) 19s ago 4m 158M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (4m) 19s ago 4m 14.6M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (4m) 1s ago 4m 16.5M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (4m) 1s ago 4m 18.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (93s) 19s ago 11m 508M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:17:27.966 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (24s) 1s ago 9m 451M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (12m) 19s ago 12m 57.7M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 199e013636ee 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (9m) 1s ago 9m 47.6M 2048M 18.2.4-871-g6a1bdf85 0adab276c97e 6e2e4585f6e1 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (10m) 19s ago 10m 17.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (9m) 1s ago 9m 15.1M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (8m) 19s ago 8m 286M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (7m) 19s ago 7m 276M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (7m) 19s ago 7m 227M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (7m) 1s ago 7m 324M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (6m) 1s ago 6m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (6m) 1s ago 6m 253M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:17:27.967 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (29s) 19s ago 10m 41.6M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 msgr2=0x7f4e7407d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 0x7f4e7407d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009880 tx=0x7f4e700314e0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 msgr2=0x7f4e84195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e9a0 tx=0x7f4e7800eda0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 shutdown_connections 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e7407b3a0 0x7f4e7407d860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e84102f40 0x7f4e84195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 --2- 172.21.15.110:0/2235903966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e84100610 0x7f4e84195670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 >> 172.21.15.110:0/2235903966 conn(0x7f4e840fa230 msgr2=0x7f4e840fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 shutdown_connections 2024-09-21T04:17:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:27.965+0000 7f4e8affb640 1 -- 172.21.15.110:0/2235903966 wait complete. 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 -- 172.21.15.110:0/3584001286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0ffca0 msgr2=0x7faa5c100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 --2- 172.21.15.110:0/3584001286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0ffca0 0x7faa5c100120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009880 tx=0x7faa4c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 -- 172.21.15.110:0/3584001286 shutdown_connections 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 --2- 172.21.15.110:0/3584001286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0ffca0 0x7faa5c100120 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 --2- 172.21.15.110:0/3584001286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0feaa0 0x7faa5c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 -- 172.21.15.110:0/3584001286 >> 172.21.15.110:0/3584001286 conn(0x7faa5c0fa230 msgr2=0x7faa5c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 -- 172.21.15.110:0/3584001286 shutdown_connections 2024-09-21T04:17:28.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.045+0000 7faa63e38640 1 -- 172.21.15.110:0/3584001286 wait complete. 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 Processor -- start 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 -- start start 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0ffca0 0x7faa5c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c19aa90 con 0x7faa5c0feaa0 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa63e38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c19abd0 con 0x7faa5c0ffca0 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34448/0 (socket says 172.21.15.110:34448) 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 -- 172.21.15.110:0/632457860 learned_addr learned my addr 172.21.15.110:0/632457860 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa613d3640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0ffca0 0x7faa5c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 -- 172.21.15.110:0/632457860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0ffca0 msgr2=0x7faa5c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0ffca0 0x7faa5c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.046+0000 7faa61bd4640 1 -- 172.21.15.110:0/632457860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c009560 con 0x7faa5c0feaa0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa61bd4640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b3d0 tx=0x7faa5800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa58011800 con 0x7faa5c0feaa0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa58011e20 con 0x7faa5c0feaa0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5800f510 con 0x7faa5c0feaa0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c19ae80 con 0x7faa5c0feaa0 2024-09-21T04:17:28.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.047+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c19b3c0 con 0x7faa5c0feaa0 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.048+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c0fef20 con 0x7faa5c0feaa0 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.048+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7faa58010380 con 0x7faa5c0feaa0 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.049+0000 7faa52ffd640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 0x7faa4807d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.049+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faa5809df80 con 0x7faa5c0feaa0 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.049+0000 7faa613d3640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 0x7faa4807d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.050+0000 7faa613d3640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 0x7faa4807d860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa4c000c00 tx=0x7faa4c0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.053+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7faa58060df0 con 0x7faa5c0feaa0 2024-09-21T04:17:28.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.317+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa5c18e6c0 con 0x7faa5c0feaa0 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.317+0000 7faa52ffd640 1 -- 172.21.15.110:0/632457860 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7faa58063f10 con 0x7faa5c0feaa0 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 1, 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:17:28.320 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 11, 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:17:28.321 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:17:28.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 msgr2=0x7faa4807d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 0x7faa4807d860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa4c000c00 tx=0x7faa4c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 msgr2=0x7faa5c068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b3d0 tx=0x7faa5800b890 comp rx=0 tx=0).stop 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 shutdown_connections 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7faa4807b3a0 0x7faa4807d860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa5c0ffca0 0x7faa5c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 --2- 172.21.15.110:0/632457860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa5c0feaa0 0x7faa5c068f60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 >> 172.21.15.110:0/632457860 conn(0x7faa5c0fa230 msgr2=0x7faa5c1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 shutdown_connections 2024-09-21T04:17:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.320+0000 7faa63e38640 1 -- 172.21.15.110:0/632457860 wait complete. 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- 172.21.15.110:0/271095519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e480fecc0 msgr2=0x7f0e480ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 --2- 172.21.15.110:0/271095519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e480fecc0 0x7f0e480ff140 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009880 tx=0x7f0e3c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- 172.21.15.110:0/271095519 shutdown_connections 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 --2- 172.21.15.110:0/271095519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e480fecc0 0x7f0e480ff140 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 --2- 172.21.15.110:0/271095519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e48100010 0x7f0e48100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- 172.21.15.110:0/271095519 >> 172.21.15.110:0/271095519 conn(0x7f0e480f9940 msgr2=0x7f0e480fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- 172.21.15.110:0/271095519 shutdown_connections 2024-09-21T04:17:28.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- 172.21.15.110:0/271095519 wait complete. 2024-09-21T04:17:28.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 Processor -- start 2024-09-21T04:17:28.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.424+0000 7f0e4e311640 1 -- start start 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e4e311640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 0x7f0e48195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e4e311640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e4e311640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e48196130 con 0x7f0e48100010 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e4e311640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e481962a0 con 0x7f0e480fecc0 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34480/0 (socket says 172.21.15.110:34480) 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 -- 172.21.15.110:0/861463585 learned_addr learned my addr 172.21.15.110:0/861463585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e47fff640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 0x7f0e48195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 -- 172.21.15.110:0/861463585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 msgr2=0x7f0e48195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 0x7f0e48195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e477fe640 1 -- 172.21.15.110:0/861463585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009560 con 0x7f0e48100010 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.425+0000 7f0e47fff640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 0x7f0e48195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e477fe640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005520 tx=0x7f0e3c005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e3c004aa0 con 0x7f0e48100010 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4819ace0 con 0x7f0e48100010 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e3c004c00 con 0x7f0e48100010 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e3c003b10 con 0x7f0e48100010 2024-09-21T04:17:28.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.426+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4819b250 con 0x7f0e48100010 2024-09-21T04:17:28.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.427+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e48107610 con 0x7f0e48100010 2024-09-21T04:17:28.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.428+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f0e3c003c70 con 0x7f0e48100010 2024-09-21T04:17:28.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.428+0000 7f0e457fa640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 0x7f0e3007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.428+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0e3c0bf680 con 0x7f0e48100010 2024-09-21T04:17:28.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.428+0000 7f0e47fff640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 0x7f0e3007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.429+0000 7f0e47fff640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 0x7f0e3007d860 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e480679a0 tx=0x7f0e3800b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.433+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f0e3c0825a0 con 0x7f0e48100010 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.659+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e4819b490 con 0x7f0e48100010 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.660+0000 7f0e457fa640 1 -- 172.21.15.110:0/861463585 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f0e3c0856c0 con 0x7f0e48100010 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:17:28.663 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:28.664 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:28.665 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:28.666 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 msgr2=0x7f0e3007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 0x7f0e3007d860 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e480679a0 tx=0x7f0e3800b390 comp rx=0 tx=0).stop 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 msgr2=0x7f0e48195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c005520 tx=0x7f0e3c005550 comp rx=0 tx=0).stop 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 shutdown_connections 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e3007b3a0 0x7f0e3007d860 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e48100010 0x7f0e48195b60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 --2- 172.21.15.110:0/861463585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e480fecc0 0x7f0e48195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 >> 172.21.15.110:0/861463585 conn(0x7f0e480f9940 msgr2=0x7f0e480fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 shutdown_connections 2024-09-21T04:17:28.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.662+0000 7f0e4e311640 1 -- 172.21.15.110:0/861463585 wait complete. 2024-09-21T04:17:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 -- 172.21.15.110:0/1722137296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 msgr2=0x7f27f80fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 --2- 172.21.15.110:0/1722137296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f80fee30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009880 tx=0x7f27f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 -- 172.21.15.110:0/1722137296 shutdown_connections 2024-09-21T04:17:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 --2- 172.21.15.110:0/1722137296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 0x7f27f81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 --2- 172.21.15.110:0/1722137296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f80fee30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 -- 172.21.15.110:0/1722137296 >> 172.21.15.110:0/1722137296 conn(0x7f27f80fa1c0 msgr2=0x7f27f80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 -- 172.21.15.110:0/1722137296 shutdown_connections 2024-09-21T04:17:28.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.747+0000 7f27feca5640 1 -- 172.21.15.110:0/1722137296 wait complete. 2024-09-21T04:17:28.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 Processor -- start 2024-09-21T04:17:28.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 -- start start 2024-09-21T04:17:28.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 0x7f27f819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f819a960 con 0x7f27f80fea30 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27feca5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f819aaa0 con 0x7f27f80ffc30 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27fca41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27fca41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34512/0 (socket says 172.21.15.110:34512) 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.748+0000 7f27fca41640 1 -- 172.21.15.110:0/1829892482 learned_addr learned my addr 172.21.15.110:0/1829892482 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27effff640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 0x7f27f819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27fca41640 1 -- 172.21.15.110:0/1829892482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 msgr2=0x7f27f819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27fca41640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 0x7f27f819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27fca41640 1 -- 172.21.15.110:0/1829892482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009560 con 0x7f27f80fea30 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27fca41640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a9c0 tx=0x7f27f40054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f4004aa0 con 0x7f27f80fea30 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27f4004c00 con 0x7f27f80fea30 2024-09-21T04:17:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f819acf0 con 0x7f27f80fea30 2024-09-21T04:17:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f4003a80 con 0x7f27f80fea30 2024-09-21T04:17:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.749+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f819b260 con 0x7f27f80fea30 2024-09-21T04:17:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.750+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f8107640 con 0x7f27f80fea30 2024-09-21T04:17:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.751+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f27f4003d40 con 0x7f27f80fea30 2024-09-21T04:17:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.752+0000 7f27edffb640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 0x7f27e007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:28.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.752+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f27f40bf740 con 0x7f27f80fea30 2024-09-21T04:17:28.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.752+0000 7f27effff640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 0x7f27e007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:28.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.752+0000 7f27effff640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 0x7f27e007d860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27f8068cc0 tx=0x7f27e800b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:28.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.756+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f27f40825e0 con 0x7f27f80fea30 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: mgrmap e38: smithi110.axmqzb(active, since 4s), standbys: smithi148.zjrahi 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/632457860' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:28.891 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:28 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: mgrmap e38: smithi110.axmqzb(active, since 4s), standbys: smithi148.zjrahi 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/632457860' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:28 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.962+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27f8104580 con 0x7f27e007b3a0 2024-09-21T04:17:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.964+0000 7f27edffb640 1 -- 172.21.15.110:0/1829892482 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f27f8104580 con 0x7f27e007b3a0 2024-09-21T04:17:28.966 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "3/23 daemons upgraded", 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:17:28.967 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 msgr2=0x7f27e007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 0x7f27e007d860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27f8068cc0 tx=0x7f27e800b390 comp rx=0 tx=0).stop 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 msgr2=0x7f27f8068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a9c0 tx=0x7f27f40054f0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 shutdown_connections 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f27e007b3a0 0x7f27e007d860 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27f80ffc30 0x7f27f819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 --2- 172.21.15.110:0/1829892482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27f80fea30 0x7f27f8068740 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:28.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 >> 172.21.15.110:0/1829892482 conn(0x7f27f80fa1c0 msgr2=0x7f27f8102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:28.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 shutdown_connections 2024-09-21T04:17:28.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:28.967+0000 7f27feca5640 1 -- 172.21.15.110:0/1829892482 wait complete. 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 -- 172.21.15.110:0/1574664006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 msgr2=0x7f3410102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 --2- 172.21.15.110:0/1574664006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410102a70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3400009c20 tx=0x7f340002a970 comp rx=0 tx=0).stop 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 -- 172.21.15.110:0/1574664006 shutdown_connections 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 --2- 172.21.15.110:0/1574664006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 0x7f34101053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 --2- 172.21.15.110:0/1574664006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410102a70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.054+0000 7f3416fa9640 1 -- 172.21.15.110:0/1574664006 >> 172.21.15.110:0/1574664006 conn(0x7f34100fa1c0 msgr2=0x7f34100fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 -- 172.21.15.110:0/1574664006 shutdown_connections 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 -- 172.21.15.110:0/1574664006 wait complete. 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 Processor -- start 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 -- start start 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 0x7f3410195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410196120 con 0x7f3410100680 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3416fa9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410196290 con 0x7f3410102fb0 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3414d45640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3414d45640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34532/0 (socket says 172.21.15.110:34532) 2024-09-21T04:17:29.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.055+0000 7f3414d45640 1 -- 172.21.15.110:0/2295487800 learned_addr learned my addr 172.21.15.110:0/2295487800 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:29.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f340ffff640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 0x7f3410195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:29.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3414d45640 1 -- 172.21.15.110:0/2295487800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 msgr2=0x7f3410195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:29.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3414d45640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 0x7f3410195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3414d45640 1 -- 172.21.15.110:0/2295487800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400009900 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3414d45640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3400009d40 tx=0x7f340002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3400030670 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3400030c90 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341019acd0 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340003a660 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.056+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341019b1c0 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.057+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410108750 con 0x7f3410100680 2024-09-21T04:17:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.058+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f340000a040 con 0x7f3410100680 2024-09-21T04:17:29.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.058+0000 7f340dffb640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 0x7f33fc07d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:29.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.058+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f340002c040 con 0x7f3410100680 2024-09-21T04:17:29.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.058+0000 7f340ffff640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 0x7f33fc07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:29.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.059+0000 7f340ffff640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 0x7f33fc07d860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33f8005d00 tx=0x7f33f8005c50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:29.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.063+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f3400082ae0 con 0x7f3410100680 2024-09-21T04:17:29.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.310+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f341019b6a0 con 0x7f3410100680 2024-09-21T04:17:29.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.311+0000 7f340dffb640 1 -- 172.21.15.110:0/2295487800 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3400085c00 con 0x7f3410100680 2024-09-21T04:17:29.313 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:17:29.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 msgr2=0x7f33fc07d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:29.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 0x7f33fc07d860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33f8005d00 tx=0x7f33f8005c50 comp rx=0 tx=0).stop 2024-09-21T04:17:29.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 msgr2=0x7f3410195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3400009d40 tx=0x7f340002f5d0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 shutdown_connections 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f33fc07b3a0 0x7f33fc07d860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3410102fb0 0x7f3410195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 --2- 172.21.15.110:0/2295487800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410100680 0x7f3410195610 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 >> 172.21.15.110:0/2295487800 conn(0x7f34100fa1c0 msgr2=0x7f34100fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 shutdown_connections 2024-09-21T04:17:29.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:29.313+0000 7f3416fa9640 1 -- 172.21.15.110:0/2295487800 wait complete. 2024-09-21T04:17:29.902 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:29.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/861463585' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:17:29.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: pgmap v6: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-21T04:17:29.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:29.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:29.903 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:29 smithi148 ceph-mon[34957]: from='client.? 172.21.15.110:0/2295487800' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/861463585' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: pgmap v6: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:29 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2295487800' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:17:30.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:30 smithi110 ceph-mon[92455]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:30 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:30 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:30 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:30 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:17:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:30 smithi148 ceph-mon[34957]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:30 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:30 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:30 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:30 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.112 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "quorum_status"} : dispatch 2024-09-21T04:17:32.113 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:31 smithi148 ceph-mon[34957]: pgmap v7: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:32.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: Updating smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "quorum_status"} : dispatch 2024-09-21T04:17:32.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:31 smithi110 ceph-mon[92455]: pgmap v7: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T04:17:33.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: Upgrade: Updating mon.smithi148 2024-09-21T04:17:33.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:33.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:17:33.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:17:33.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:33.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: Deploying daemon mon.smithi148 on smithi148 2024-09-21T04:17:33.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:33 smithi110 ceph-mon[92455]: pgmap v8: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: Upgrade: Updating mon.smithi148 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: Deploying daemon mon.smithi148 on smithi148 2024-09-21T04:17:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 ceph-mon[34957]: pgmap v8: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T04:17:34.100 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:33 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:17:34.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148[34953]: 2024-09-21T04:17:34.096+0000 7ff161c41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:34.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148[34953]: 2024-09-21T04:17:34.096+0000 7ff161c41640 -1 mon.smithi148@1(peon) e2 *** Got Signal Terminated *** 2024-09-21T04:17:34.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 podman[78534]: 2024-09-21 04:17:34.290723075 +0000 UTC m=+0.297070548 container died 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=reef-6a1bdf8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T04:17:34.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 podman[78534]: 2024-09-21 04:17:34.680176572 +0000 UTC m=+0.686524040 container cleanup 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, io.buildah.version=1.37.2, RELEASE=reef-6a1bdf8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T04:17:34.699 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 bash[78534]: ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148 2024-09-21T04:17:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:34 smithi148 podman[78546]: 2024-09-21 04:17:34.928732421 +0000 UTC m=+0.636529583 container remove 6e2e4585f6e18e437bb01d7ca0a14f71cea74f770b00af21bd271d5ce92b65a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-6a1bdf8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T04:17:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service: Deactivated successfully. 2024-09-21T04:17:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 systemd[1]: Stopped Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:17:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service: Consumed 6.446s CPU time. 2024-09-21T04:17:35.553 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 systemd[1]: Starting Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:17:35.553 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 podman[78675]: 2024-09-21 04:17:35.419559803 +0000 UTC m=+0.023398738 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:17:35.553 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 podman[78675]: 2024-09-21 04:17:35.550653575 +0000 UTC m=+0.154492499 container create 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 podman[78675]: 2024-09-21 04:17:35.728297105 +0000 UTC m=+0.332136029 container init 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 podman[78675]: 2024-09-21 04:17:35.731960915 +0000 UTC m=+0.335799838 container start 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev), process ceph-mon, pid 2 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: pidfile_write: ignore empty --pid-file 2024-09-21T04:17:35.831 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: load: jerasure load: lrc 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Git sha 0 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Compile date 2024-09-20 23:32:13 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: DB SUMMARY 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: DB Session ID: ZFHMOF5UMMQZZ4R5W2K8 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1400 Bytes 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi148/store.db dir, Total Num: 1, files: 000027.sst 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi148/store.db: 000025.log size: 3591692 ; 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:17:35.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.env: 0x5648ae46d120 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.info_log: 0x5648af18d8e0 2024-09-21T04:17:35.833 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.statistics: (nil) 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.use_fsync: 0 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:17:35.835 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.db_log_dir: 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.wal_dir: 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:17:35.836 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.write_buffer_manager: 0x5648af191b80 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:17:35.837 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.unordered_write: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.row_cache: None 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.wal_filter: None 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:17:35.838 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.wal_compression: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:17:35.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:17:35.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_open_files: -1 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Compression algorithms supported: 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kZSTD supported: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:17:35.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000010 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.merge_operator: 2024-09-21T04:17:35.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_filter: None 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5648af18d540) 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_type: 0 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_index_type: 0 2024-09-21T04:17:35.843 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_shortening: 1 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: checksum: 4 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: no_block_cache: 0 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache: 0x5648af181610 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_options: 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: capacity : 536870912 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_shard_bits : 4 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: strict_capacity_limit : 0 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_compressed: (nil) 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: persistent_cache: (nil) 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size: 4096 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size_deviation: 10 2024-09-21T04:17:35.844 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_restart_interval: 16 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_block_restart_interval: 1 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_block_size: 4096 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: partition_filters: 0 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: use_delta_encoding: 1 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: filter_policy: bloomfilter 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: whole_key_filtering: 1 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: verify_compression: 0 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: format_version: 5 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_index_compression: 1 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_align: 0 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_auto_readahead_size: 262144 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: prepopulate_block_cache: 0 2024-09-21T04:17:35.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression: NoCompression 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.num_levels: 7 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:17:35.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:17:35.847 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:17:35.848 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:17:35.849 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:17:35.850 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.ttl: 2592000 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:17:35.851 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12159, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d8f8c1a-7a78-495f-8fbb-f74794b1d5ac 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892255759523, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892255767822, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2256050, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12164, "largest_seqno": 12845, "table_properties": {"data_size": 2252058, "index_size": 2116, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 837, "raw_key_size": 8246, "raw_average_key_size": 25, "raw_value_size": 2244743, "raw_average_value_size": 6843, "num_data_blocks": 96, "num_entries": 328, "num_filter_entries": 328, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726892255, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d8f8c1a-7a78-495f-8fbb-f74794b1d5ac", "db_session_id": "ZFHMOF5UMMQZZ4R5W2K8", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892255767966, "job": 1, "event": "recovery_finished"} 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi148/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:17:35.852 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5648af1b0e00 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: DB pointer 0x5648af2b2000 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** DB Stats ** 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-21T04:17:35.853 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 2.15 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L6 1/0 7.02 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 2/0 9.17 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 594.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:17:35.854 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 163.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 163.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache BinnedLRUCache@0x5648af181610#2 capacity: 512.00 MB usage: 3.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.91 KB,0.000172853%) IndexBlock(1,2.33 KB,0.000444055%) Misc(1,0.00 KB,0%) 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:17:35.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: starting mon.smithi148 rank 1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi148 fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???) e2 preinit fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).mds e15 new map 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).mds e15 print_map 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e15 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: 1 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Filesystem 'cephfs' (1) 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: fs_name cephfs 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: epoch 15 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:17:35.856 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: tableserver 0 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: root 0 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_timeout 60 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_autoclose 300 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_file_size 1099511627776 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_xattr_size 65536 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: required_client_features {} 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure 0 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure_osd_epoch 46 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_mds 1 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: in 0 2024-09-21T04:17:35.857 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: up {0=14562} 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: failed 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: damaged 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: stopped 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_pools [3] 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_pool 2 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: inline_data disabled 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: balancer 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: bal_rank_mask -1 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: standby_count_wanted 1 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: qdb_cluster leader: 0 members: 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Standby daemons: 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 ceph-mon[78701]: mon.smithi148@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 bash[78675]: 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 2024-09-21T04:17:35.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:35 smithi148 systemd[1]: Started Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: mgrmap e38: smithi110.axmqzb(active, since 13s), standbys: smithi148.zjrahi 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: overall HEALTH_OK 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: mon.smithi110 calling monitor election 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: mon.smithi148 calling monitor election 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi110"} : dispatch 2024-09-21T04:17:37.528 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi148"} : dispatch 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: monmap epoch 3 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: last_changed 2024-09-21T04:17:36.221166+0000 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: created 2024-09-21T04:05:19.513898+0000 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: min_mon_release 19 (squid) 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: election_strategy: 1 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: mgrmap e38: smithi110.axmqzb(active, since 13s), standbys: smithi148.zjrahi 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: overall HEALTH_OK 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:37.529 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:37.530 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:37.530 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:37 smithi148 ceph-mon[78701]: pgmap v10: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T04:17:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: mgrmap e38: smithi110.axmqzb(active, since 13s), standbys: smithi148.zjrahi 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: overall HEALTH_OK 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: mon.smithi110 calling monitor election 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: mon.smithi148 calling monitor election 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi110"} : dispatch 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mon metadata", "id": "smithi148"} : dispatch 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: mon.smithi110 is new leader, mons smithi110,smithi148 in quorum (ranks 0,1) 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: monmap epoch 3 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: fsid 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: last_changed 2024-09-21T04:17:36.221166+0000 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: created 2024-09-21T04:05:19.513898+0000 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: min_mon_release 19 (squid) 2024-09-21T04:17:37.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: election_strategy: 1 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.smithi148 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: osdmap e49: 6 total, 6 up, 6 in 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: mgrmap e38: smithi110.axmqzb(active, since 13s), standbys: smithi148.zjrahi 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: overall HEALTH_OK 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:17:37.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:37 smithi110 ceph-mon[92455]: pgmap v10: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T04:17:38.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:38.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:39 smithi110 ceph-mon[92455]: pgmap v11: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T04:17:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:39 smithi148 ceph-mon[78701]: pgmap v11: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-21T04:17:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:39.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:40.771 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:40 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:40.772 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:40 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:40.783 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:40 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:40.783 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:40 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:42.078 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:41 smithi148 ceph-mon[78701]: pgmap v12: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:42.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:41 smithi110 ceph-mon[92455]: pgmap v12: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: Reconfiguring mon.smithi110 (monmap changed)... 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:17:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:43.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: Reconfiguring daemon mon.smithi110 on smithi110 2024-09-21T04:17:43.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:43 smithi110 ceph-mon[92455]: pgmap v13: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:43.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: Reconfiguring mon.smithi110 (monmap changed)... 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: Reconfiguring daemon mon.smithi110 on smithi110 2024-09-21T04:17:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:43 smithi148 ceph-mon[78701]: pgmap v13: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:45.254 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:45.254 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:45.254 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: Reconfiguring mgr.smithi110.axmqzb (monmap changed)... 2024-09-21T04:17:45.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T04:17:45.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T04:17:45.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:45.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: Reconfiguring daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:17:45.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:45 smithi110 ceph-mon[92455]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: Reconfiguring mgr.smithi110.axmqzb (monmap changed)... 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi110.axmqzb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: Reconfiguring daemon mgr.smithi110.axmqzb on smithi110 2024-09-21T04:17:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:45 smithi148 ceph-mon[78701]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: Reconfiguring ceph-exporter.smithi110 (monmap changed)... 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:46.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:17:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: Unable to update caps for client.ceph-exporter.smithi110 2024-09-21T04:17:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi110"} : dispatch 2024-09-21T04:17:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:46 smithi148 ceph-mon[78701]: Reconfiguring daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: Reconfiguring ceph-exporter.smithi110 (monmap changed)... 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: Unable to update caps for client.ceph-exporter.smithi110 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi110"} : dispatch 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:46.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:46 smithi110 ceph-mon[92455]: Reconfiguring daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:17:47.597 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:47 smithi110 ceph-mon[92455]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:47.598 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:47.598 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:47.598 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:17:47.598 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:47 smithi148 ceph-mon[78701]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:17:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: Reconfiguring crash.smithi110 (monmap changed)... 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: Reconfiguring daemon crash.smithi110 on smithi110 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:17:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: Reconfiguring crash.smithi110 (monmap changed)... 2024-09-21T04:17:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: Reconfiguring daemon crash.smithi110 on smithi110 2024-09-21T04:17:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:48.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:17:48.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:49.809 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:49 smithi110 ceph-mon[92455]: Reconfiguring osd.0 (monmap changed)... 2024-09-21T04:17:49.809 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:49 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.0 on smithi110 2024-09-21T04:17:49.809 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:49 smithi110 ceph-mon[92455]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:49 smithi148 ceph-mon[78701]: Reconfiguring osd.0 (monmap changed)... 2024-09-21T04:17:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:49 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.0 on smithi110 2024-09-21T04:17:49.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:49 smithi148 ceph-mon[78701]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: Reconfiguring osd.1 (monmap changed)... 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.1 on smithi110 2024-09-21T04:17:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:50 smithi110 ceph-mon[92455]: pgmap v17: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: Reconfiguring osd.1 (monmap changed)... 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.1 on smithi110 2024-09-21T04:17:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:50 smithi148 ceph-mon[78701]: pgmap v17: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: Reconfiguring osd.2 (monmap changed)... 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:52 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.2 on smithi110 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: Reconfiguring osd.2 (monmap changed)... 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:52 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.2 on smithi110 2024-09-21T04:17:54.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: pgmap v18: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:54.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:54.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:54.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: Reconfiguring mds.cephfs.smithi110.trhyxr (monmap changed)... 2024-09-21T04:17:54.135 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:17:54.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:54.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: Reconfiguring daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:17:54.136 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: pgmap v18: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: Reconfiguring mds.cephfs.smithi110.trhyxr (monmap changed)... 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: Reconfiguring daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:17:54.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: Reconfiguring mds.cephfs.smithi110.faxwju (monmap changed)... 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: Reconfiguring daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:17:55.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:55 smithi110 ceph-mon[92455]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:55.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: Reconfiguring mds.cephfs.smithi110.faxwju (monmap changed)... 2024-09-21T04:17:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:17:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: Reconfiguring daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:17:55.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:55 smithi148 ceph-mon[78701]: pgmap v19: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi148"} : dispatch 2024-09-21T04:17:57.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:56 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:17:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi148"} : dispatch 2024-09-21T04:17:57.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:56 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: Reconfiguring ceph-exporter.smithi148 (monmap changed)... 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: Unable to update caps for client.ceph-exporter.smithi148 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: Reconfiguring daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:58.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:17:58.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: Reconfiguring ceph-exporter.smithi148 (monmap changed)... 2024-09-21T04:17:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: Unable to update caps for client.ceph-exporter.smithi148 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: Reconfiguring daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: pgmap v20: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:17:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: Reconfiguring crash.smithi148 (monmap changed)... 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: Reconfiguring daemon crash.smithi148 on smithi148 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T04:17:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: Reconfiguring crash.smithi148 (monmap changed)... 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: Reconfiguring daemon crash.smithi148 on smithi148 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi148.zjrahi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T04:17:59.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 -- 172.21.15.110:0/774183564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 msgr2=0x7fc0e81000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 --2- 172.21.15.110:0/774183564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e81000b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009880 tx=0x7fc0d002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 -- 172.21.15.110:0/774183564 shutdown_connections 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 --2- 172.21.15.110:0/774183564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e81000b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 --2- 172.21.15.110:0/774183564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 -- 172.21.15.110:0/774183564 >> 172.21.15.110:0/774183564 conn(0x7fc0e80fa1c0 msgr2=0x7fc0e80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:59.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 -- 172.21.15.110:0/774183564 shutdown_connections 2024-09-21T04:17:59.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.407+0000 7fc0ed189640 1 -- 172.21.15.110:0/774183564 wait complete. 2024-09-21T04:17:59.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 Processor -- start 2024-09-21T04:17:59.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 -- start start 2024-09-21T04:17:59.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e8068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819a930 con 0x7fc0e80ffc30 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.408+0000 7fc0ed189640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819aa70 con 0x7fc0e80fea30 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60564/0 (socket says 172.21.15.110:60564) 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 -- 172.21.15.110:0/956884109 learned_addr learned my addr 172.21.15.110:0/956884109 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e6d9d640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e8068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 -- 172.21.15.110:0/956884109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 msgr2=0x7fc0e8068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e8068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 -- 172.21.15.110:0/956884109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0009560 con 0x7fc0e80ffc30 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e6d9d640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e8068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0e659c640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0004650 tx=0x7fc0d00055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0d0004aa0 con 0x7fc0e80ffc30 2024-09-21T04:17:59.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e819acc0 con 0x7fc0e80ffc30 2024-09-21T04:17:59.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0d0005d50 con 0x7fc0e80ffc30 2024-09-21T04:17:59.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0d00039e0 con 0x7fc0e80ffc30 2024-09-21T04:17:59.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.409+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e819b230 con 0x7fc0e80ffc30 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.411+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e8107640 con 0x7fc0e80ffc30 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.411+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fc0d0031c00 con 0x7fc0e80ffc30 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.412+0000 7fc0dffff640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 0x7fc0d407d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.412+0000 7fc0e6d9d640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 0x7fc0d407d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.412+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc0d00bf620 con 0x7fc0e80ffc30 2024-09-21T04:17:59.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.412+0000 7fc0e6d9d640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 0x7fc0d407d860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ffa90 tx=0x7fc0e0005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:59.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.416+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fc0d00824c0 con 0x7fc0e80ffc30 2024-09-21T04:17:59.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.614+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e8104580 con 0x7fc0d407b3a0 2024-09-21T04:17:59.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.615+0000 7fc0dffff640 1 -- 172.21.15.110:0/956884109 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc0e8104580 con 0x7fc0d407b3a0 2024-09-21T04:17:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 msgr2=0x7fc0d407d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 0x7fc0d407d860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ffa90 tx=0x7fc0e0005f90 comp rx=0 tx=0).stop 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 msgr2=0x7fc0e819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0004650 tx=0x7fc0d00055a0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 shutdown_connections 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc0d407b3a0 0x7fc0d407d860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e80ffc30 0x7fc0e819a360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 --2- 172.21.15.110:0/956884109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0e80fea30 0x7fc0e8068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.617+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 >> 172.21.15.110:0/956884109 conn(0x7fc0e80fa1c0 msgr2=0x7fc0e8102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.618+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 shutdown_connections 2024-09-21T04:17:59.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.618+0000 7fc0ed189640 1 -- 172.21.15.110:0/956884109 wait complete. 2024-09-21T04:17:59.629 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:17:59.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 -- 172.21.15.110:0/1638744571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d4100d50 msgr2=0x7f72d41011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 --2- 172.21.15.110:0/1638744571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d4100d50 0x7f72d41011d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009880 tx=0x7f72c402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 -- 172.21.15.110:0/1638744571 shutdown_connections 2024-09-21T04:17:59.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 --2- 172.21.15.110:0/1638744571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d4100d50 0x7f72d41011d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 --2- 172.21.15.110:0/1638744571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40ffb50 0x7f72d40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.700+0000 7f72dbd15640 1 -- 172.21.15.110:0/1638744571 >> 172.21.15.110:0/1638744571 conn(0x7f72d40fb2c0 msgr2=0x7f72d40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 -- 172.21.15.110:0/1638744571 shutdown_connections 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 -- 172.21.15.110:0/1638744571 wait complete. 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 Processor -- start 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 -- start start 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 0x7f72d4195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4196170 con 0x7f72d40ffb50 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72dbd15640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d41962e0 con 0x7f72d4100d50 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72d9ab1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72d9ab1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60586/0 (socket says 172.21.15.110:60586) 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.701+0000 7f72d9ab1640 1 -- 172.21.15.110:0/593949424 learned_addr learned my addr 172.21.15.110:0/593949424 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d92b0640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 0x7f72d4195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d9ab1640 1 -- 172.21.15.110:0/593949424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 msgr2=0x7f72d4195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d9ab1640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 0x7f72d4195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d9ab1640 1 -- 172.21.15.110:0/593949424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4009560 con 0x7f72d40ffb50 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d92b0640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 0x7f72d4195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72d9ab1640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c920 tx=0x7f72d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d00041f0 con 0x7f72d40ffb50 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d419ad30 con 0x7f72d40ffb50 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72d0007e70 con 0x7f72d40ffb50 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d00049d0 con 0x7f72d40ffb50 2024-09-21T04:17:59.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.702+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d419b2a0 con 0x7f72d40ffb50 2024-09-21T04:17:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.703+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d418f6f0 con 0x7f72d40ffb50 2024-09-21T04:17:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.704+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f72d0004350 con 0x7f72d40ffb50 2024-09-21T04:17:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.704+0000 7f72caffd640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 0x7f72c007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.704+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f72d009e370 con 0x7f72d40ffb50 2024-09-21T04:17:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.704+0000 7f72d92b0640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 0x7f72c007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.705+0000 7f72d92b0640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 0x7f72c007d860 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72c4031540 tx=0x7f72c4005de0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:59.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.709+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f72d0061160 con 0x7f72d40ffb50 2024-09-21T04:17:59.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.902+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d402c4f0 con 0x7f72c007b3a0 2024-09-21T04:17:59.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.903+0000 7f72caffd640 1 -- 172.21.15.110:0/593949424 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f72d402c4f0 con 0x7f72c007b3a0 2024-09-21T04:17:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 msgr2=0x7f72c007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 0x7f72c007d860 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72c4031540 tx=0x7f72c4005de0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 msgr2=0x7f72d4195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c920 tx=0x7f72d000cde0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 shutdown_connections 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.905+0000 7f72dbd15640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f72c007b3a0 0x7f72c007d860 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.906+0000 7f72dbd15640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d4100d50 0x7f72d4195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.906+0000 7f72dbd15640 1 --2- 172.21.15.110:0/593949424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d40ffb50 0x7f72d4195660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.906+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 >> 172.21.15.110:0/593949424 conn(0x7f72d40fb2c0 msgr2=0x7f72d4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.906+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 shutdown_connections 2024-09-21T04:17:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.906+0000 7f72dbd15640 1 -- 172.21.15.110:0/593949424 wait complete. 2024-09-21T04:17:59.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.988+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2482992004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b940ffb70 msgr2=0x7f4b940fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.988+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2482992004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b940ffb70 0x7f4b940fff70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009880 tx=0x7f4b9002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2482992004 shutdown_connections 2024-09-21T04:17:59.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2482992004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b94100d70 0x7f4b941011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2482992004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b940ffb70 0x7f4b940fff70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2482992004 >> 172.21.15.110:0/2482992004 conn(0x7f4b940fb2e0 msgr2=0x7f4b940fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:17:59.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:59 smithi110 ceph-mon[92455]: Reconfiguring mgr.smithi148.zjrahi (monmap changed)... 2024-09-21T04:17:59.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:59 smithi110 ceph-mon[92455]: Reconfiguring daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:17:59.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:59 smithi110 ceph-mon[92455]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:17:59.993 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:17:59 smithi110 ceph-mon[92455]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2482992004 shutdown_connections 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2482992004 wait complete. 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 Processor -- start 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.989+0000 7f4b9c59e640 1 -- start start 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b9c59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ffb70 0x7f4b94195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b9c59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b9c59e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b94196180 con 0x7f4b94100d70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b9c59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b941962f0 con 0x7f4b940ffb70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60614/0 (socket says 172.21.15.110:60614) 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 -- 172.21.15.110:0/2772363604 learned_addr learned my addr 172.21.15.110:0/2772363604 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b9a33a640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ffb70 0x7f4b94195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 -- 172.21.15.110:0/2772363604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ffb70 msgr2=0x7f4b94195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ffb70 0x7f4b94195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 -- 172.21.15.110:0/2772363604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009560 con 0x7f4b94100d70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.990+0000 7f4b99b39640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400e9a0 tx=0x7f4b8400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.991+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b8400cd50 con 0x7f4b94100d70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.991+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9419ad90 con 0x7f4b94100d70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.991+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b8400ceb0 con 0x7f4b94100d70 2024-09-21T04:17:59.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.991+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b84010660 con 0x7f4b94100d70 2024-09-21T04:17:59.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.991+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9419b330 con 0x7f4b94100d70 2024-09-21T04:17:59.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.992+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b940fff70 con 0x7f4b94100d70 2024-09-21T04:17:59.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.993+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f4b840048a0 con 0x7f4b94100d70 2024-09-21T04:17:59.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.993+0000 7f4b8b7fe640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 0x7f4b8007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:17:59.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.993+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4b84014070 con 0x7f4b94100d70 2024-09-21T04:17:59.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.993+0000 7f4b9a33a640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 0x7f4b8007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:17:59.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.994+0000 7f4b9a33a640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 0x7f4b8007d860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009230 tx=0x7f4b900314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:17:59.997+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4b84065010 con 0x7f4b94100d70 2024-09-21T04:18:00.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.183+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b9402c4d0 con 0x7f4b8007b3a0 2024-09-21T04:18:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:59 smithi148 ceph-mon[78701]: Reconfiguring mgr.smithi148.zjrahi (monmap changed)... 2024-09-21T04:18:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:59 smithi148 ceph-mon[78701]: Reconfiguring daemon mgr.smithi148.zjrahi on smithi148 2024-09-21T04:18:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:59 smithi148 ceph-mon[78701]: pgmap v21: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:17:59 smithi148 ceph-mon[78701]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.193+0000 7f4b8b7fe640 1 -- 172.21.15.110:0/2772363604 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4b9402c4d0 con 0x7f4b8007b3a0 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (9m) 31s ago 11m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (2m) 31s ago 11m 7612k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (2m) 21s ago 10m 7314k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (11m) 31s ago 11m 6899k - 18.2.4-871-g6a1bdf85 0adab276c97e d3fb28644b0d 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (10m) 21s ago 10m 6891k - 18.2.4-871-g6a1bdf85 0adab276c97e 38defad59a1d 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (9m) 31s ago 10m 87.5M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:18:00.196 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (5m) 31s ago 5m 155M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (5m) 31s ago 5m 15.0M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (5m) 21s ago 5m 16.7M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (5m) 21s ago 5m 18.5M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (2m) 31s ago 12m 486M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (56s) 21s ago 10m 452M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (44s) 31s ago 12m 27.7M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (24s) 21s ago 10m 21.3M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (11m) 31s ago 11m 17.5M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (10m) 21s ago 10m 15.1M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (8m) 31s ago 8m 287M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (8m) 31s ago 8m 276M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (7m) 31s ago 7m 228M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:18:00.197 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (7m) 21s ago 7m 325M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:18:00.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (7m) 21s ago 7m 259M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:18:00.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (6m) 21s ago 6m 253M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:18:00.198 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (61s) 31s ago 10m 42.4M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:18:00.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 msgr2=0x7f4b8007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 0x7f4b8007d860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009230 tx=0x7f4b900314e0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 msgr2=0x7f4b94195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400e9a0 tx=0x7f4b8400eda0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 shutdown_connections 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b8007b3a0 0x7f4b8007d860 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b94100d70 0x7f4b94195bb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 --2- 172.21.15.110:0/2772363604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ffb70 0x7f4b94195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.195+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 >> 172.21.15.110:0/2772363604 conn(0x7f4b940fb2e0 msgr2=0x7f4b941044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.196+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 shutdown_connections 2024-09-21T04:18:00.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.196+0000 7f4b9c59e640 1 -- 172.21.15.110:0/2772363604 wait complete. 2024-09-21T04:18:00.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 -- 172.21.15.110:0/976379927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00102f60 msgr2=0x7fda00105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 --2- 172.21.15.110:0/976379927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00102f60 0x7fda00105350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00098b0 tx=0x7fd9f002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 -- 172.21.15.110:0/976379927 shutdown_connections 2024-09-21T04:18:00.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 --2- 172.21.15.110:0/976379927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00102f60 0x7fda00105350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 --2- 172.21.15.110:0/976379927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00100630 0x7fda00102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 -- 172.21.15.110:0/976379927 >> 172.21.15.110:0/976379927 conn(0x7fda000fa230 msgr2=0x7fda000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 -- 172.21.15.110:0/976379927 shutdown_connections 2024-09-21T04:18:00.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.276+0000 7fda07d47640 1 -- 172.21.15.110:0/976379927 wait complete. 2024-09-21T04:18:00.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 Processor -- start 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 -- start start 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00102f60 0x7fda000fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000feb20 con 0x7fda00100630 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda07d47640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000fec90 con 0x7fda00102f60 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda05ae3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda05ae3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60628/0 (socket says 172.21.15.110:60628) 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.277+0000 7fda05ae3640 1 -- 172.21.15.110:0/4109734229 learned_addr learned my addr 172.21.15.110:0/4109734229 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda052e2640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00102f60 0x7fda000fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda05ae3640 1 -- 172.21.15.110:0/4109734229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00102f60 msgr2=0x7fda000fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda05ae3640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00102f60 0x7fda000fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda05ae3640 1 -- 172.21.15.110:0/4109734229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0009560 con 0x7fda00100630 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda05ae3640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e970 tx=0x7fd9fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc00cc90 con 0x7fda00100630 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda000fef70 con 0x7fda00100630 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9fc00cdf0 con 0x7fda00100630 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc010430 con 0x7fda00100630 2024-09-21T04:18:00.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.278+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0019d6c0 con 0x7fda00100630 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.280+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00100a10 con 0x7fda00100630 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.280+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fd9fc0046c0 con 0x7fda00100630 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.280+0000 7fd9f6ffd640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 0x7fd9ec07d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.281+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd9fc014070 con 0x7fda00100630 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.281+0000 7fda052e2640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 0x7fd9ec07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.281+0000 7fda052e2640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 0x7fd9ec07d860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009880 tx=0x7fd9f00314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.285+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd9fc061910 con 0x7fda00100630 2024-09-21T04:18:00.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.539+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda0019da90 con 0x7fda00100630 2024-09-21T04:18:00.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.540+0000 7fd9f6ffd640 1 -- 172.21.15.110:0/4109734229 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7fd9fc064a30 con 0x7fda00100630 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:00.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 10, 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:18:00.544 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 msgr2=0x7fd9ec07d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 0x7fd9ec07d860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009880 tx=0x7fd9f00314e0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 msgr2=0x7fda000fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e970 tx=0x7fd9fc00ed70 comp rx=0 tx=0).stop 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 shutdown_connections 2024-09-21T04:18:00.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b3a0 0x7fd9ec07d860 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00102f60 0x7fda000fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 --2- 172.21.15.110:0/4109734229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda00100630 0x7fda000fff90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 >> 172.21.15.110:0/4109734229 conn(0x7fda000fa230 msgr2=0x7fda000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 shutdown_connections 2024-09-21T04:18:00.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.542+0000 7fda07d47640 1 -- 172.21.15.110:0/4109734229 wait complete. 2024-09-21T04:18:00.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 -- 172.21.15.110:0/2234246536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800feaa0 msgr2=0x7fcf800feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 --2- 172.21.15.110:0/2234246536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800feaa0 0x7fcf800feea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009880 tx=0x7fcf7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 -- 172.21.15.110:0/2234246536 shutdown_connections 2024-09-21T04:18:00.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 --2- 172.21.15.110:0/2234246536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800ffca0 0x7fcf80100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 --2- 172.21.15.110:0/2234246536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800feaa0 0x7fcf800feea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 -- 172.21.15.110:0/2234246536 >> 172.21.15.110:0/2234246536 conn(0x7fcf800fa230 msgr2=0x7fcf800fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 -- 172.21.15.110:0/2234246536 shutdown_connections 2024-09-21T04:18:00.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.624+0000 7fcf875db640 1 -- 172.21.15.110:0/2234246536 wait complete. 2024-09-21T04:18:00.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 Processor -- start 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 -- start start 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800feaa0 0x7fcf80195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80196120 con 0x7fcf800ffca0 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf875db640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80196290 con 0x7fcf800feaa0 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60644/0 (socket says 172.21.15.110:60644) 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 -- 172.21.15.110:0/2547358552 learned_addr learned my addr 172.21.15.110:0/2547358552 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf85377640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800feaa0 0x7fcf80195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 -- 172.21.15.110:0/2547358552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800feaa0 msgr2=0x7fcf80195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800feaa0 0x7fcf80195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.625+0000 7fcf84b76640 1 -- 172.21.15.110:0/2547358552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c009560 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf84b76640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000d720 tx=0x7fcf7000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf700051f0 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf70009ce0 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8019ad30 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf70010460 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.626+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8019b300 con 0x7fcf800ffca0 2024-09-21T04:18:00.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.627+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf801087b0 con 0x7fcf800ffca0 2024-09-21T04:18:00.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.627+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fcf70005350 con 0x7fcf800ffca0 2024-09-21T04:18:00.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.628+0000 7fcf767fc640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 0x7fcf6c07d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.628+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcf7009dfa0 con 0x7fcf800ffca0 2024-09-21T04:18:00.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.628+0000 7fcf85377640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 0x7fcf6c07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.628+0000 7fcf85377640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 0x7fcf6c07d860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009230 tx=0x7fcf7c0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.632+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fcf700d5710 con 0x7fcf800ffca0 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.850+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcf80108940 con 0x7fcf800ffca0 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.850+0000 7fcf767fc640 1 -- 172.21.15.110:0/2547358552 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7fcf70060e60 con 0x7fcf800ffca0 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:18:00.853 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:18:00.854 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:18:00.855 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:00.856 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 msgr2=0x7fcf6c07d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 0x7fcf6c07d860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009230 tx=0x7fcf7c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 msgr2=0x7fcf80195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000d720 tx=0x7fcf7000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 shutdown_connections 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcf6c07b3a0 0x7fcf6c07d860 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf800ffca0 0x7fcf80195b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 --2- 172.21.15.110:0/2547358552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf800feaa0 0x7fcf80195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 >> 172.21.15.110:0/2547358552 conn(0x7fcf800fa230 msgr2=0x7fcf800fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 shutdown_connections 2024-09-21T04:18:00.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.852+0000 7fcf875db640 1 -- 172.21.15.110:0/2547358552 wait complete. 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 -- 172.21.15.110:0/3808126004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 msgr2=0x7f4ef4105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 --2- 172.21.15.110:0/3808126004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4105330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40098b0 tx=0x7f4ee402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 -- 172.21.15.110:0/3808126004 shutdown_connections 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 --2- 172.21.15.110:0/3808126004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4105330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 --2- 172.21.15.110:0/3808126004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 0x7f4ef4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 -- 172.21.15.110:0/3808126004 >> 172.21.15.110:0/3808126004 conn(0x7f4ef40fa230 msgr2=0x7f4ef40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 -- 172.21.15.110:0/3808126004 shutdown_connections 2024-09-21T04:18:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 -- 172.21.15.110:0/3808126004 wait complete. 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.935+0000 7f4ef9188640 1 Processor -- start 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef9188640 1 -- start start 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef9188640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 0x7f4ef4195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef9188640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef9188640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4196160 con 0x7f4ef4102f40 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef9188640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41962d0 con 0x7f4ef4100610 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60666/0 (socket says 172.21.15.110:60666) 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 -- 172.21.15.110:0/406482370 learned_addr learned my addr 172.21.15.110:0/406482370 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef2d9d640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 0x7f4ef4195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 -- 172.21.15.110:0/406482370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 msgr2=0x7f4ef4195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 0x7f4ef4195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 -- 172.21.15.110:0/406482370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009560 con 0x7f4ef4102f40 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.936+0000 7f4ef259c640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4004650 tx=0x7f4ee40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.937+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee4004aa0 con 0x7f4ef4102f40 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.937+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ee4005d50 con 0x7f4ef4102f40 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.937+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef419ad10 con 0x7f4ef4102f40 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.937+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee40039e0 con 0x7f4ef4102f40 2024-09-21T04:18:00.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.937+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef419b280 con 0x7f4ef4102f40 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.938+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef4108790 con 0x7f4ef4102f40 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.938+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f4ee4003c50 con 0x7f4ef4102f40 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.939+0000 7f4eebfff640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 0x7f4ee007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.939+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4ee40bf560 con 0x7f4ef4102f40 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.939+0000 7f4ef2d9d640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 0x7f4ee007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.939+0000 7f4ef2d9d640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 0x7f4ee007d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009bd0 tx=0x7f4edc005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:00.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:00.943+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4ee4082480 con 0x7f4ef4102f40 2024-09-21T04:18:01.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.138+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ef41039a0 con 0x7f4ee007b3a0 2024-09-21T04:18:01.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: Reconfiguring mon.smithi148 (monmap changed)... 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: Reconfiguring daemon mon.smithi148 on smithi148 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4109734229' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:01.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:00 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2547358552' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.139+0000 7f4eebfff640 1 -- 172.21.15.110:0/406482370 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f4ef41039a0 con 0x7f4ee007b3a0 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:18:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "4/23 daemons upgraded", 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mon daemons", 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:18:01.144 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 msgr2=0x7f4ee007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 0x7f4ee007d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009bd0 tx=0x7f4edc005f90 comp rx=0 tx=0).stop 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 msgr2=0x7f4ef4195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4004650 tx=0x7f4ee40055a0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 shutdown_connections 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4ee007b3a0 0x7f4ee007d860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ef4102f40 0x7f4ef4195b90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 --2- 172.21.15.110:0/406482370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4100610 0x7f4ef4195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 >> 172.21.15.110:0/406482370 conn(0x7f4ef40fa230 msgr2=0x7f4ef40fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:01.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 shutdown_connections 2024-09-21T04:18:01.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.142+0000 7f4ef9188640 1 -- 172.21.15.110:0/406482370 wait complete. 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- 172.21.15.110:0/1742064607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe624100d90 msgr2=0x7fe624101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 --2- 172.21.15.110:0/1742064607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe624100d90 0x7fe624101210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe620009880 tx=0x7fe62002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- 172.21.15.110:0/1742064607 shutdown_connections 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 --2- 172.21.15.110:0/1742064607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe624100d90 0x7fe624101210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 --2- 172.21.15.110:0/1742064607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6240ffb90 0x7fe6240fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- 172.21.15.110:0/1742064607 >> 172.21.15.110:0/1742064607 conn(0x7fe6240fb320 msgr2=0x7fe6240fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- 172.21.15.110:0/1742064607 shutdown_connections 2024-09-21T04:18:01.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- 172.21.15.110:0/1742064607 wait complete. 2024-09-21T04:18:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 Processor -- start 2024-09-21T04:18:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.223+0000 7fe62beb6640 1 -- start start 2024-09-21T04:18:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 0x7fe6240728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:01.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624072de0 con 0x7fe6240ffb90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624072f50 con 0x7fe624100d90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60682/0 (socket says 172.21.15.110:60682) 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 -- 172.21.15.110:0/2483785263 learned_addr learned my addr 172.21.15.110:0/2483785263 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629451640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 0x7fe6240728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 -- 172.21.15.110:0/2483785263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 msgr2=0x7fe6240728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 0x7fe6240728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 -- 172.21.15.110:0/2483785263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620009560 con 0x7fe6240ffb90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629451640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 0x7fe6240728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe629c52640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe61400b3d0 tx=0x7fe61400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe614011800 con 0x7fe6240ffb90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe614011e20 con 0x7fe6240ffb90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe624073230 con 0x7fe6240ffb90 2024-09-21T04:18:01.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61400f510 con 0x7fe6240ffb90 2024-09-21T04:18:01.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.224+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62419d740 con 0x7fe6240ffb90 2024-09-21T04:18:01.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.225+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6241091f0 con 0x7fe6240ffb90 2024-09-21T04:18:01.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.226+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fe614010380 con 0x7fe6240ffb90 2024-09-21T04:18:01.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.227+0000 7fe61affd640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 0x7fe61007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:01.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.227+0000 7fe629451640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 0x7fe61007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:01.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.227+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe61409e180 con 0x7fe6240ffb90 2024-09-21T04:18:01.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.227+0000 7fe629451640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 0x7fe61007d860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe620000c00 tx=0x7fe6200314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:01.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.231+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe6241091f0 con 0x7fe6240ffb90 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: Reconfiguring mon.smithi148 (monmap changed)... 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: Reconfiguring daemon mon.smithi148 on smithi148 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4109734229' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: pgmap v22: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:01.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:00 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2547358552' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:18:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.483+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe624100060 con 0x7fe6240ffb90 2024-09-21T04:18:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.483+0000 7fe61affd640 1 -- 172.21.15.110:0/2483785263 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe614060f70 con 0x7fe6240ffb90 2024-09-21T04:18:01.486 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.485+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 msgr2=0x7fe61007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.485+0000 7fe62beb6640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 0x7fe61007d860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe620000c00 tx=0x7fe6200314e0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.485+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 msgr2=0x7fe624074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.485+0000 7fe62beb6640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe61400b3d0 tx=0x7fe61400b890 comp rx=0 tx=0).stop 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 shutdown_connections 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe61007b3a0 0x7fe61007d860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe624100d90 0x7fe6240728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 --2- 172.21.15.110:0/2483785263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6240ffb90 0x7fe624074250 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 >> 172.21.15.110:0/2483785263 conn(0x7fe6240fb320 msgr2=0x7fe624103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 shutdown_connections 2024-09-21T04:18:01.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:01.486+0000 7fe62beb6640 1 -- 172.21.15.110:0/2483785263 wait complete. 2024-09-21T04:18:02.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:02.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:02.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:02.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: Reconfiguring osd.3 (monmap changed)... 2024-09-21T04:18:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:18:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.3 on smithi148 2024-09-21T04:18:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:02 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2483785263' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: Reconfiguring osd.3 (monmap changed)... 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.3 on smithi148 2024-09-21T04:18:02.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:02 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2483785263' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: Reconfiguring osd.4 (monmap changed)... 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:03.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.4 on smithi148 2024-09-21T04:18:03.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:03 smithi110 ceph-mon[92455]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: Reconfiguring osd.4 (monmap changed)... 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.4 on smithi148 2024-09-21T04:18:03.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:03 smithi148 ceph-mon[78701]: pgmap v23: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: Reconfiguring osd.5 (monmap changed)... 2024-09-21T04:18:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:18:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: Reconfiguring daemon osd.5 on smithi148 2024-09-21T04:18:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:04 smithi148 ceph-mon[78701]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: Reconfiguring osd.5 (monmap changed)... 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: Reconfiguring daemon osd.5 on smithi148 2024-09-21T04:18:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:04 smithi110 ceph-mon[92455]: pgmap v24: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:06.607 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:06.608 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:06.608 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: Reconfiguring mds.cephfs.smithi148.tpueww (monmap changed)... 2024-09-21T04:18:06.608 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:18:06.608 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:06.608 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:06 smithi148 ceph-mon[78701]: Reconfiguring daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: Reconfiguring mds.cephfs.smithi148.tpueww (monmap changed)... 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:06.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:06 smithi110 ceph-mon[92455]: Reconfiguring daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: Reconfiguring mds.cephfs.smithi148.fpatxi (monmap changed)... 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: Reconfiguring daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:18:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:07 smithi110 ceph-mon[92455]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: Reconfiguring mds.cephfs.smithi148.fpatxi (monmap changed)... 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: Reconfiguring daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:18:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:07 smithi148 ceph-mon[78701]: pgmap v25: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all mon 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"} : dispatch 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]': finished 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"} : dispatch 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-09-21T04:18:09.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all mon 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"} : dispatch 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]': finished 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"} : dispatch 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-09-21T04:18:09.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:09 smithi110 ceph-mon[92455]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:18:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:09 smithi148 ceph-mon[78701]: pgmap v26: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:18:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:10.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:10 smithi110 ceph-mon[92455]: Upgrade: Updating crash.smithi110 (1/2) 2024-09-21T04:18:10.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:10 smithi110 ceph-mon[92455]: Deploying daemon crash.smithi110 on smithi110 2024-09-21T04:18:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:10 smithi148 ceph-mon[78701]: Upgrade: Updating crash.smithi110 (1/2) 2024-09-21T04:18:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:10 smithi148 ceph-mon[78701]: Deploying daemon crash.smithi110 on smithi110 2024-09-21T04:18:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:11 smithi110 ceph-mon[92455]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:12.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:11 smithi148 ceph-mon[78701]: pgmap v27: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:14.109 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:13 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:14.109 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:13 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:14.109 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:13 smithi148 ceph-mon[78701]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:14.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:13 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:14.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:13 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:14.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:13 smithi110 ceph-mon[92455]: pgmap v28: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: Upgrade: Updating crash.smithi148 (2/2) 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: Deploying daemon crash.smithi148 on smithi148 2024-09-21T04:18:15.141 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:14 smithi110 ceph-mon[92455]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:15.251 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: Upgrade: Updating crash.smithi148 (2/2) 2024-09-21T04:18:15.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:15.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:18:15.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:15.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: Deploying daemon crash.smithi148 on smithi148 2024-09-21T04:18:15.252 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:14 smithi148 ceph-mon[78701]: pgmap v29: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:16 smithi148 ceph-mon[78701]: pgmap v30: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:17.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:16 smithi110 ceph-mon[92455]: pgmap v30: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:18.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:18.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:18.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:19.616 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:19 smithi148 ceph-mon[78701]: pgmap v31: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:19.628 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:19 smithi110 ceph-mon[92455]: pgmap v31: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:20 smithi110 ceph-mon[92455]: pgmap v32: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:21.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:20 smithi148 ceph-mon[78701]: pgmap v32: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: pgmap v33: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:23.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:23.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.487 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.487 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.487 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:23.487 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: pgmap v33: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:23.487 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:24.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:24.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:24.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:24.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:25.767 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:25 smithi110 ceph-mon[92455]: pgmap v34: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:25.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:25 smithi148 ceph-mon[78701]: pgmap v34: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:27.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all crash 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]': finished 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: Upgrade: osd.0 is safe to restart 2024-09-21T04:18:27.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:27 smithi110 ceph-mon[92455]: pgmap v35: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all crash 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"} : dispatch 2024-09-21T04:18:27.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]': finished 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"} : dispatch 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: Upgrade: osd.0 is safe to restart 2024-09-21T04:18:27.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:27 smithi148 ceph-mon[78701]: pgmap v35: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:28 smithi110 ceph-mon[92455]: Upgrade: Updating osd.0 2024-09-21T04:18:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:28 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:28 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:18:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:28 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:28 smithi110 ceph-mon[92455]: Deploying daemon osd.0 on smithi110 2024-09-21T04:18:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:28 smithi148 ceph-mon[78701]: Upgrade: Updating osd.0 2024-09-21T04:18:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:28 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:28 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:18:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:28 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:28 smithi148 ceph-mon[78701]: Deploying daemon osd.0 on smithi110 2024-09-21T04:18:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:29 smithi110 ceph-mon[92455]: pgmap v36: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:29.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:29 smithi148 ceph-mon[78701]: pgmap v36: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:31.047 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:30 smithi110 ceph-mon[92455]: pgmap v37: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:31.047 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:30 smithi110 systemd[1]: Stopping Ceph osd.0 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:18:31.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:31 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:18:31.044+0000 7f88167e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:18:31.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:31 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:18:31.044+0000 7f88167e0640 -1 osd.0 49 *** Got signal Terminated *** 2024-09-21T04:18:31.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:31 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[49607]: 2024-09-21T04:18:31.044+0000 7f88167e0640 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:18:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:30 smithi148 ceph-mon[78701]: pgmap v37: 65 pgs: 65 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 -- 172.21.15.110:0/3304455333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800fea80 msgr2=0x7fe0800fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3304455333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800fea80 0x7fe0800fee80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009880 tx=0x7fe07c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 -- 172.21.15.110:0/3304455333 shutdown_connections 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3304455333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800ffc80 0x7fe080100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3304455333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800fea80 0x7fe0800fee80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 -- 172.21.15.110:0/3304455333 >> 172.21.15.110:0/3304455333 conn(0x7fe0800fa230 msgr2=0x7fe0800fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 -- 172.21.15.110:0/3304455333 shutdown_connections 2024-09-21T04:18:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.571+0000 7fe0872d4640 1 -- 172.21.15.110:0/3304455333 wait complete. 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 Processor -- start 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 -- start start 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800fea80 0x7fe080102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080101ab0 con 0x7fe0800ffc80 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe0872d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080101c20 con 0x7fe0800fea80 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe08486f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe08486f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50046/0 (socket says 172.21.15.110:50046) 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe08486f640 1 -- 172.21.15.110:0/3266564980 learned_addr learned my addr 172.21.15.110:0/3266564980 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.572+0000 7fe085070640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800fea80 0x7fe080102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe08486f640 1 -- 172.21.15.110:0/3266564980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800fea80 msgr2=0x7fe080102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe08486f640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800fea80 0x7fe080102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe08486f640 1 -- 172.21.15.110:0/3266564980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c009560 con 0x7fe0800ffc80 2024-09-21T04:18:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe08486f640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe07000d6f0 tx=0x7fe07000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe070009a30 con 0x7fe0800ffc80 2024-09-21T04:18:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0700054e0 con 0x7fe0800ffc80 2024-09-21T04:18:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe070010460 con 0x7fe0800ffc80 2024-09-21T04:18:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe080101f00 con 0x7fe0800ffc80 2024-09-21T04:18:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.573+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08019d6c0 con 0x7fe0800ffc80 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.574+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0800fee80 con 0x7fe0800ffc80 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.575+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fe070009b90 con 0x7fe0800ffc80 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.575+0000 7fe0767fc640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 0x7fe06c07d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.575+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe07009dfd0 con 0x7fe0800ffc80 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.575+0000 7fe085070640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 0x7fe06c07d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.576+0000 7fe085070640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 0x7fe06c07d850 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:31.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.580+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe070060ef0 con 0x7fe0800ffc80 2024-09-21T04:18:31.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.778+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe080103500 con 0x7fe06c07b390 2024-09-21T04:18:31.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.779+0000 7fe0767fc640 1 -- 172.21.15.110:0/3266564980 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe080103500 con 0x7fe06c07b390 2024-09-21T04:18:31.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.781+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 msgr2=0x7fe06c07d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.781+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 0x7fe06c07d850 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.781+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 msgr2=0x7fe080101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.781+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe07000d6f0 tx=0x7fe07000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 shutdown_connections 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe06c07b390 0x7fe06c07d850 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0800ffc80 0x7fe080101570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 --2- 172.21.15.110:0/3266564980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0800fea80 0x7fe080102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 >> 172.21.15.110:0/3266564980 conn(0x7fe0800fa230 msgr2=0x7fe0800fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 shutdown_connections 2024-09-21T04:18:31.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.782+0000 7fe0872d4640 1 -- 172.21.15.110:0/3266564980 wait complete. 2024-09-21T04:18:31.793 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:18:31.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 -- 172.21.15.110:0/3276830141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 msgr2=0x7f22d4105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 --2- 172.21.15.110:0/3276830141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4105330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22c80098b0 tx=0x7f22c802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 -- 172.21.15.110:0/3276830141 shutdown_connections 2024-09-21T04:18:31.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 --2- 172.21.15.110:0/3276830141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4105330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 --2- 172.21.15.110:0/3276830141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 0x7f22d4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 -- 172.21.15.110:0/3276830141 >> 172.21.15.110:0/3276830141 conn(0x7f22d40fa230 msgr2=0x7f22d40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:31.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 -- 172.21.15.110:0/3276830141 shutdown_connections 2024-09-21T04:18:31.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.864+0000 7f22d8aba640 1 -- 172.21.15.110:0/3276830141 wait complete. 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 Processor -- start 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 -- start start 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 0x7f22d4195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4196180 con 0x7f22d4102f40 2024-09-21T04:18:31.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d8aba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d41962f0 con 0x7f22d4100610 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50068/0 (socket says 172.21.15.110:50068) 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.865+0000 7f22d1d9b640 1 -- 172.21.15.110:0/433545155 learned_addr learned my addr 172.21.15.110:0/433545155 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d259c640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 0x7f22d4195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d1d9b640 1 -- 172.21.15.110:0/433545155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 msgr2=0x7f22d4195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d1d9b640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 0x7f22d4195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d1d9b640 1 -- 172.21.15.110:0/433545155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8009560 con 0x7f22d4102f40 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d1d9b640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22c80055a0 tx=0x7f22c80055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c8004aa0 con 0x7f22d4102f40 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d419ad30 con 0x7f22d4102f40 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22c8004c00 con 0x7f22d4102f40 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c8003b50 con 0x7f22d4102f40 2024-09-21T04:18:31.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.866+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d419b2a0 con 0x7f22d4102f40 2024-09-21T04:18:31.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.867+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4108790 con 0x7f22d4102f40 2024-09-21T04:18:31.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.868+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f22c8003cb0 con 0x7f22d4102f40 2024-09-21T04:18:31.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.868+0000 7f22c37fe640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 0x7f22bc07d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:31.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.868+0000 7f22d259c640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 0x7f22bc07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:31.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.868+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f22c80bf800 con 0x7f22d4102f40 2024-09-21T04:18:31.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.869+0000 7f22d259c640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 0x7f22bc07d860 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005fd0 tx=0x7f22c4005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:31.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:31.873+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f22c80825f0 con 0x7f22d4102f40 2024-09-21T04:18:32.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.078+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22d4103960 con 0x7f22bc07b3a0 2024-09-21T04:18:32.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.079+0000 7f22c37fe640 1 -- 172.21.15.110:0/433545155 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f22d4103960 con 0x7f22bc07b3a0 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 msgr2=0x7f22bc07d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 0x7f22bc07d860 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005fd0 tx=0x7f22c4005850 comp rx=0 tx=0).stop 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 msgr2=0x7f22d4195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22c80055a0 tx=0x7f22c80055d0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 shutdown_connections 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f22bc07b3a0 0x7f22bc07d860 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d4102f40 0x7f22d4195bb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 --2- 172.21.15.110:0/433545155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22d4100610 0x7f22d4195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 >> 172.21.15.110:0/433545155 conn(0x7f22d40fa230 msgr2=0x7f22d40fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 shutdown_connections 2024-09-21T04:18:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.082+0000 7f22d8aba640 1 -- 172.21.15.110:0/433545155 wait complete. 2024-09-21T04:18:32.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 -- 172.21.15.110:0/344249133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 msgr2=0x7fcbd4100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/344249133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4100120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009880 tx=0x7fcbc802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.165 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:31 smithi110 ceph-mon[92455]: osd.0 marked itself down and dead 2024-09-21T04:18:32.165 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:31 smithi110 ceph-mon[92455]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:32.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 -- 172.21.15.110:0/344249133 shutdown_connections 2024-09-21T04:18:32.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/344249133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4100120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/344249133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 0x7fcbd40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 -- 172.21.15.110:0/344249133 >> 172.21.15.110:0/344249133 conn(0x7fcbd40fa230 msgr2=0x7fcbd40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 -- 172.21.15.110:0/344249133 shutdown_connections 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.162+0000 7fcbd8f86640 1 -- 172.21.15.110:0/344249133 wait complete. 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 Processor -- start 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 -- start start 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 0x7fcbd41944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4194f70 con 0x7fcbd40ffca0 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.163+0000 7fcbd8f86640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd41950e0 con 0x7fcbd40feaa0 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36422/0 (socket says 172.21.15.110:36422) 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 -- 172.21.15.110:0/1121402890 learned_addr learned my addr 172.21.15.110:0/1121402890 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:32.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd259c640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 0x7fcbd41944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 -- 172.21.15.110:0/1121402890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 msgr2=0x7fcbd41944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 0x7fcbd41944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 -- 172.21.15.110:0/1121402890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc8009560 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd1d9b640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005610 tx=0x7fcbc8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc8004ba0 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbc8004d00 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd4199b60 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.164+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc8003bd0 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.165+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd419a080 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.166+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd4107690 con 0x7fcbd40ffca0 2024-09-21T04:18:32.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.166+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7fcbc803c470 con 0x7fcbd40ffca0 2024-09-21T04:18:32.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.167+0000 7fcbc37fe640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 0x7fcbbc07d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.167+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcbc80bf7a0 con 0x7fcbd40ffca0 2024-09-21T04:18:32.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.167+0000 7fcbd259c640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 0x7fcbbc07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.167+0000 7fcbd259c640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 0x7fcbbc07d860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40ffb00 tx=0x7fcbc4005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.171+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fcbc80826c0 con 0x7fcbd40ffca0 2024-09-21T04:18:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.368+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbd4104450 con 0x7fcbbc07b3a0 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.377+0000 7fcbc37fe640 1 -- 172.21.15.110:0/1121402890 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcbd4104450 con 0x7fcbbc07b3a0 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (9m) 10s ago 11m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (2m) 10s ago 12m 7688k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (2m) 12s ago 11m 7369k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (20s) 10s ago 11m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:18:32.380 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (15s) 12s ago 11m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (9m) 10s ago 11m 84.9M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (5m) 10s ago 5m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (5m) 10s ago 5m 15.7M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (5m) 12s ago 5m 17.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (5m) 12s ago 5m 18.8M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (2m) 10s ago 13m 507M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (88s) 12s ago 10m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (76s) 10s ago 13m 41.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (56s) 12s ago 10m 34.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (11m) 10s ago 11m 16.9M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (10m) 12s ago 10m 15.3M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (9m) 10s ago 9m 285M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 53d83efb5956 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (8m) 10s ago 8m 278M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:18:32.381 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (8m) 10s ago 8m 228M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:18:32.382 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (8m) 12s ago 8m 326M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:18:32.382 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (7m) 12s ago 7m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:18:32.382 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (7m) 12s ago 7m 254M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:18:32.382 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (93s) 10s ago 11m 50.8M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:18:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 msgr2=0x7fcbbc07d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 0x7fcbbc07d860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40ffb00 tx=0x7fcbc4005f90 comp rx=0 tx=0).stop 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 msgr2=0x7fcbd4194a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005610 tx=0x7fcbc8005640 comp rx=0 tx=0).stop 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 shutdown_connections 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fcbbc07b3a0 0x7fcbbc07d860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd40ffca0 0x7fcbd4194a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 --2- 172.21.15.110:0/1121402890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd40feaa0 0x7fcbd41944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 >> 172.21.15.110:0/1121402890 conn(0x7fcbd40fa230 msgr2=0x7fcbd40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 shutdown_connections 2024-09-21T04:18:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.379+0000 7fcbd8f86640 1 -- 172.21.15.110:0/1121402890 wait complete. 2024-09-21T04:18:32.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:32 smithi110 podman[99636]: 2024-09-21 04:18:32.357974576 +0000 UTC m=+1.432404305 container died 53d83efb595606db849f5d509b1cd04e7ad8c61a5fd785de5a7ecbb6f04bcb0a (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=reef-6a1bdf8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T04:18:32.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:31 smithi148 ceph-mon[78701]: osd.0 marked itself down and dead 2024-09-21T04:18:32.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:31 smithi148 ceph-mon[78701]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 -- 172.21.15.110:0/2398677985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 msgr2=0x7f47740ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 --2- 172.21.15.110:0/2398677985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f47740ffc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4764009880 tx=0x7f476402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 -- 172.21.15.110:0/2398677985 shutdown_connections 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 --2- 172.21.15.110:0/2398677985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f47740ffc40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 --2- 172.21.15.110:0/2398677985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f4774101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 -- 172.21.15.110:0/2398677985 >> 172.21.15.110:0/2398677985 conn(0x7f47740fb330 msgr2=0x7f47740fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 -- 172.21.15.110:0/2398677985 shutdown_connections 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.462+0000 7f477b47c640 1 -- 172.21.15.110:0/2398677985 wait complete. 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 Processor -- start 2024-09-21T04:18:32.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 -- start start 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f47740728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774072de0 con 0x7f47740ff7c0 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f477b47c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774072f50 con 0x7f4774101280 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f4779218640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f4779218640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36426/0 (socket says 172.21.15.110:36426) 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f4779218640 1 -- 172.21.15.110:0/3265877393 learned_addr learned my addr 172.21.15.110:0/3265877393 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.463+0000 7f4778a17640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f47740728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f4779218640 1 -- 172.21.15.110:0/3265877393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 msgr2=0x7f47740728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f4779218640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f47740728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f4779218640 1 -- 172.21.15.110:0/3265877393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009560 con 0x7f47740ff7c0 2024-09-21T04:18:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f4778a17640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f47740728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f4779218640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f477000c8f0 tx=0x7f477000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47700041f0 con 0x7f47740ff7c0 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4770007e70 con 0x7f47740ff7c0 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47700049d0 con 0x7f47740ff7c0 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774073230 con 0x7f47740ff7c0 2024-09-21T04:18:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.464+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477419d7c0 con 0x7f47740ff7c0 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.465+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47740ffc40 con 0x7f47740ff7c0 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.466+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f4770004350 con 0x7f47740ff7c0 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.466+0000 7f476a7fc640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 0x7f476007d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.467+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f477009e3a0 con 0x7f47740ff7c0 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.467+0000 7f4778a17640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 0x7f476007d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.467+0000 7f4778a17640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 0x7f476007d860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4764031540 tx=0x7f4764005de0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.471+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4770061190 con 0x7f47740ff7c0 2024-09-21T04:18:32.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.726+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f477402c3e0 con 0x7f47740ff7c0 2024-09-21T04:18:32.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.727+0000 7f476a7fc640 1 -- 172.21.15.110:0/3265877393 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+729 (secure 0 0 0) 0x7f47700642b0 con 0x7f47740ff7c0 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 5 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:32.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 9, 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:18:32.731 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 msgr2=0x7f476007d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 0x7f476007d860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4764031540 tx=0x7f4764005de0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 msgr2=0x7f4774074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f477000c8f0 tx=0x7f477000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 shutdown_connections 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f476007b3a0 0x7f476007d860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4774101280 0x7f47740728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 --2- 172.21.15.110:0/3265877393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47740ff7c0 0x7f4774074250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 >> 172.21.15.110:0/3265877393 conn(0x7f47740fb330 msgr2=0x7f4774104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 shutdown_connections 2024-09-21T04:18:32.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.729+0000 7f477b47c640 1 -- 172.21.15.110:0/3265877393 wait complete. 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- 172.21.15.110:0/2228001533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 msgr2=0x7f4270101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 --2- 172.21.15.110:0/2228001533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270101680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4260009880 tx=0x7f426002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- 172.21.15.110:0/2228001533 shutdown_connections 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 --2- 172.21.15.110:0/2228001533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f42700ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 --2- 172.21.15.110:0/2228001533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270101680 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- 172.21.15.110:0/2228001533 >> 172.21.15.110:0/2228001533 conn(0x7f42700fb330 msgr2=0x7f42700fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- 172.21.15.110:0/2228001533 shutdown_connections 2024-09-21T04:18:32.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- 172.21.15.110:0/2228001533 wait complete. 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 Processor -- start 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.810+0000 7f42769d3640 1 -- start start 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f42769d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f4270195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f42769d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f42769d3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42701962a0 con 0x7f4270101280 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f42769d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270196410 con 0x7f42700ff7c0 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36446/0 (socket says 172.21.15.110:36446) 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 -- 172.21.15.110:0/4137507342 learned_addr learned my addr 172.21.15.110:0/4137507342 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426ffff640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f4270195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 -- 172.21.15.110:0/4137507342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 msgr2=0x7f4270195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f4270195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 -- 172.21.15.110:0/4137507342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009560 con 0x7f4270101280 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426ffff640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f4270195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.811+0000 7f426f7fe640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f426400d900 tx=0x7f426400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.812+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4264009c00 con 0x7f4270101280 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.812+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f426400b660 con 0x7f4270101280 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.812+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427019aeb0 con 0x7f4270101280 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.812+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f426400f680 con 0x7f4270101280 2024-09-21T04:18:32.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.812+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427019b450 con 0x7f4270101280 2024-09-21T04:18:32.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.813+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42700ffc40 con 0x7f4270101280 2024-09-21T04:18:32.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.814+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f4264010ab0 con 0x7f4270101280 2024-09-21T04:18:32.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.815+0000 7f426d7fa640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 0x7f425807d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:32.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.815+0000 7f426ffff640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 0x7f425807d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:32.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.815+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f426409e2d0 con 0x7f4270101280 2024-09-21T04:18:32.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.815+0000 7f426ffff640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 0x7f425807d850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4260006010 tx=0x7f4260005d50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:32.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:32.818+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f42640611e0 con 0x7f4270101280 2024-09-21T04:18:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.051+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4270108880 con 0x7f4270101280 2024-09-21T04:18:33.054 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:32 smithi110 podman[99636]: 2024-09-21 04:18:32.904101239 +0000 UTC m=+1.978530967 container cleanup 53d83efb595606db849f5d509b1cd04e7ad8c61a5fd785de5a7ecbb6f04bcb0a (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=reef-6a1bdf8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T04:18:33.054 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:32 smithi110 bash[99636]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0 2024-09-21T04:18:33.054 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:18:33.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: osdmap e50: 6 total, 5 up, 6 in 2024-09-21T04:18:33.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:33.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:33.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3265877393' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:33.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:32 smithi110 ceph-mon[92455]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.051+0000 7f426d7fa640 1 -- 172.21.15.110:0/4137507342 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f4264020790 con 0x7f4270101280 2024-09-21T04:18:33.055 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:18:33.056 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:18:33.057 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:33.058 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:33.059 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:18:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:18:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.053+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 msgr2=0x7f425807d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.053+0000 7f42769d3640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 0x7f425807d850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4260006010 tx=0x7f4260005d50 comp rx=0 tx=0).stop 2024-09-21T04:18:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.053+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 msgr2=0x7f4270195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.053+0000 7f42769d3640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f426400d900 tx=0x7f426400ddc0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 shutdown_connections 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f425807b390 0x7f425807d850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4270101280 0x7f4270195cd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 --2- 172.21.15.110:0/4137507342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42700ff7c0 0x7f4270195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 >> 172.21.15.110:0/4137507342 conn(0x7f42700fb330 msgr2=0x7f4270104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 shutdown_connections 2024-09-21T04:18:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.054+0000 7f42769d3640 1 -- 172.21.15.110:0/4137507342 wait complete. 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- 172.21.15.110:0/2635629892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57301005c0 msgr2=0x7f57301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 --2- 172.21.15.110:0/2635629892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57301005c0 0x7f57301029b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57200098b0 tx=0x7f572002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- 172.21.15.110:0/2635629892 shutdown_connections 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 --2- 172.21.15.110:0/2635629892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730102ef0 0x7f57301052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 --2- 172.21.15.110:0/2635629892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57301005c0 0x7f57301029b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- 172.21.15.110:0/2635629892 >> 172.21.15.110:0/2635629892 conn(0x7f57300fa1c0 msgr2=0x7f57300fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- 172.21.15.110:0/2635629892 shutdown_connections 2024-09-21T04:18:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- 172.21.15.110:0/2635629892 wait complete. 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 Processor -- start 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.137+0000 7f57359f9640 1 -- start start 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57359f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57301005c0 0x7f5730195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57359f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57359f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730196260 con 0x7f5730102ef0 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57359f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301963d0 con 0x7f57301005c0 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36472/0 (socket says 172.21.15.110:36472) 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 -- 172.21.15.110:0/3097279967 learned_addr learned my addr 172.21.15.110:0/3097279967 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f572effd640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57301005c0 0x7f5730195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 -- 172.21.15.110:0/3097279967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57301005c0 msgr2=0x7f5730195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57301005c0 0x7f5730195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 -- 172.21.15.110:0/3097279967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720009560 con 0x7f5730102ef0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f5727fff640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f571800b400 tx=0x7f571800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5718011800 con 0x7f5730102ef0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.138+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5718011e20 con 0x7f5730102ef0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.139+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573019ae70 con 0x7f5730102ef0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.139+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571800f2e0 con 0x7f5730102ef0 2024-09-21T04:18:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.139+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573019b410 con 0x7f5730102ef0 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.139+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57301087c0 con 0x7f5730102ef0 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.140+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f5718010380 con 0x7f5730102ef0 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.141+0000 7f57349f7640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 0x7f571c07d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.141+0000 7f572effd640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 0x7f571c07d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.141+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f571809e030 con 0x7f5730102ef0 2024-09-21T04:18:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.142+0000 7f572effd640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 0x7f571c07d850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5720006010 tx=0x7f5720031500 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:33.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.145+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f5718060ea0 con 0x7f5730102ef0 2024-09-21T04:18:33.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.346+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f573019b6e0 con 0x7f571c07b390 2024-09-21T04:18:33.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.348+0000 7f57349f7640 1 -- 172.21.15.110:0/3097279967 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f573019b6e0 con 0x7f571c07b390 2024-09-21T04:18:33.350 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "6/23 daemons upgraded", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:18:33.351 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:18:33.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 msgr2=0x7f571c07d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 0x7f571c07d850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5720006010 tx=0x7f5720031500 comp rx=0 tx=0).stop 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 msgr2=0x7f5730195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f571800b400 tx=0x7f571800b8c0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 shutdown_connections 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f571c07b390 0x7f571c07d850 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5730102ef0 0x7f5730195c90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 --2- 172.21.15.110:0/3097279967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57301005c0 0x7f5730195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 >> 172.21.15.110:0/3097279967 conn(0x7f57300fa1c0 msgr2=0x7f57300fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 shutdown_connections 2024-09-21T04:18:33.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.350+0000 7f57359f9640 1 -- 172.21.15.110:0/3097279967 wait complete. 2024-09-21T04:18:33.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 -- 172.21.15.110:0/1839797335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 msgr2=0x7f676c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 --2- 172.21.15.110:0/1839797335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c1052e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67540098b0 tx=0x7f675402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 -- 172.21.15.110:0/1839797335 shutdown_connections 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 --2- 172.21.15.110:0/1839797335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c1052e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 --2- 172.21.15.110:0/1839797335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 0x7f676c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 -- 172.21.15.110:0/1839797335 >> 172.21.15.110:0/1839797335 conn(0x7f676c0fa1c0 msgr2=0x7f676c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 -- 172.21.15.110:0/1839797335 shutdown_connections 2024-09-21T04:18:33.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.437+0000 7f6771be4640 1 -- 172.21.15.110:0/1839797335 wait complete. 2024-09-21T04:18:33.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:18:33.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: osdmap e50: 6 total, 5 up, 6 in 2024-09-21T04:18:33.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:33.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:33.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3265877393' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:33.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:32 smithi148 ceph-mon[78701]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.438+0000 7f6771be4640 1 Processor -- start 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.438+0000 7f6771be4640 1 -- start start 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f6771be4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 0x7f676c0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f6771be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f6771be4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c100460 con 0x7f676c102ef0 2024-09-21T04:18:33.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f6771be4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0feae0 con 0x7f676c1005c0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36490/0 (socket says 172.21.15.110:36490) 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 -- 172.21.15.110:0/3237898477 learned_addr learned my addr 172.21.15.110:0/3237898477 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676b7fe640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 0x7f676c0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 -- 172.21.15.110:0/3237898477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 msgr2=0x7f676c0fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 0x7f676c0fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 -- 172.21.15.110:0/3237898477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009560 con 0x7f676c102ef0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.439+0000 7f676affd640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f675402a9f0 tx=0x7f6754005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.440+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6754004b20 con 0x7f676c102ef0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.440+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6754004c80 con 0x7f676c102ef0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.440+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c0fed60 con 0x7f676c102ef0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.440+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c0ff2d0 con 0x7f676c102ef0 2024-09-21T04:18:33.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.440+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6754003bc0 con 0x7f676c102ef0 2024-09-21T04:18:33.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.441+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c108720 con 0x7f676c102ef0 2024-09-21T04:18:33.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.441+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 103424+0+0 (secure 0 0 0) 0x7f6754031640 con 0x7f676c102ef0 2024-09-21T04:18:33.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.442+0000 7f6768ff9640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 0x7f675807d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:18:33.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.442+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f67540bf760 con 0x7f676c102ef0 2024-09-21T04:18:33.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.442+0000 7f676b7fe640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 0x7f675807d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:18:33.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.442+0000 7f676b7fe640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 0x7f675807d860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6764007ec0 tx=0x7f67640075b0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:18:33.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.446+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f6754082600 con 0x7f676c102ef0 2024-09-21T04:18:33.661 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:33 smithi110 podman[99720]: 2024-09-21 04:18:33.315009099 +0000 UTC m=+0.960078880 container remove 53d83efb595606db849f5d509b1cd04e7ad8c61a5fd785de5a7ecbb6f04bcb0a (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=reef-6a1bdf8, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:18:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.722+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f676c19d8e0 con 0x7f676c102ef0 2024-09-21T04:18:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.723+0000 7f6768ff9640 1 -- 172.21.15.110:0/3237898477 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f6754085720 con 0x7f676c102ef0 2024-09-21T04:18:33.726 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 osds down 2024-09-21T04:18:33.726 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-21T04:18:33.726 INFO:teuthology.orchestra.run.smithi110.stdout: osd.0 (root=default,host=smithi110) is down 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 msgr2=0x7f675807d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 0x7f675807d860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6764007ec0 tx=0x7f67640075b0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 msgr2=0x7f676c0fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f675402a9f0 tx=0x7f6754005610 comp rx=0 tx=0).stop 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 shutdown_connections 2024-09-21T04:18:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f675807b3a0 0x7f675807d860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f676c102ef0 0x7f676c0fe570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 --2- 172.21.15.110:0/3237898477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f676c1005c0 0x7f676c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:18:33.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 >> 172.21.15.110:0/3237898477 conn(0x7f676c0fa1c0 msgr2=0x7f676c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:18:33.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 shutdown_connections 2024-09-21T04:18:33.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:18:33.725+0000 7f6771be4640 1 -- 172.21.15.110:0/3237898477 wait complete. 2024-09-21T04:18:33.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:33 smithi110 podman[99896]: 2024-09-21 04:18:33.564529729 +0000 UTC m=+0.018191912 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:18:33.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:33 smithi110 podman[99896]: 2024-09-21 04:18:33.666018589 +0000 UTC m=+0.119680747 container create 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731) 2024-09-21T04:18:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:33 smithi110 ceph-mon[92455]: osdmap e51: 6 total, 5 up, 6 in 2024-09-21T04:18:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:33 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4137507342' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:18:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:33 smithi110 ceph-mon[92455]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:33 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3237898477' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:18:34.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:33 smithi110 podman[99896]: 2024-09-21 04:18:33.927801321 +0000 UTC m=+0.381463493 container init 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T04:18:34.418 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:33 smithi110 podman[99896]: 2024-09-21 04:18:33.932159131 +0000 UTC m=+0.385821298 container start 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T04:18:34.418 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 podman[99896]: 2024-09-21 04:18:34.036274635 +0000 UTC m=+0.489936793 container attach 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T04:18:34.418 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 podman[99896]: 2024-09-21 04:18:34.122191392 +0000 UTC m=+0.575853569 container died 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T04:18:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:33 smithi148 ceph-mon[78701]: osdmap e51: 6 total, 5 up, 6 in 2024-09-21T04:18:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:33 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4137507342' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:18:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:33 smithi148 ceph-mon[78701]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:18:34.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:33 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3237898477' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:18:34.928 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 podman[99921]: 2024-09-21 04:18:34.690190149 +0000 UTC m=+0.557431196 container remove 08c260a3c875f5596e6c5dffe0629190937287d9d3193db48c02069082cf95ec (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T04:18:34.928 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.0.service: Deactivated successfully. 2024-09-21T04:18:34.928 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 systemd[1]: Stopped Ceph osd.0 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:18:34.928 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.0.service: Consumed 1min 854ms CPU time. 2024-09-21T04:18:35.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:34 smithi148 ceph-mon[78701]: pgmap v41: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:35.257 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:34 smithi110 ceph-mon[92455]: pgmap v41: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:35.257 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:34 smithi110 systemd[1]: Starting Ceph osd.0 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:18:35.257 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 podman[100020]: 2024-09-21 04:18:35.039104342 +0000 UTC m=+0.016067186 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:18:35.257 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 podman[100020]: 2024-09-21 04:18:35.14111973 +0000 UTC m=+0.118082574 container create f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 podman[100020]: 2024-09-21 04:18:35.360501157 +0000 UTC m=+0.337464011 container init f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 podman[100020]: 2024-09-21 04:18:35.363835095 +0000 UTC m=+0.340797930 container start f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 podman[100020]: 2024-09-21 04:18:35.472517297 +0000 UTC m=+0.449480148 container attach f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 bash[100020]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:35.509 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:35 smithi110 bash[100020]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:36.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:35 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 45/222 objects degraded (20.270%), 17 pgs degraded (PG_DEGRADED) 2024-09-21T04:18:36.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:35 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 45/222 objects degraded (20.270%), 17 pgs degraded (PG_DEGRADED) 2024-09-21T04:18:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:36 smithi110 ceph-mon[92455]: pgmap v42: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:36 smithi148 ceph-mon[78701]: pgmap v42: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:38.667 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:18:38.667 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:38.667 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:18:38.667 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:38.667 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:38.668 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:18:38.668 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:18:38.668 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:18:38.668 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-21T04:18:38.668 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-21T04:18:39.100 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate[100035]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 bash[100020]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-21T04:18:39.101 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:38 smithi110 podman[100020]: 2024-09-21 04:18:38.734413254 +0000 UTC m=+3.711376094 container died f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:18:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:39 smithi110 ceph-mon[92455]: pgmap v43: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:39 smithi148 ceph-mon[78701]: pgmap v43: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:39.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 podman[100258]: 2024-09-21 04:18:39.456873498 +0000 UTC m=+0.716701356 container remove f755a6d1fa8cfdbd3df5175530f3a1ebcb83134ea33b828476ef015d61592d01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T04:18:39.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 podman[100319]: 2024-09-21 04:18:39.571570827 +0000 UTC m=+0.016283700 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:18:39.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 podman[100319]: 2024-09-21 04:18:39.674613786 +0000 UTC m=+0.119326699 container create a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:18:40.167 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 podman[100319]: 2024-09-21 04:18:39.902763507 +0000 UTC m=+0.347476378 container init a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T04:18:40.167 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 podman[100319]: 2024-09-21 04:18:39.905978938 +0000 UTC m=+0.350691812 container start a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T04:18:40.167 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 ceph-osd[100339]: -- 172.21.15.110:0/1902846257 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608b499c1e0 con 0x5608b492c000 2024-09-21T04:18:40.167 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 bash[100319]: a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 2024-09-21T04:18:40.167 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:39 smithi110 systemd[1]: Started Ceph osd.0 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:18:41.286 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:41.286 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:41.286 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:41.286 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:41 smithi110 ceph-mon[92455]: pgmap v44: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:41 smithi148 ceph-mon[78701]: pgmap v44: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:41.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:41 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:18:41.541+0000 7ffa6afc3640 -1 Falling back to public interface 2024-09-21T04:18:43.354 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:42 smithi110 ceph-mon[92455]: pgmap v45: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:42 smithi148 ceph-mon[78701]: pgmap v45: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:46.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:45 smithi110 ceph-mon[92455]: pgmap v46: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:46.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:45 smithi148 ceph-mon[78701]: pgmap v46: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:46.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:46.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:47.896 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:47 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:18:47.642+0000 7ffa6afc3640 -1 osd.0 49 log_to_monitors true 2024-09-21T04:18:48.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:47 smithi110 ceph-mon[92455]: pgmap v47: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:48.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:47 smithi110 ceph-mon[92455]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T04:18:48.168 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:18:47 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:18:47.915+0000 7ffa62d6d640 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:18:48.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:47 smithi148 ceph-mon[78701]: pgmap v47: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:48.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:47 smithi148 ceph-mon[78701]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: osdmap e52: 6 total, 5 up, 6 in 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:18:49.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: osdmap e52: 6 total, 5 up, 6 in 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:18:49.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: pgmap v49: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417] boot 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: osdmap e53: 6 total, 6 up, 6 in 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:18:50.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:49 smithi110 ceph-mon[92455]: osdmap e54: 6 total, 6 up, 6 in 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: pgmap v49: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: osd.0 [v2:172.21.15.110:6802/1806639417,v1:172.21.15.110:6803/1806639417] boot 2024-09-21T04:18:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: osdmap e53: 6 total, 6 up, 6 in 2024-09-21T04:18:50.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:18:50.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:49 smithi148 ceph-mon[78701]: osdmap e54: 6 total, 6 up, 6 in 2024-09-21T04:18:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:50 smithi148 ceph-mon[78701]: pgmap v52: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:51.253 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:50 smithi110 ceph-mon[92455]: pgmap v52: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:53.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:52 smithi110 ceph-mon[92455]: pgmap v53: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:53.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:52 smithi148 ceph-mon[78701]: pgmap v53: 65 pgs: 17 active+undersized, 17 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/222 objects degraded (20.270%) 2024-09-21T04:18:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:18:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:18:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:55 smithi110 ceph-mon[92455]: pgmap v54: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 441 B/s wr, 0 op/s 2024-09-21T04:18:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:55 smithi148 ceph-mon[78701]: pgmap v54: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 441 B/s wr, 0 op/s 2024-09-21T04:18:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:56 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/222 objects degraded (20.270%), 17 pgs degraded) 2024-09-21T04:18:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:56 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:18:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:56 smithi110 ceph-mon[92455]: mgrmap e39: smithi110.axmqzb(active, since 92s), standbys: smithi148.zjrahi 2024-09-21T04:18:56.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:56 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/222 objects degraded (20.270%), 17 pgs degraded) 2024-09-21T04:18:56.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:56 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:18:56.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:56 smithi148 ceph-mon[78701]: mgrmap e39: smithi110.axmqzb(active, since 92s), standbys: smithi148.zjrahi 2024-09-21T04:18:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:57 smithi110 ceph-mon[92455]: pgmap v55: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-09-21T04:18:57.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:57 smithi148 ceph-mon[78701]: pgmap v55: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-09-21T04:18:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:18:58 smithi110 ceph-mon[92455]: pgmap v56: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 308 B/s wr, 0 op/s 2024-09-21T04:18:59.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:18:58 smithi148 ceph-mon[78701]: pgmap v56: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 308 B/s wr, 0 op/s 2024-09-21T04:19:01.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:00 smithi110 ceph-mon[92455]: pgmap v57: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 280 B/s wr, 0 op/s 2024-09-21T04:19:01.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:00 smithi148 ceph-mon[78701]: pgmap v57: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 280 B/s wr, 0 op/s 2024-09-21T04:19:03.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:02 smithi110 ceph-mon[92455]: pgmap v58: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T04:19:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:02 smithi148 ceph-mon[78701]: pgmap v58: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 -- 172.21.15.110:0/3924142256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f41005c0 msgr2=0x7f19f41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 --2- 172.21.15.110:0/3924142256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f41005c0 0x7f19f41029b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19e40098b0 tx=0x7f19e402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 -- 172.21.15.110:0/3924142256 shutdown_connections 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 --2- 172.21.15.110:0/3924142256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f4102ef0 0x7f19f41052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 --2- 172.21.15.110:0/3924142256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f41005c0 0x7f19f41029b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 -- 172.21.15.110:0/3924142256 >> 172.21.15.110:0/3924142256 conn(0x7f19f40fa1c0 msgr2=0x7f19f40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 -- 172.21.15.110:0/3924142256 shutdown_connections 2024-09-21T04:19:03.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.818+0000 7f19fa243640 1 -- 172.21.15.110:0/3924142256 wait complete. 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.819+0000 7f19fa243640 1 Processor -- start 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19fa243640 1 -- start start 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19fa243640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f41005c0 0x7f19f4195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19fa243640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19fa243640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41960e0 con 0x7f19f4102ef0 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19fa243640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4196250 con 0x7f19f41005c0 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58634/0 (socket says 172.21.15.110:58634) 2024-09-21T04:19:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 -- 172.21.15.110:0/895933462 learned_addr learned my addr 172.21.15.110:0/895933462 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f37fe640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f41005c0 0x7f19f4195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 -- 172.21.15.110:0/895933462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f41005c0 msgr2=0x7f19f4195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f41005c0 0x7f19f4195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.820+0000 7f19f2ffd640 1 -- 172.21.15.110:0/895933462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009560 con 0x7f19f4102ef0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19f2ffd640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c8f0 tx=0x7f19e800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e8004410 con 0x7f19f4102ef0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f419ad30 con 0x7f19f4102ef0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19e8007c60 con 0x7f19f4102ef0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e800f450 con 0x7f19f4102ef0 2024-09-21T04:19:03.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.821+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f419b2b0 con 0x7f19f4102ef0 2024-09-21T04:19:03.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.822+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4108750 con 0x7f19f4102ef0 2024-09-21T04:19:03.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.823+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f19e80077a0 con 0x7f19f4102ef0 2024-09-21T04:19:03.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.823+0000 7f19f0ff9640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 0x7f19dc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:03.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.823+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19e809e1c0 con 0x7f19f4102ef0 2024-09-21T04:19:03.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.823+0000 7f19f37fe640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 0x7f19dc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:03.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.824+0000 7f19f37fe640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 0x7f19dc07d740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19e4006010 tx=0x7f19e40314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:03.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:03.828+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f19e8061260 con 0x7f19f4102ef0 2024-09-21T04:19:04.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.035+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19f4103990 con 0x7f19dc07b280 2024-09-21T04:19:04.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.037+0000 7f19f0ff9640 1 -- 172.21.15.110:0/895933462 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f19f4103990 con 0x7f19dc07b280 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 msgr2=0x7f19dc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 0x7f19dc07d740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19e4006010 tx=0x7f19e40314e0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 msgr2=0x7f19f4195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c8f0 tx=0x7f19e800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 shutdown_connections 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f19dc07b280 0x7f19dc07d740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f4102ef0 0x7f19f4195ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 --2- 172.21.15.110:0/895933462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f41005c0 0x7f19f4195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 >> 172.21.15.110:0/895933462 conn(0x7f19f40fa1c0 msgr2=0x7f19f40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 shutdown_connections 2024-09-21T04:19:04.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.040+0000 7f19fa243640 1 -- 172.21.15.110:0/895933462 wait complete. 2024-09-21T04:19:04.053 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:19:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 -- 172.21.15.110:0/196672497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800ffc30 msgr2=0x7f40801000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 --2- 172.21.15.110:0/196672497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800ffc30 0x7f40801000b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4074009880 tx=0x7f407402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 -- 172.21.15.110:0/196672497 shutdown_connections 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 --2- 172.21.15.110:0/196672497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800ffc30 0x7f40801000b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 --2- 172.21.15.110:0/196672497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800fea30 0x7f40800fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.123+0000 7f408515b640 1 -- 172.21.15.110:0/196672497 >> 172.21.15.110:0/196672497 conn(0x7f40800fa1c0 msgr2=0x7f40800fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.124+0000 7f408515b640 1 -- 172.21.15.110:0/196672497 shutdown_connections 2024-09-21T04:19:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.124+0000 7f408515b640 1 -- 172.21.15.110:0/196672497 wait complete. 2024-09-21T04:19:04.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.124+0000 7f408515b640 1 Processor -- start 2024-09-21T04:19:04.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.124+0000 7f408515b640 1 -- start start 2024-09-21T04:19:04.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 0x7f4080195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080196160 con 0x7f40800fea30 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40801962d0 con 0x7f40800ffc30 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58654/0 (socket says 172.21.15.110:58654) 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 -- 172.21.15.110:0/1373782920 learned_addr learned my addr 172.21.15.110:0/1373782920 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407e59c640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 0x7f4080195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 -- 172.21.15.110:0/1373782920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 msgr2=0x7f4080195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 0x7f4080195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 -- 172.21.15.110:0/1373782920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4074009560 con 0x7f40800fea30 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407e59c640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 0x7f4080195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:19:04.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f407ed9d640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f407000e970 tx=0x7f407000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4070009700 con 0x7f40800fea30 2024-09-21T04:19:04.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f407000cc90 con 0x7f40800fea30 2024-09-21T04:19:04.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408019ad70 con 0x7f40800fea30 2024-09-21T04:19:04.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4070010430 con 0x7f40800fea30 2024-09-21T04:19:04.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.125+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408019b2b0 con 0x7f40800fea30 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.126+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4080107620 con 0x7f40800fea30 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.127+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4070003f70 con 0x7f40800fea30 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.128+0000 7f4067fff640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 0x7f406807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.128+0000 7f407e59c640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 0x7f406807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.128+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4070014070 con 0x7f40800fea30 2024-09-21T04:19:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.128+0000 7f407e59c640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 0x7f406807d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4074000c00 tx=0x7f40740314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.132+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4070061940 con 0x7f40800fea30 2024-09-21T04:19:04.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.334+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40800683c0 con 0x7f406807b280 2024-09-21T04:19:04.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.336+0000 7f4067fff640 1 -- 172.21.15.110:0/1373782920 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f40800683c0 con 0x7f406807b280 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 msgr2=0x7f406807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 0x7f406807d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4074000c00 tx=0x7f40740314e0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 msgr2=0x7f4080195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f407000e970 tx=0x7f407000ee30 comp rx=0 tx=0).stop 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 shutdown_connections 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f406807b280 0x7f406807d740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40800ffc30 0x7f4080195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 --2- 172.21.15.110:0/1373782920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40800fea30 0x7f4080195650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 >> 172.21.15.110:0/1373782920 conn(0x7f40800fa1c0 msgr2=0x7f4080067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 shutdown_connections 2024-09-21T04:19:04.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.338+0000 7f408515b640 1 -- 172.21.15.110:0/1373782920 wait complete. 2024-09-21T04:19:04.408 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:04.408 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:04 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:19:04.408 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:04 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:19:04.408 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:04 smithi110 ceph-mon[92455]: Upgrade: osd.1 is safe to restart 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.424+0000 7f5082c27640 1 -- 172.21.15.110:0/1995713697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 msgr2=0x7f507c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.424+0000 7f5082c27640 1 --2- 172.21.15.110:0/1995713697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c102a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5070009880 tx=0x7f507002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 -- 172.21.15.110:0/1995713697 shutdown_connections 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 --2- 172.21.15.110:0/1995713697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 --2- 172.21.15.110:0/1995713697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c102a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 -- 172.21.15.110:0/1995713697 >> 172.21.15.110:0/1995713697 conn(0x7f507c0fa1c0 msgr2=0x7f507c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 -- 172.21.15.110:0/1995713697 shutdown_connections 2024-09-21T04:19:04.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 -- 172.21.15.110:0/1995713697 wait complete. 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 Processor -- start 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.425+0000 7f5082c27640 1 -- start start 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5082c27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5082c27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5082c27640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1960b0 con 0x7f507c100620 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5082c27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c196220 con 0x7f507c102f50 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58678/0 (socket says 172.21.15.110:58678) 2024-09-21T04:19:04.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 -- 172.21.15.110:0/1019351861 learned_addr learned my addr 172.21.15.110:0/1019351861 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f507bfff640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 -- 172.21.15.110:0/1019351861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 msgr2=0x7f507c195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 -- 172.21.15.110:0/1019351861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009560 con 0x7f507c100620 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f507bfff640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f50809c3640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50700055a0 tx=0x7f50700055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5070004b20 con 0x7f507c100620 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5070004c80 con 0x7f507c100620 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.426+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c19acc0 con 0x7f507c100620 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.427+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5070003bc0 con 0x7f507c100620 2024-09-21T04:19:04.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.427+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c19b240 con 0x7f507c100620 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.428+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c1086e0 con 0x7f507c100620 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.428+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f5070003d20 con 0x7f507c100620 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.429+0000 7f5079ffb640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 0x7f506c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.429+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f50700bf550 con 0x7f507c100620 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.429+0000 7f507bfff640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 0x7f506c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.429+0000 7f507bfff640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 0x7f506c07d740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f50680058c0 tx=0x7f5068005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.433+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f50700825e0 con 0x7f507c100620 2024-09-21T04:19:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:04 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:19:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:04 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:19:04.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:04 smithi148 ceph-mon[78701]: Upgrade: osd.1 is safe to restart 2024-09-21T04:19:04.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.625+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f507c1039b0 con 0x7f506c07b280 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.636+0000 7f5079ffb640 1 -- 172.21.15.110:0/1019351861 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f507c1039b0 con 0x7f506c07b280 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (10m) 19s ago 12m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (3m) 19s ago 12m 7717k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (3m) 44s ago 11m 7369k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (52s) 19s ago 12m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:19:04.639 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (47s) 44s ago 11m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (10m) 19s ago 12m 84.9M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (6m) 19s ago 6m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (6m) 19s ago 6m 15.9M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (6m) 44s ago 6m 17.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (6m) 44s ago 6m 18.8M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (3m) 19s ago 13m 509M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (2m) 44s ago 11m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (108s) 19s ago 13m 43.1M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (89s) 44s ago 11m 34.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (12m) 19s ago 12m 16.9M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (11m) 44s ago 11m 15.3M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (25s) 19s ago 9m 12.2M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (9m) 19s ago 9m 278M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 1505358e6eb2 2024-09-21T04:19:04.640 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (9m) 19s ago 9m 229M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:19:04.641 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (8m) 44s ago 8m 326M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:19:04.641 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (8m) 44s ago 8m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:19:04.641 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (7m) 44s ago 7m 254M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:19:04.641 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 19s ago 11m 55.7M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:19:04.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 msgr2=0x7f506c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 0x7f506c07d740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f50680058c0 tx=0x7f5068005850 comp rx=0 tx=0).stop 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 msgr2=0x7f507c1955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50700055a0 tx=0x7f50700055d0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 shutdown_connections 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f506c07b280 0x7f506c07d740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f507c102f50 0x7f507c195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 --2- 172.21.15.110:0/1019351861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c100620 0x7f507c1955a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 >> 172.21.15.110:0/1019351861 conn(0x7f507c0fa1c0 msgr2=0x7f507c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 shutdown_connections 2024-09-21T04:19:04.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.638+0000 7f5082c27640 1 -- 172.21.15.110:0/1019351861 wait complete. 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 -- 172.21.15.110:0/3011782345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9480ffb70 msgr2=0x7fe9480fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/3011782345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9480ffb70 0x7fe9480fff70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe938009880 tx=0x7fe93802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 -- 172.21.15.110:0/3011782345 shutdown_connections 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/3011782345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe948100d70 0x7fe9481011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/3011782345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9480ffb70 0x7fe9480fff70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 -- 172.21.15.110:0/3011782345 >> 172.21.15.110:0/3011782345 conn(0x7fe9480fb2c0 msgr2=0x7fe9480fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 -- 172.21.15.110:0/3011782345 shutdown_connections 2024-09-21T04:19:04.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 -- 172.21.15.110:0/3011782345 wait complete. 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.720+0000 7fe94fa5d640 1 Processor -- start 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94fa5d640 1 -- start start 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94fa5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9480ffb70 0x7fe948195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94fa5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94fa5d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948196150 con 0x7fe948100d70 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94fa5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481962c0 con 0x7fe9480ffb70 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58698/0 (socket says 172.21.15.110:58698) 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 -- 172.21.15.110:0/1854092677 learned_addr learned my addr 172.21.15.110:0/1854092677 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94d7f9640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9480ffb70 0x7fe948195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 -- 172.21.15.110:0/1854092677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9480ffb70 msgr2=0x7fe948195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9480ffb70 0x7fe948195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 -- 172.21.15.110:0/1854092677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938009560 con 0x7fe948100d70 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe94cff8640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9a0 tx=0x7fe94400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.721+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94400cd50 con 0x7fe948100d70 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.722+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94819ad60 con 0x7fe948100d70 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.722+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe94400ceb0 con 0x7fe948100d70 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.722+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe944010660 con 0x7fe948100d70 2024-09-21T04:19:04.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.722+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94819b330 con 0x7fe948100d70 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.723+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948108780 con 0x7fe948100d70 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.723+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fe9440048a0 con 0x7fe948100d70 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.724+0000 7fe93e7fc640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 0x7fe93407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.724+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe944014070 con 0x7fe948100d70 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.724+0000 7fe94d7f9640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 0x7fe93407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:04.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.724+0000 7fe94d7f9640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 0x7fe93407d740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe938009230 tx=0x7fe9380314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:04.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.728+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe944064ff0 con 0x7fe948100d70 2024-09-21T04:19:04.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.981+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe948108910 con 0x7fe948100d70 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.982+0000 7fe93e7fc640 1 -- 172.21.15.110:0/1854092677 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fe944064990 con 0x7fe948100d70 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:19:04.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 5, 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 9, 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 5 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:19:04.986 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.984+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 msgr2=0x7fe93407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.984+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 0x7fe93407d740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe938009230 tx=0x7fe9380314e0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.984+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 msgr2=0x7fe948195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.984+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9a0 tx=0x7fe94400eda0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 shutdown_connections 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe93407b280 0x7fe93407d740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe948100d70 0x7fe948195b80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 --2- 172.21.15.110:0/1854092677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9480ffb70 0x7fe948195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 >> 172.21.15.110:0/1854092677 conn(0x7fe9480fb2c0 msgr2=0x7fe948103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 shutdown_connections 2024-09-21T04:19:04.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:04.985+0000 7fe94fa5d640 1 -- 172.21.15.110:0/1854092677 wait complete. 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- 172.21.15.110:0/1628517431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 msgr2=0x7f75400feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 --2- 172.21.15.110:0/1628517431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f75400feea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f753c009c20 tx=0x7f753c02a970 comp rx=0 tx=0).stop 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- 172.21.15.110:0/1628517431 shutdown_connections 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 --2- 172.21.15.110:0/1628517431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 --2- 172.21.15.110:0/1628517431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f75400feea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- 172.21.15.110:0/1628517431 >> 172.21.15.110:0/1628517431 conn(0x7f75400fa230 msgr2=0x7f75400fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- 172.21.15.110:0/1628517431 shutdown_connections 2024-09-21T04:19:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- 172.21.15.110:0/1628517431 wait complete. 2024-09-21T04:19:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 Processor -- start 2024-09-21T04:19:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 -- start start 2024-09-21T04:19:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.066+0000 7f75473a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75473a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75473a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7540196140 con 0x7f75400feaa0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75473a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75401962b0 con 0x7f75400ffca0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58720/0 (socket says 172.21.15.110:58720) 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 -- 172.21.15.110:0/3376935827 learned_addr learned my addr 172.21.15.110:0/3376935827 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7544941640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 -- 172.21.15.110:0/3376935827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 msgr2=0x7f7540195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 -- 172.21.15.110:0/3376935827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c009900 con 0x7f75400feaa0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7544941640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f7545142640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f753c02f620 tx=0x7f753c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f753c030840 con 0x7f75400feaa0 2024-09-21T04:19:05.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754019acf0 con 0x7f75400feaa0 2024-09-21T04:19:05.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f753c004040 con 0x7f75400feaa0 2024-09-21T04:19:05.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f753c03a640 con 0x7f75400feaa0 2024-09-21T04:19:05.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.067+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754019b210 con 0x7f75400feaa0 2024-09-21T04:19:05.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.068+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7540107690 con 0x7f75400feaa0 2024-09-21T04:19:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.069+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f753c00a040 con 0x7f75400feaa0 2024-09-21T04:19:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.070+0000 7f75367fc640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 0x7f752c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.070+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f753c02c040 con 0x7f75400feaa0 2024-09-21T04:19:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.070+0000 7f7544941640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 0x7f752c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.070+0000 7f7544941640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 0x7f752c07d730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7540196b50 tx=0x7f7530005e50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.074+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f753c082b50 con 0x7f75400feaa0 2024-09-21T04:19:05.278 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:05 smithi110 ceph-mon[92455]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.278 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:05 smithi110 ceph-mon[92455]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.279 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:05 smithi110 ceph-mon[92455]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.279 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:05 smithi110 ceph-mon[92455]: pgmap v59: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T04:19:05.279 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:05 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1854092677' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:05.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.291+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f754019b6f0 con 0x7f75400feaa0 2024-09-21T04:19:05.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.291+0000 7f75367fc640 1 -- 172.21.15.110:0/3376935827 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f753c0bf790 con 0x7f75400feaa0 2024-09-21T04:19:05.294 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:19:05.294 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:19:05.294 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:19:05.295 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:19:05.295 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:19:05.295 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:05.295 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:19:05.295 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:19:05.296 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:19:05.297 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 msgr2=0x7f752c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 0x7f752c07d730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7540196b50 tx=0x7f7530005e50 comp rx=0 tx=0).stop 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 msgr2=0x7f7540195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f753c02f620 tx=0x7f753c02f650 comp rx=0 tx=0).stop 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 shutdown_connections 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f752c07b270 0x7f752c07d730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75400ffca0 0x7f7540195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 --2- 172.21.15.110:0/3376935827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75400feaa0 0x7f7540195630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.293+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 >> 172.21.15.110:0/3376935827 conn(0x7f75400fa230 msgr2=0x7f7540068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.294+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 shutdown_connections 2024-09-21T04:19:05.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.294+0000 7f75473a6640 1 -- 172.21.15.110:0/3376935827 wait complete. 2024-09-21T04:19:05.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 -- 172.21.15.110:0/126679242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0ffca0 msgr2=0x7f8dcc100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 --2- 172.21.15.110:0/126679242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc100120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009880 tx=0x7f8dc002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 -- 172.21.15.110:0/126679242 shutdown_connections 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 --2- 172.21.15.110:0/126679242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc100120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 --2- 172.21.15.110:0/126679242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 -- 172.21.15.110:0/126679242 >> 172.21.15.110:0/126679242 conn(0x7f8dcc0fa230 msgr2=0x7f8dcc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 -- 172.21.15.110:0/126679242 shutdown_connections 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.376+0000 7f8dd1948640 1 -- 172.21.15.110:0/126679242 wait complete. 2024-09-21T04:19:05.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 Processor -- start 2024-09-21T04:19:05.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 -- start start 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc1962c0 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dd1948640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc196430 con 0x7f8dcc0ffca0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dcaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dcaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58736/0 (socket says 172.21.15.110:58736) 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.377+0000 7f8dcaffd640 1 -- 172.21.15.110:0/1930223733 learned_addr learned my addr 172.21.15.110:0/1930223733 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dca7fc640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dcaffd640 1 -- 172.21.15.110:0/1930223733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0ffca0 msgr2=0x7f8dcc195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dcaffd640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dcaffd640 1 -- 172.21.15.110:0/1930223733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009560 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dcaffd640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00b400 tx=0x7f8dbc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dbc011800 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc19ae80 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dbc011e20 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dbc00f2e0 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.378+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc19b3c0 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.379+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc107730 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.380+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8dbc010380 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.381+0000 7f8dd0946640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 0x7f8db407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.381+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8dbc09df30 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.381+0000 7f8dca7fc640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 0x7f8db407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.381+0000 7f8dca7fc640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 0x7f8db407d730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000c00 tx=0x7f8dc00314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.385+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8dbc0d5710 con 0x7f8dcc0feaa0 2024-09-21T04:19:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:05 smithi148 ceph-mon[78701]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:05 smithi148 ceph-mon[78701]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:05 smithi148 ceph-mon[78701]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:05 smithi148 ceph-mon[78701]: pgmap v59: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T04:19:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:05 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1854092677' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:05.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.587+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dcc19b670 con 0x7f8db407b270 2024-09-21T04:19:05.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.589+0000 7f8dd0946640 1 -- 172.21.15.110:0/1930223733 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8dcc19b670 con 0x7f8db407b270 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "7/23 daemons upgraded", 2024-09-21T04:19:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:19:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:19:05.593 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:19:05.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 msgr2=0x7f8db407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 0x7f8db407d730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000c00 tx=0x7f8dc00314e0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 msgr2=0x7f8dcc1957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00b400 tx=0x7f8dbc00b8c0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 shutdown_connections 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8db407b270 0x7f8db407d730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dcc0ffca0 0x7f8dcc195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 --2- 172.21.15.110:0/1930223733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dcc0feaa0 0x7f8dcc1957b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.591+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 >> 172.21.15.110:0/1930223733 conn(0x7f8dcc0fa230 msgr2=0x7f8dcc0fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.592+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 shutdown_connections 2024-09-21T04:19:05.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.592+0000 7f8dd1948640 1 -- 172.21.15.110:0/1930223733 wait complete. 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 -- 172.21.15.110:0/3858552249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 msgr2=0x7fca8c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 --2- 172.21.15.110:0/3858552249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c105360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009880 tx=0x7fca7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 -- 172.21.15.110:0/3858552249 shutdown_connections 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 --2- 172.21.15.110:0/3858552249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c105360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 --2- 172.21.15.110:0/3858552249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 0x7fca8c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 -- 172.21.15.110:0/3858552249 >> 172.21.15.110:0/3858552249 conn(0x7fca8c0fa1c0 msgr2=0x7fca8c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.677+0000 7fca943e8640 1 -- 172.21.15.110:0/3858552249 shutdown_connections 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 -- 172.21.15.110:0/3858552249 wait complete. 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 Processor -- start 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 -- start start 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 0x7fca8c195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c196140 con 0x7fca8c102f70 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca943e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c1962b0 con 0x7fca8c100640 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca91983640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca91983640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58760/0 (socket says 172.21.15.110:58760) 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.678+0000 7fca91983640 1 -- 172.21.15.110:0/1719714431 learned_addr learned my addr 172.21.15.110:0/1719714431 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca92184640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 0x7fca8c195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca91983640 1 -- 172.21.15.110:0/1719714431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 msgr2=0x7fca8c195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca91983640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 0x7fca8c195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca91983640 1 -- 172.21.15.110:0/1719714431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c009560 con 0x7fca8c102f70 2024-09-21T04:19:05.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca91983640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0054f0 tx=0x7fca7c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca7c004aa0 con 0x7fca8c102f70 2024-09-21T04:19:05.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca7c004c00 con 0x7fca8c102f70 2024-09-21T04:19:05.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c19aca0 con 0x7fca8c102f70 2024-09-21T04:19:05.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca7c003ac0 con 0x7fca8c102f70 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.679+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c19b210 con 0x7fca8c102f70 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.680+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c108710 con 0x7fca8c102f70 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.681+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fca7c003d80 con 0x7fca8c102f70 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.681+0000 7fca837fe640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 0x7fca7807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.681+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fca7c0bf530 con 0x7fca8c102f70 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.681+0000 7fca92184640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 0x7fca7807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:05.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.682+0000 7fca92184640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 0x7fca7807d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca8400bbe0 tx=0x7fca8400b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:05.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.686+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fca7c0825f0 con 0x7fca8c102f70 2024-09-21T04:19:05.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.945+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca8c1088f0 con 0x7fca8c102f70 2024-09-21T04:19:05.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.946+0000 7fca837fe640 1 -- 172.21.15.110:0/1719714431 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fca7c085710 con 0x7fca8c102f70 2024-09-21T04:19:05.949 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 msgr2=0x7fca7807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 0x7fca7807d740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca8400bbe0 tx=0x7fca8400b390 comp rx=0 tx=0).stop 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 msgr2=0x7fca8c195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0054f0 tx=0x7fca7c005520 comp rx=0 tx=0).stop 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 shutdown_connections 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fca7807b280 0x7fca7807d740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca8c102f70 0x7fca8c195b70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 --2- 172.21.15.110:0/1719714431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c100640 0x7fca8c195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:05.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 >> 172.21.15.110:0/1719714431 conn(0x7fca8c0fa1c0 msgr2=0x7fca8c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:05.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 shutdown_connections 2024-09-21T04:19:05.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:05.948+0000 7fca943e8640 1 -- 172.21.15.110:0/1719714431 wait complete. 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3376935827' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: Upgrade: Updating osd.1 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: Deploying daemon osd.1 on smithi110 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:06.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:06 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1719714431' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:19:06.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3376935827' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:19:06.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: Upgrade: Updating osd.1 2024-09-21T04:19:06.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:06.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:19:06.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:06.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: Deploying daemon osd.1 on smithi110 2024-09-21T04:19:06.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:06.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:06 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1719714431' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:19:07.358 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:07 smithi110 ceph-mon[92455]: pgmap v60: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:07.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:07 smithi148 ceph-mon[78701]: pgmap v60: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:08.618 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:08 smithi110 systemd[1]: Stopping Ceph osd.1 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:19:08.917 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:08 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[56910]: 2024-09-21T04:19:08.615+0000 7f3f2d6f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:19:08.917 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:08 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[56910]: 2024-09-21T04:19:08.615+0000 7f3f2d6f2640 -1 osd.1 54 *** Got signal Terminated *** 2024-09-21T04:19:08.917 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:08 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[56910]: 2024-09-21T04:19:08.615+0000 7f3f2d6f2640 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:19:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:09 smithi110 ceph-mon[92455]: osd.1 marked itself down and dead 2024-09-21T04:19:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:09 smithi110 ceph-mon[92455]: pgmap v61: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:09 smithi148 ceph-mon[78701]: osd.1 marked itself down and dead 2024-09-21T04:19:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:09 smithi148 ceph-mon[78701]: pgmap v61: 65 pgs: 65 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:09.969 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:09 smithi110 podman[105447]: 2024-09-21 04:19:09.698040722 +0000 UTC m=+1.192676891 container died 1505358e6eb25d05c4443bc71a83112ea893dd77b1b29a55da10c1b43de5ce6b (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-6a1bdf8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T04:19:10.360 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:10 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:19:10.361 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:10 smithi110 ceph-mon[92455]: osdmap e55: 6 total, 5 up, 6 in 2024-09-21T04:19:10.361 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105447]: 2024-09-21 04:19:10.065468363 +0000 UTC m=+1.560104517 container cleanup 1505358e6eb25d05c4443bc71a83112ea893dd77b1b29a55da10c1b43de5ce6b (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=reef-6a1bdf8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:19:10.361 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 bash[105447]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1 2024-09-21T04:19:10.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:10 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:19:10.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:10 smithi148 ceph-mon[78701]: osdmap e55: 6 total, 5 up, 6 in 2024-09-21T04:19:10.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105464]: 2024-09-21 04:19:10.358685124 +0000 UTC m=+0.653834413 container remove 1505358e6eb25d05c4443bc71a83112ea893dd77b1b29a55da10c1b43de5ce6b (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-6a1bdf8, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T04:19:10.981 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105553]: 2024-09-21 04:19:10.676133178 +0000 UTC m=+0.104267626 container create 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T04:19:10.981 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105553]: 2024-09-21 04:19:10.587844247 +0000 UTC m=+0.015978718 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:10.981 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105553]: 2024-09-21 04:19:10.870828746 +0000 UTC m=+0.298963211 container init 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T04:19:10.981 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105553]: 2024-09-21 04:19:10.875433457 +0000 UTC m=+0.303567904 container start 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T04:19:10.981 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:10 smithi110 podman[105553]: 2024-09-21 04:19:10.979109243 +0000 UTC m=+0.407243693 container attach 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T04:19:11.360 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:11 smithi110 ceph-mon[92455]: osdmap e56: 6 total, 5 up, 6 in 2024-09-21T04:19:11.360 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:11 smithi110 ceph-mon[92455]: pgmap v64: 65 pgs: 12 stale+active+clean, 53 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:11.360 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 podman[105553]: 2024-09-21 04:19:11.062837918 +0000 UTC m=+0.490972373 container died 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T04:19:11.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:11 smithi148 ceph-mon[78701]: osdmap e56: 6 total, 5 up, 6 in 2024-09-21T04:19:11.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:11 smithi148 ceph-mon[78701]: pgmap v64: 65 pgs: 12 stale+active+clean, 53 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:12.120 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 podman[105577]: 2024-09-21 04:19:11.666552094 +0000 UTC m=+0.597585486 container remove 6f7f0c721cc057f53cc67d317e229ae2a49cbbb4bdc31a236b38f81f01525412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:19:12.120 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.1.service: Deactivated successfully. 2024-09-21T04:19:12.120 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 systemd[1]: Stopped Ceph osd.1 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:19:12.120 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.1.service: Consumed 1min 7.251s CPU time. 2024-09-21T04:19:12.120 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:11 smithi110 systemd[1]: Starting Ceph osd.1 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:19:12.397 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 podman[105677]: 2024-09-21 04:19:12.117332949 +0000 UTC m=+0.112461772 container create 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:19:12.398 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 podman[105677]: 2024-09-21 04:19:12.020493829 +0000 UTC m=+0.015622670 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:12.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 podman[105677]: 2024-09-21 04:19:12.3955817 +0000 UTC m=+0.390710533 container init 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T04:19:12.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 podman[105677]: 2024-09-21 04:19:12.399138522 +0000 UTC m=+0.394267346 container start 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T04:19:12.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 podman[105677]: 2024-09-21 04:19:12.478660495 +0000 UTC m=+0.473789331 container attach 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T04:19:12.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:12.668 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 bash[105677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:12.668 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:12.668 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:12 smithi110 bash[105677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:12 smithi110 ceph-mon[92455]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:12 smithi148 ceph-mon[78701]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:15.228 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:14 smithi148 ceph-mon[78701]: pgmap v66: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:15.228 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:14 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 32/222 objects degraded (14.414%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:14 smithi110 ceph-mon[92455]: pgmap v66: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:14 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 32/222 objects degraded (14.414%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-09-21T04:19:15.823 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-09-21T04:19:15.824 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:19:15.824 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:19:16.152 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate[105692]: --> ceph-volume lvm activate successful for osd ID: 1 2024-09-21T04:19:16.153 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 bash[105677]: --> ceph-volume lvm activate successful for osd ID: 1 2024-09-21T04:19:16.154 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:15 smithi110 podman[105677]: 2024-09-21 04:19:15.849047003 +0000 UTC m=+3.844175837 container died 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T04:19:16.799 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 podman[105931]: 2024-09-21 04:19:16.466400744 +0000 UTC m=+0.610673006 container remove 863f9d055ad6b63ac006ab8d5d682d6a81c5697d33c547736ea732fddc161039 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T04:19:16.799 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 podman[105991]: 2024-09-21 04:19:16.575457781 +0000 UTC m=+0.015681234 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:16.799 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 podman[105991]: 2024-09-21 04:19:16.692285074 +0000 UTC m=+0.132508512 container create f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:19:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:16 smithi110 ceph-mon[92455]: pgmap v67: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:17.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 podman[105991]: 2024-09-21 04:19:16.878580967 +0000 UTC m=+0.318804414 container init f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T04:19:17.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 podman[105991]: 2024-09-21 04:19:16.88183722 +0000 UTC m=+0.322060659 container start f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-21T04:19:17.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 ceph-osd[106010]: -- 172.21.15.110:0/2064077446 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f9c5681e0 con 0x555f9b707c00 2024-09-21T04:19:17.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 bash[105991]: f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 2024-09-21T04:19:17.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:16 smithi110 systemd[1]: Started Ceph osd.1 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:19:17.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:16 smithi148 ceph-mon[78701]: pgmap v67: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:18.337 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:18.337 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:18.337 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:18 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:18.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:18.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:18.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:18 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:18.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:18 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:19:18.523+0000 7fdb52ffe640 -1 Falling back to public interface 2024-09-21T04:19:19.324 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:19 smithi110 ceph-mon[92455]: pgmap v68: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:19 smithi148 ceph-mon[78701]: pgmap v68: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:21.267 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:20 smithi110 ceph-mon[92455]: pgmap v69: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:21.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:20 smithi148 ceph-mon[78701]: pgmap v69: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:22 smithi110 ceph-mon[92455]: pgmap v70: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:22 smithi148 ceph-mon[78701]: pgmap v70: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:24.242 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:24.242 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:24.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:24.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:25.137 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:24 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:19:24.819+0000 7fdb52ffe640 -1 osd.1 54 log_to_monitors true 2024-09-21T04:19:25.403 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:25 smithi110 ceph-mon[92455]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T04:19:25.403 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:25 smithi110 ceph-mon[92455]: pgmap v71: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:25.403 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:19:25 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:19:25.160+0000 7fdb4ada8640 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:19:25.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:25 smithi148 ceph-mon[78701]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T04:19:25.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:25 smithi148 ceph-mon[78701]: pgmap v71: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: osdmap e57: 6 total, 5 up, 6 in 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:19:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:19:26.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:26 smithi110 ceph-mon[92455]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-21T04:19:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:19:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: osdmap e57: 6 total, 5 up, 6 in 2024-09-21T04:19:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:19:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:19:26.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:26 smithi148 ceph-mon[78701]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697] boot 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: osdmap e58: 6 total, 6 up, 6 in 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: pgmap v74: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:27 smithi110 ceph-mon[92455]: osdmap e59: 6 total, 6 up, 6 in 2024-09-21T04:19:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:19:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: osd.1 [v2:172.21.15.110:6810/662869697,v1:172.21.15.110:6811/662869697] boot 2024-09-21T04:19:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: osdmap e58: 6 total, 6 up, 6 in 2024-09-21T04:19:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:19:27.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: pgmap v74: 65 pgs: 21 active+undersized, 13 active+undersized+degraded, 31 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:19:27.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:27 smithi148 ceph-mon[78701]: osdmap e59: 6 total, 6 up, 6 in 2024-09-21T04:19:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:28 smithi110 ceph-mon[92455]: pgmap v76: 65 pgs: 5 active+undersized, 7 active+undersized+degraded, 53 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16/222 objects degraded (7.207%) 2024-09-21T04:19:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:28 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 16/222 objects degraded (7.207%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:28 smithi148 ceph-mon[78701]: pgmap v76: 65 pgs: 5 active+undersized, 7 active+undersized+degraded, 53 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16/222 objects degraded (7.207%) 2024-09-21T04:19:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:28 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 16/222 objects degraded (7.207%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:30 smithi110 ceph-mon[92455]: pgmap v77: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:30 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/222 objects degraded (7.207%), 7 pgs degraded) 2024-09-21T04:19:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:30 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:19:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:30 smithi148 ceph-mon[78701]: pgmap v77: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:30 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/222 objects degraded (7.207%), 7 pgs degraded) 2024-09-21T04:19:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:30 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:19:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:32 smithi110 ceph-mon[92455]: pgmap v78: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:32 smithi148 ceph-mon[78701]: pgmap v78: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:34 smithi110 ceph-mon[92455]: pgmap v79: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:34 smithi148 ceph-mon[78701]: pgmap v79: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:36.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 -- 172.21.15.110:0/4158290872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 msgr2=0x7fd914102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 --2- 172.21.15.110:0/4158290872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914102a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9100098b0 tx=0x7fd91002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 -- 172.21.15.110:0/4158290872 shutdown_connections 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 --2- 172.21.15.110:0/4158290872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 --2- 172.21.15.110:0/4158290872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914102a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 -- 172.21.15.110:0/4158290872 >> 172.21.15.110:0/4158290872 conn(0x7fd9140fa230 msgr2=0x7fd9140fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 -- 172.21.15.110:0/4158290872 shutdown_connections 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.042+0000 7fd91c303640 1 -- 172.21.15.110:0/4158290872 wait complete. 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 Processor -- start 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 -- start start 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914196150 con 0x7fd914100630 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91c303640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141962c0 con 0x7fd914102f60 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91a09f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91a09f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53844/0 (socket says 172.21.15.110:53844) 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.043+0000 7fd91a09f640 1 -- 172.21.15.110:0/3831424028 learned_addr learned my addr 172.21.15.110:0/3831424028 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91989e640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91a09f640 1 -- 172.21.15.110:0/3831424028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 msgr2=0x7fd914195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91a09f640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91a09f640 1 -- 172.21.15.110:0/3831424028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009560 con 0x7fd914100630 2024-09-21T04:19:36.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91989e640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91a09f640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd9100054f0 tx=0x7fd910005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd910004aa0 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd910004c00 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91419ad00 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd910003ac0 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.044+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91419b220 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.045+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9141087c0 con 0x7fd914100630 2024-09-21T04:19:36.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.046+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd910003d30 con 0x7fd914100630 2024-09-21T04:19:36.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.047+0000 7fd90b7fe640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 0x7fd90007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.047+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd9100bf570 con 0x7fd914100630 2024-09-21T04:19:36.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.047+0000 7fd91989e640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 0x7fd90007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.047+0000 7fd91989e640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 0x7fd90007d730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9040058c0 tx=0x7fd904005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.051+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd910082600 con 0x7fd914100630 2024-09-21T04:19:36.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.247+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9141038e0 con 0x7fd90007b270 2024-09-21T04:19:36.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.248+0000 7fd90b7fe640 1 -- 172.21.15.110:0/3831424028 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd9141038e0 con 0x7fd90007b270 2024-09-21T04:19:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 msgr2=0x7fd90007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 0x7fd90007d730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9040058c0 tx=0x7fd904005850 comp rx=0 tx=0).stop 2024-09-21T04:19:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 msgr2=0x7fd914195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd9100054f0 tx=0x7fd910005520 comp rx=0 tx=0).stop 2024-09-21T04:19:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 shutdown_connections 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd90007b270 0x7fd90007d730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914102f60 0x7fd914195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 --2- 172.21.15.110:0/3831424028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd914100630 0x7fd914195640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 >> 172.21.15.110:0/3831424028 conn(0x7fd9140fa230 msgr2=0x7fd9140fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 shutdown_connections 2024-09-21T04:19:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.250+0000 7fd91c303640 1 -- 172.21.15.110:0/3831424028 wait complete. 2024-09-21T04:19:36.262 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.333+0000 7f21a8821640 1 -- 172.21.15.110:0/3742832199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 msgr2=0x7f21a40fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.333+0000 7f21a8821640 1 --2- 172.21.15.110:0/3742832199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a40fee30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2194009880 tx=0x7f219402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 -- 172.21.15.110:0/3742832199 shutdown_connections 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 --2- 172.21.15.110:0/3742832199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 0x7f21a41000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 --2- 172.21.15.110:0/3742832199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a40fee30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 -- 172.21.15.110:0/3742832199 >> 172.21.15.110:0/3742832199 conn(0x7f21a40fa1c0 msgr2=0x7f21a40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 -- 172.21.15.110:0/3742832199 shutdown_connections 2024-09-21T04:19:36.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 -- 172.21.15.110:0/3742832199 wait complete. 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 Processor -- start 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.334+0000 7f21a8821640 1 -- start start 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 0x7f21a4194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4195040 con 0x7f21a40fea30 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41951b0 con 0x7f21a40ffc30 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53868/0 (socket says 172.21.15.110:53868) 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 -- 172.21.15.110:0/1342002116 learned_addr learned my addr 172.21.15.110:0/1342002116 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a1d9b640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 0x7f21a4194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 -- 172.21.15.110:0/1342002116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 msgr2=0x7f21a4194a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 0x7f21a4194a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 -- 172.21.15.110:0/1342002116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194009560 con 0x7f21a40fea30 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a259c640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2194004650 tx=0x7f21940055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219403e070 con 0x7f21a40fea30 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2194005d50 con 0x7f21a40fea30 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a4067be0 con 0x7f21a40fea30 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2194003820 con 0x7f21a40fea30 2024-09-21T04:19:36.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.335+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a4068150 con 0x7f21a40fea30 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.337+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4107640 con 0x7f21a40fea30 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.337+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f21940319d0 con 0x7f21a40fea30 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.338+0000 7f21937fe640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 0x7f218c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.338+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f21940c0670 con 0x7f21a40fea30 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.338+0000 7f21a1d9b640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 0x7f218c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.338+0000 7f21a1d9b640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 0x7f218c07d740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21a4195a50 tx=0x7f2198005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.342+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f2194083700 con 0x7f21a40fea30 2024-09-21T04:19:36.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.535+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21a40688f0 con 0x7f218c07b280 2024-09-21T04:19:36.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.536+0000 7f21937fe640 1 -- 172.21.15.110:0/1342002116 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f21a40688f0 con 0x7f218c07b280 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.538+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 msgr2=0x7f218c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.538+0000 7f21a8821640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 0x7f218c07d740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21a4195a50 tx=0x7f2198005f90 comp rx=0 tx=0).stop 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.538+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 msgr2=0x7f21a4194530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.538+0000 7f21a8821640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2194004650 tx=0x7f21940055a0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 shutdown_connections 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f218c07b280 0x7f218c07d740 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40ffc30 0x7f21a4194a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 --2- 172.21.15.110:0/1342002116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a40fea30 0x7f21a4194530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 >> 172.21.15.110:0/1342002116 conn(0x7f21a40fa1c0 msgr2=0x7f21a4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 shutdown_connections 2024-09-21T04:19:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.539+0000 7f21a8821640 1 -- 172.21.15.110:0/1342002116 wait complete. 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 -- 172.21.15.110:0/1621290035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 msgr2=0x7fadec0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 --2- 172.21.15.110:0/1621290035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec0fff90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faddc009880 tx=0x7faddc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 -- 172.21.15.110:0/1621290035 shutdown_connections 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 --2- 172.21.15.110:0/1621290035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 0x7fadec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 --2- 172.21.15.110:0/1621290035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec0fff90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 -- 172.21.15.110:0/1621290035 >> 172.21.15.110:0/1621290035 conn(0x7fadec0fb320 msgr2=0x7fadec0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 -- 172.21.15.110:0/1621290035 shutdown_connections 2024-09-21T04:19:36.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.620+0000 7fadf2475640 1 -- 172.21.15.110:0/1621290035 wait complete. 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 Processor -- start 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 -- start start 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 0x7fadec195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec196150 con 0x7fadec0ffb90 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadf2475640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1962c0 con 0x7fadec100d90 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadebfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadebfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53886/0 (socket says 172.21.15.110:53886) 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.621+0000 7fadebfff640 1 -- 172.21.15.110:0/1987055176 learned_addr learned my addr 172.21.15.110:0/1987055176 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadeb7fe640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 0x7fadec195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadebfff640 1 -- 172.21.15.110:0/1987055176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 msgr2=0x7fadec195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadebfff640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 0x7fadec195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadebfff640 1 -- 172.21.15.110:0/1987055176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc009560 con 0x7fadec0ffb90 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadebfff640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faddc005610 tx=0x7faddc005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faddc004ba0 con 0x7fadec0ffb90 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faddc004d00 con 0x7fadec0ffb90 2024-09-21T04:19:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec19acb0 con 0x7fadec0ffb90 2024-09-21T04:19:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faddc003bd0 con 0x7fadec0ffb90 2024-09-21T04:19:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.622+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec19b1a0 con 0x7fadec0ffb90 2024-09-21T04:19:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.623+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec18f730 con 0x7fadec0ffb90 2024-09-21T04:19:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.624+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7faddc031640 con 0x7fadec0ffb90 2024-09-21T04:19:36.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.624+0000 7fade97fa640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 0x7fadd407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.625+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faddc0bf690 con 0x7fadec0ffb90 2024-09-21T04:19:36.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.625+0000 7fadeb7fe640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 0x7fadd407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.625+0000 7fadeb7fe640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 0x7fadd407d730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadec196b10 tx=0x7fade0005c60 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.629+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7faddc0826a0 con 0x7fadec0ffb90 2024-09-21T04:19:36.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.815+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fadec02c4f0 con 0x7fadd407b270 2024-09-21T04:19:36.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.824+0000 7fade97fa640 1 -- 172.21.15.110:0/1987055176 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fadec02c4f0 con 0x7fadd407b270 2024-09-21T04:19:36.827 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:19:36.827 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (10m) 14s ago 12m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:19:36.827 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (3m) 14s ago 13m 8132k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (3m) 76s ago 12m 7369k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (84s) 14s ago 13m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (80s) 76s ago 12m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (10m) 14s ago 12m 85.2M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (6m) 14s ago 6m 154M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (6m) 14s ago 6m 16.2M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (6m) 76s ago 6m 17.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (6m) 76s ago 6m 18.8M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (3m) 14s ago 14m 511M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (2m) 76s ago 11m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (2m) 14s ago 14m 46.2M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (2m) 76s ago 11m 34.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (12m) 14s ago 12m 17.0M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:19:36.828 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (12m) 76s ago 12m 15.3M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (57s) 14s ago 10m 102M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (20s) 14s ago 10m 12.2M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (9m) 14s ago 9m 230M 1216M 18.2.4-871-g6a1bdf85 0adab276c97e 6ce0dccd7801 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (9m) 76s ago 9m 326M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (8m) 76s ago 8m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (8m) 76s ago 8m 254M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:19:36.829 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 14s ago 12m 56.1M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:19:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 msgr2=0x7fadd407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 0x7fadd407d730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadec196b10 tx=0x7fade0005c60 comp rx=0 tx=0).stop 2024-09-21T04:19:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 msgr2=0x7fadec195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faddc005610 tx=0x7faddc005640 comp rx=0 tx=0).stop 2024-09-21T04:19:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 shutdown_connections 2024-09-21T04:19:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fadd407b270 0x7fadd407d730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec100d90 0x7fadec195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 --2- 172.21.15.110:0/1987055176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadec0ffb90 0x7fadec195640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 >> 172.21.15.110:0/1987055176 conn(0x7fadec0fb320 msgr2=0x7fadec104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 shutdown_connections 2024-09-21T04:19:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.827+0000 7fadf2475640 1 -- 172.21.15.110:0/1987055176 wait complete. 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.908+0000 7fb35c3f3640 1 -- 172.21.15.110:0/2003609954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540ffc30 msgr2=0x7fb3541000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.908+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/2003609954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540ffc30 0x7fb3541000b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb344009880 tx=0x7fb34402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 -- 172.21.15.110:0/2003609954 shutdown_connections 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/2003609954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540ffc30 0x7fb3541000b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/2003609954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540fea30 0x7fb3540fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 -- 172.21.15.110:0/2003609954 >> 172.21.15.110:0/2003609954 conn(0x7fb3540fa1c0 msgr2=0x7fb3540fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 -- 172.21.15.110:0/2003609954 shutdown_connections 2024-09-21T04:19:36.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 -- 172.21.15.110:0/2003609954 wait complete. 2024-09-21T04:19:36.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 Processor -- start 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.909+0000 7fb35c3f3640 1 -- start start 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35c3f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35c3f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540ffc30 0x7fb354195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35c3f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb354196160 con 0x7fb3540fea30 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35c3f3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541962d0 con 0x7fb3540ffc30 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53904/0 (socket says 172.21.15.110:53904) 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 -- 172.21.15.110:0/1146084457 learned_addr learned my addr 172.21.15.110:0/1146084457 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35998e640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540ffc30 0x7fb354195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 -- 172.21.15.110:0/1146084457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540ffc30 msgr2=0x7fb354195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540ffc30 0x7fb354195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:36.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 -- 172.21.15.110:0/1146084457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344009560 con 0x7fb3540fea30 2024-09-21T04:19:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35a18f640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c920 tx=0x7fb35000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3500041f0 con 0x7fb3540fea30 2024-09-21T04:19:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb350007e70 con 0x7fb3540fea30 2024-09-21T04:19:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.910+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35419ad70 con 0x7fb3540fea30 2024-09-21T04:19:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.911+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35419b2b0 con 0x7fb3540fea30 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.911+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3500049d0 con 0x7fb3540fea30 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.912+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb354107640 con 0x7fb3540fea30 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.912+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb350004350 con 0x7fb3540fea30 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.913+0000 7fb34b7fe640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 0x7fb34007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.913+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb35009e070 con 0x7fb3540fea30 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.913+0000 7fb35998e640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 0x7fb34007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:36.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.913+0000 7fb35998e640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 0x7fb34007d740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb344000c00 tx=0x7fb344004400 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:36.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:36.917+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb350061100 con 0x7fb3540fea30 2024-09-21T04:19:37.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:36 smithi110 ceph-mon[92455]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:36 smithi110 ceph-mon[92455]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:36 smithi110 ceph-mon[92455]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:36 smithi110 ceph-mon[92455]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:37.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.171+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb35419b550 con 0x7fb3540fea30 2024-09-21T04:19:37.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.172+0000 7fb34b7fe640 1 -- 172.21.15.110:0/1146084457 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fb350064220 con 0x7fb3540fea30 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4, 2024-09-21T04:19:37.175 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 8, 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:19:37.176 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 msgr2=0x7fb34007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 0x7fb34007d740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb344000c00 tx=0x7fb344004400 comp rx=0 tx=0).stop 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 msgr2=0x7fb354195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c920 tx=0x7fb35000cde0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 shutdown_connections 2024-09-21T04:19:37.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb34007b280 0x7fb34007d740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3540ffc30 0x7fb354195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 --2- 172.21.15.110:0/1146084457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3540fea30 0x7fb354195650 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 >> 172.21.15.110:0/1146084457 conn(0x7fb3540fa1c0 msgr2=0x7fb354067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 shutdown_connections 2024-09-21T04:19:37.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.174+0000 7fb35c3f3640 1 -- 172.21.15.110:0/1146084457 wait complete. 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 -- 172.21.15.110:0/2906218775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 msgr2=0x7ff528101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 --2- 172.21.15.110:0/2906218775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528101680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009880 tx=0x7ff51c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 -- 172.21.15.110:0/2906218775 shutdown_connections 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 --2- 172.21.15.110:0/2906218775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 0x7ff5280ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 --2- 172.21.15.110:0/2906218775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528101680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 -- 172.21.15.110:0/2906218775 >> 172.21.15.110:0/2906218775 conn(0x7ff5280fb330 msgr2=0x7ff5280fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 -- 172.21.15.110:0/2906218775 shutdown_connections 2024-09-21T04:19:37.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.255+0000 7ff52ee90640 1 -- 172.21.15.110:0/2906218775 wait complete. 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 Processor -- start 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 -- start start 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 0x7ff528195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5281962a0 con 0x7ff528101280 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52ee90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff528196410 con 0x7ff5280ff7c0 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53922/0 (socket says 172.21.15.110:53922) 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 -- 172.21.15.110:0/734265747 learned_addr learned my addr 172.21.15.110:0/734265747 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:37.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff52cc2c640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 0x7ff528195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 -- 172.21.15.110:0/734265747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 msgr2=0x7ff528195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 0x7ff528195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 -- 172.21.15.110:0/734265747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009560 con 0x7ff528101280 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.256+0000 7ff527fff640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d900 tx=0x7ff51400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.257+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff514009c00 con 0x7ff528101280 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.257+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff514009d60 con 0x7ff528101280 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.257+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52819aeb0 con 0x7ff528101280 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.257+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5140049d0 con 0x7ff528101280 2024-09-21T04:19:37.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.257+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52819b3d0 con 0x7ff528101280 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.258+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5280ffc40 con 0x7ff528101280 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.258+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7ff514004020 con 0x7ff528101280 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.259+0000 7ff525ffb640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 0x7ff51807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.259+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff51409df30 con 0x7ff528101280 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.259+0000 7ff52cc2c640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 0x7ff51807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.259+0000 7ff52cc2c640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 0x7ff51807d730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005940 tx=0x7ff51c0058d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.263+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7ff5140d5710 con 0x7ff528101280 2024-09-21T04:19:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:36 smithi148 ceph-mon[78701]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:36 smithi148 ceph-mon[78701]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:36 smithi148 ceph-mon[78701]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:36 smithi148 ceph-mon[78701]: pgmap v80: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:37.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.477+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff52818f810 con 0x7ff528101280 2024-09-21T04:19:37.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.478+0000 7ff525ffb640 1 -- 172.21.15.110:0/734265747 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7ff514017790 con 0x7ff528101280 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:19:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:19:37.482 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:19:37.483 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 msgr2=0x7ff51807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 0x7ff51807d730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005940 tx=0x7ff51c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 msgr2=0x7ff528195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d900 tx=0x7ff51400ddc0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 shutdown_connections 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff51807b270 0x7ff51807d730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff528101280 0x7ff528195cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 --2- 172.21.15.110:0/734265747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5280ff7c0 0x7ff528195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 >> 172.21.15.110:0/734265747 conn(0x7ff5280fb330 msgr2=0x7ff528104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 shutdown_connections 2024-09-21T04:19:37.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.481+0000 7ff52ee90640 1 -- 172.21.15.110:0/734265747 wait complete. 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.562+0000 7fd954799640 1 -- 172.21.15.110:0/2193409267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c100d90 msgr2=0x7fd94c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.562+0000 7fd954799640 1 --2- 172.21.15.110:0/2193409267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c100d90 0x7fd94c101210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009880 tx=0x7fd93c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- 172.21.15.110:0/2193409267 shutdown_connections 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 --2- 172.21.15.110:0/2193409267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c100d90 0x7fd94c101210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 --2- 172.21.15.110:0/2193409267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c0ffb90 0x7fd94c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- 172.21.15.110:0/2193409267 >> 172.21.15.110:0/2193409267 conn(0x7fd94c0fb320 msgr2=0x7fd94c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- 172.21.15.110:0/2193409267 shutdown_connections 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- 172.21.15.110:0/2193409267 wait complete. 2024-09-21T04:19:37.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 Processor -- start 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- start start 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 0x7fd94c195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c196120 con 0x7fd94c0ffb90 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.563+0000 7fd954799640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c196290 con 0x7fd94c100d90 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53930/0 (socket says 172.21.15.110:53930) 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 -- 172.21.15.110:0/3452873268 learned_addr learned my addr 172.21.15.110:0/3452873268 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd951d34640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 0x7fd94c195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 -- 172.21.15.110:0/3452873268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 msgr2=0x7fd94c195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 0x7fd94c195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 -- 172.21.15.110:0/3452873268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009560 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd951d34640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 0x7fd94c195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd952535640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e970 tx=0x7fd94800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9480096e0 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9480044f0 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c19ad30 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd948010430 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.564+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c19b2a0 con 0x7fd94c0ffb90 2024-09-21T04:19:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.565+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c18f730 con 0x7fd94c0ffb90 2024-09-21T04:19:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.566+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd94800cc90 con 0x7fd94c0ffb90 2024-09-21T04:19:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.566+0000 7fd9437fe640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 0x7fd93807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.566+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd948014070 con 0x7fd94c0ffb90 2024-09-21T04:19:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.566+0000 7fd951d34640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 0x7fd93807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.567+0000 7fd951d34640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 0x7fd93807d730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd93c031540 tx=0x7fd93c005de0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.571+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd948061860 con 0x7fd94c0ffb90 2024-09-21T04:19:37.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.765+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd94c02c550 con 0x7fd93807b270 2024-09-21T04:19:37.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.767+0000 7fd9437fe640 1 -- 172.21.15.110:0/3452873268 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd94c02c550 con 0x7fd93807b270 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:19:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "8/23 daemons upgraded", 2024-09-21T04:19:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:19:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:19:37.771 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:19:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 msgr2=0x7fd93807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 0x7fd93807d730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd93c031540 tx=0x7fd93c005de0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 msgr2=0x7fd94c195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd94800e970 tx=0x7fd94800ee30 comp rx=0 tx=0).stop 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 shutdown_connections 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd93807b270 0x7fd93807d730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd94c100d90 0x7fd94c195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 --2- 172.21.15.110:0/3452873268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd94c0ffb90 0x7fd94c195610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 >> 172.21.15.110:0/3452873268 conn(0x7fd94c0fb320 msgr2=0x7fd94c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 shutdown_connections 2024-09-21T04:19:37.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.769+0000 7fd954799640 1 -- 172.21.15.110:0/3452873268 wait complete. 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.850+0000 7f971a5e1640 1 -- 172.21.15.110:0/3417139435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 msgr2=0x7f97140fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.850+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3417139435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f97140fee30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9704009880 tx=0x7f970402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 -- 172.21.15.110:0/3417139435 shutdown_connections 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3417139435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 0x7f97141000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3417139435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f97140fee30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 -- 172.21.15.110:0/3417139435 >> 172.21.15.110:0/3417139435 conn(0x7f97140fa1c0 msgr2=0x7f97140fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:37.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 -- 172.21.15.110:0/3417139435 shutdown_connections 2024-09-21T04:19:37.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 -- 172.21.15.110:0/3417139435 wait complete. 2024-09-21T04:19:37.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 Processor -- start 2024-09-21T04:19:37.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 -- start start 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.851+0000 7f971a5e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f971a5e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 0x7f971419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f971a5e1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971419a960 con 0x7f97140fea30 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f971a5e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971419aaa0 con 0x7f97140ffc30 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53938/0 (socket says 172.21.15.110:53938) 2024-09-21T04:19:37.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 -- 172.21.15.110:0/3080455706 learned_addr learned my addr 172.21.15.110:0/3080455706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f97137fe640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 0x7f971419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 -- 172.21.15.110:0/3080455706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 msgr2=0x7f971419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 0x7f971419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 -- 172.21.15.110:0/3080455706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9704009560 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f9713fff640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f970402a9c0 tx=0x7f97040054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9704004aa0 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971419acf0 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9704004c00 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9704003a80 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.852+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971419b260 con 0x7f97140fea30 2024-09-21T04:19:37.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.853+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9714107640 con 0x7f97140fea30 2024-09-21T04:19:37.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.854+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f9704003d40 con 0x7f97140fea30 2024-09-21T04:19:37.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.854+0000 7f97117fa640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 0x7f96fc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:19:37.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.855+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f97040bf570 con 0x7f97140fea30 2024-09-21T04:19:37.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.855+0000 7f97137fe640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 0x7f96fc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:19:37.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.855+0000 7f97137fe640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 0x7f96fc07d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9708005fd0 tx=0x7f9708005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:19:37.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:37.859+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f9704082580 con 0x7f97140fea30 2024-09-21T04:19:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.109+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f971419b710 con 0x7f97140fea30 2024-09-21T04:19:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:37 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1146084457' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:37 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/734265747' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:19:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:37 smithi110 ceph-mon[92455]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:38.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.110+0000 7f97117fa640 1 -- 172.21.15.110:0/3080455706 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f97040856a0 con 0x7f97140fea30 2024-09-21T04:19:38.113 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 msgr2=0x7f96fc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 0x7f96fc07d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9708005fd0 tx=0x7f9708005850 comp rx=0 tx=0).stop 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 msgr2=0x7f9714068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f970402a9c0 tx=0x7f97040054f0 comp rx=0 tx=0).stop 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 shutdown_connections 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f96fc07b280 0x7f96fc07d740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97140ffc30 0x7f971419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 --2- 172.21.15.110:0/3080455706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97140fea30 0x7f9714068740 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:19:38.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 >> 172.21.15.110:0/3080455706 conn(0x7f97140fa1c0 msgr2=0x7f9714102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:19:38.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 shutdown_connections 2024-09-21T04:19:38.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:19:38.112+0000 7f971a5e1640 1 -- 172.21.15.110:0/3080455706 wait complete. 2024-09-21T04:19:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:37 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1146084457' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:19:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:37 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/734265747' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:19:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:37 smithi148 ceph-mon[78701]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:19:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:38 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3080455706' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:19:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:38 smithi110 ceph-mon[92455]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:38 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3080455706' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:19:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:38 smithi148 ceph-mon[78701]: pgmap v81: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:40.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:40 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:19:41.107 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:40 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:19:42.053 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:41 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:19:42.053 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:41 smithi110 ceph-mon[92455]: Upgrade: osd.2 is safe to restart 2024-09-21T04:19:42.053 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:41 smithi110 ceph-mon[92455]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:42.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:41 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:19:42.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:41 smithi148 ceph-mon[78701]: Upgrade: osd.2 is safe to restart 2024-09-21T04:19:42.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:41 smithi148 ceph-mon[78701]: pgmap v82: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: Upgrade: Updating osd.2 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: Deploying daemon osd.2 on smithi110 2024-09-21T04:19:43.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:43 smithi110 ceph-mon[92455]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: Upgrade: Updating osd.2 2024-09-21T04:19:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:19:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:19:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: Deploying daemon osd.2 on smithi110 2024-09-21T04:19:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:43 smithi148 ceph-mon[78701]: pgmap v83: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:45.222 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:44 smithi110 ceph-mon[92455]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:45.222 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:44 smithi110 systemd[1]: Stopping Ceph osd.2 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:19:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:44 smithi148 ceph-mon[78701]: pgmap v84: 65 pgs: 65 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:45.667 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:45 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[64226]: 2024-09-21T04:19:45.218+0000 7f83f4450640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:19:45.667 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:45 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[64226]: 2024-09-21T04:19:45.218+0000 7f83f4450640 -1 osd.2 59 *** Got signal Terminated *** 2024-09-21T04:19:45.667 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:45 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[64226]: 2024-09-21T04:19:45.218+0000 7f83f4450640 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:19:46.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:45 smithi110 ceph-mon[92455]: osd.2 marked itself down and dead 2024-09-21T04:19:46.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:45 smithi148 ceph-mon[78701]: osd.2 marked itself down and dead 2024-09-21T04:19:46.714 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:46 smithi110 podman[111122]: 2024-09-21 04:19:46.440327306 +0000 UTC m=+1.342740338 container died 6ce0dccd78012ef2ab2a8b8ac677222d1b3f4076aa57592f3c725455293da680 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=reef-6a1bdf8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T04:19:47.131 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:46 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:19:47.131 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:46 smithi110 ceph-mon[92455]: osdmap e60: 6 total, 5 up, 6 in 2024-09-21T04:19:47.131 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:46 smithi110 ceph-mon[92455]: pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:47.131 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:46 smithi110 podman[111122]: 2024-09-21 04:19:46.810393204 +0000 UTC m=+1.712806236 container cleanup 6ce0dccd78012ef2ab2a8b8ac677222d1b3f4076aa57592f3c725455293da680 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-6a1bdf8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T04:19:47.131 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:46 smithi110 bash[111122]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2 2024-09-21T04:19:47.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111148]: 2024-09-21 04:19:47.128882009 +0000 UTC m=+0.686771082 container remove 6ce0dccd78012ef2ab2a8b8ac677222d1b3f4076aa57592f3c725455293da680 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=reef-6a1bdf8, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:19:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:46 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:19:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:46 smithi148 ceph-mon[78701]: osdmap e60: 6 total, 5 up, 6 in 2024-09-21T04:19:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:46 smithi148 ceph-mon[78701]: pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:19:47.759 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.361677344 +0000 UTC m=+0.016024325 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:47.759 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.463232646 +0000 UTC m=+0.117579609 container create ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:19:47.760 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.682305724 +0000 UTC m=+0.336652693 container init ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:19:47.760 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.685580581 +0000 UTC m=+0.339927552 container start ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:19:47.760 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.757323618 +0000 UTC m=+0.411670586 container attach ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:19:48.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:47 smithi110 ceph-mon[92455]: osdmap e61: 6 total, 5 up, 6 in 2024-09-21T04:19:48.167 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:47 smithi110 podman[111236]: 2024-09-21 04:19:47.87906928 +0000 UTC m=+0.533416254 container died ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T04:19:48.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:47 smithi148 ceph-mon[78701]: osdmap e61: 6 total, 5 up, 6 in 2024-09-21T04:19:48.800 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:48 smithi110 podman[111259]: 2024-09-21 04:19:48.545137861 +0000 UTC m=+0.660406528 container remove ef7b7900c1cd8d4f377ad9a66064af1c55da261ae342328e71098f88ca6fa071 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T04:19:48.800 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:48 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.2.service: Deactivated successfully. 2024-09-21T04:19:48.800 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:48 smithi110 systemd[1]: Stopped Ceph osd.2 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:19:48.800 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:48 smithi110 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.2.service: Consumed 55.873s CPU time. 2024-09-21T04:19:49.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:48 smithi110 ceph-mon[92455]: pgmap v88: 65 pgs: 12 active+undersized, 4 stale+active+clean, 8 active+undersized+degraded, 41 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17/222 objects degraded (7.658%) 2024-09-21T04:19:49.143 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:48 smithi110 systemd[1]: Starting Ceph osd.2 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:19:49.143 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 podman[111357]: 2024-09-21 04:19:48.911750769 +0000 UTC m=+0.015934455 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:49.143 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 podman[111357]: 2024-09-21 04:19:49.038442097 +0000 UTC m=+0.142625776 container create 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T04:19:49.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 podman[111357]: 2024-09-21 04:19:49.240990921 +0000 UTC m=+0.345174601 container init 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:19:49.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 podman[111357]: 2024-09-21 04:19:49.244449578 +0000 UTC m=+0.348633260 container start 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T04:19:49.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 podman[111357]: 2024-09-21 04:19:49.332663326 +0000 UTC m=+0.436847008 container attach 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:19:49.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:49.418 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 bash[111357]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:49.418 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:49.418 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:49 smithi110 bash[111357]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:49.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:48 smithi148 ceph-mon[78701]: pgmap v88: 65 pgs: 12 active+undersized, 4 stale+active+clean, 8 active+undersized+degraded, 41 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17/222 objects degraded (7.658%) 2024-09-21T04:19:50.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:49 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 17/222 objects degraded (7.658%), 8 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:50.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:49 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 17/222 objects degraded (7.658%), 8 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:51.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:50 smithi110 ceph-mon[92455]: pgmap v89: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:51.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:50 smithi148 ceph-mon[78701]: pgmap v89: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:19:52.645 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:19:52.646 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:19:52.646 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-09-21T04:19:52.646 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-09-21T04:19:52.646 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:19:52.646 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:19:52.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate[111372]: --> ceph-volume lvm activate successful for osd ID: 2 2024-09-21T04:19:52.918 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 bash[111357]: --> ceph-volume lvm activate successful for osd ID: 2 2024-09-21T04:19:52.918 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:52 smithi110 podman[111357]: 2024-09-21 04:19:52.671494415 +0000 UTC m=+3.775678096 container died 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T04:19:53.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:52 smithi110 ceph-mon[92455]: pgmap v90: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:53.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:52 smithi148 ceph-mon[78701]: pgmap v90: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:53.588 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 podman[111611]: 2024-09-21 04:19:53.270203802 +0000 UTC m=+0.586973729 container remove 1f86367de7f047f6f6221719c063d77fc5fdbeab25e76decfd9489dd813bae6c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-activate, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:19:53.588 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 podman[111671]: 2024-09-21 04:19:53.378533455 +0000 UTC m=+0.015747985 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:19:53.588 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 podman[111671]: 2024-09-21 04:19:53.479687814 +0000 UTC m=+0.116902377 container create 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T04:19:54.137 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 podman[111671]: 2024-09-21 04:19:53.682328876 +0000 UTC m=+0.319543400 container init 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T04:19:54.137 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 podman[111671]: 2024-09-21 04:19:53.685606064 +0000 UTC m=+0.322820581 container start 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:19:54.137 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 ceph-osd[111691]: -- 172.21.15.110:0/1339260368 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa110261e0 con 0x55fa10fb6000 2024-09-21T04:19:54.137 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 bash[111671]: 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c 2024-09-21T04:19:54.137 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:53 smithi110 systemd[1]: Started Ceph osd.2 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:19:54.343 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.343 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:54.343 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.343 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.343 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:19:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:19:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:19:55.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:55 smithi110 ceph-mon[92455]: pgmap v91: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:55.366 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:19:55 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:19:55.082+0000 7f82e512b640 -1 Falling back to public interface 2024-09-21T04:19:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:55 smithi148 ceph-mon[78701]: pgmap v91: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:56.747 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:56 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:56 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded (PG_DEGRADED) 2024-09-21T04:19:57.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:57 smithi110 ceph-mon[92455]: pgmap v92: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:19:57.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:57 smithi148 ceph-mon[78701]: pgmap v92: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:00.010 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:59 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.010 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:59 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.010 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:59 smithi110 ceph-mon[92455]: pgmap v93: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:00.010 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:59 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.011 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:19:59 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:59 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:59 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:59 smithi148 ceph-mon[78701]: pgmap v93: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:00.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:59 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:19:59 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:00.991 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:20:00.782+0000 7f82e512b640 -1 osd.2 59 log_to_monitors true 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: [WRN] OSD_DOWN: 1 osds down 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: osd.2 (root=default,host=smithi110) is down 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: [WRN] PG_DEGRADED: Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-21T04:20:00.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.c is active+undersized+degraded, acting [3,0] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.e is active+undersized+degraded, acting [0,3] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-21T04:20:00.992 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:00 smithi110 ceph-mon[92455]: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-09-21T04:20:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded 2024-09-21T04:20:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: [WRN] OSD_DOWN: 1 osds down 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: osd.2 (root=default,host=smithi110) is down 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: [WRN] PG_DEGRADED: Degraded data redundancy: 31/222 objects degraded (13.964%), 11 pgs degraded 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.c is active+undersized+degraded, acting [3,0] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.e is active+undersized+degraded, acting [0,3] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-21T04:20:01.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:00 smithi148 ceph-mon[78701]: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-09-21T04:20:02.140 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:01 smithi110 ceph-mon[92455]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T04:20:02.140 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:01 smithi110 ceph-mon[92455]: pgmap v94: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:02.140 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:20:01 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:20:01.776+0000 7f82dced5640 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:20:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:01 smithi148 ceph-mon[78701]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T04:20:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:01 smithi148 ceph-mon[78701]: pgmap v94: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: osdmap e62: 6 total, 5 up, 6 in 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:20:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:20:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: osdmap e62: 6 total, 5 up, 6 in 2024-09-21T04:20:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T04:20:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:03.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336] boot 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: osdmap e63: 6 total, 6 up, 6 in 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:20:04.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:03 smithi110 ceph-mon[92455]: pgmap v97: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: osd.2 [v2:172.21.15.110:6818/4043062336,v1:172.21.15.110:6819/4043062336] boot 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: osdmap e63: 6 total, 6 up, 6 in 2024-09-21T04:20:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:20:04.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:03 smithi148 ceph-mon[78701]: pgmap v97: 65 pgs: 16 active+undersized, 11 active+undersized+degraded, 38 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/222 objects degraded (13.964%) 2024-09-21T04:20:05.142 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:04 smithi110 ceph-mon[92455]: osdmap e64: 6 total, 6 up, 6 in 2024-09-21T04:20:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:04 smithi148 ceph-mon[78701]: osdmap e64: 6 total, 6 up, 6 in 2024-09-21T04:20:06.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:05 smithi110 ceph-mon[92455]: pgmap v99: 65 pgs: 5 peering, 6 active+undersized, 6 active+undersized+degraded, 48 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/222 objects degraded (9.459%) 2024-09-21T04:20:06.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:05 smithi148 ceph-mon[78701]: pgmap v99: 65 pgs: 5 peering, 6 active+undersized, 6 active+undersized+degraded, 48 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/222 objects degraded (9.459%) 2024-09-21T04:20:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:06 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 21/222 objects degraded (9.459%), 6 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:07.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:06 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 21/222 objects degraded (9.459%), 6 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:07 smithi110 ceph-mon[92455]: pgmap v100: 65 pgs: 5 peering, 60 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:08.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:07 smithi148 ceph-mon[78701]: pgmap v100: 65 pgs: 5 peering, 60 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:08.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.209+0000 7fd6b29a8640 1 -- 172.21.15.110:0/1871721261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 msgr2=0x7fd6ac102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.209+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/1871721261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac102a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009880 tx=0x7fd69c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 -- 172.21.15.110:0/1871721261 shutdown_connections 2024-09-21T04:20:08.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/1871721261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac1053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/1871721261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac102a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 -- 172.21.15.110:0/1871721261 >> 172.21.15.110:0/1871721261 conn(0x7fd6ac0fa230 msgr2=0x7fd6ac0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:08.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 -- 172.21.15.110:0/1871721261 shutdown_connections 2024-09-21T04:20:08.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 -- 172.21.15.110:0/1871721261 wait complete. 2024-09-21T04:20:08.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 Processor -- start 2024-09-21T04:20:08.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.210+0000 7fd6b29a8640 1 -- start start 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6b29a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6b29a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6b29a8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1960e0 con 0x7fd6ac100690 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6b29a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac196250 con 0x7fd6ac102fc0 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49856/0 (socket says 172.21.15.110:49856) 2024-09-21T04:20:08.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 -- 172.21.15.110:0/2927143224 learned_addr learned my addr 172.21.15.110:0/2927143224 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6ab7fe640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 -- 172.21.15.110:0/2927143224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 msgr2=0x7fd6ac195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 -- 172.21.15.110:0/2927143224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009560 con 0x7fd6ac100690 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6ab7fe640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.211+0000 7fd6abfff640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005640 tx=0x7fd69c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.212+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c004ba0 con 0x7fd6ac100690 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.212+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac19acf0 con 0x7fd6ac100690 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.212+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd69c004d00 con 0x7fd6ac100690 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.212+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c003c30 con 0x7fd6ac100690 2024-09-21T04:20:08.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.212+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac19b260 con 0x7fd6ac100690 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.213+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac108770 con 0x7fd6ac100690 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.213+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd69c031640 con 0x7fd6ac100690 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.214+0000 7fd6a97fa640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 0x7fd69407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.214+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd69c0bf5b0 con 0x7fd6ac100690 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.214+0000 7fd6ab7fe640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 0x7fd69407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.214+0000 7fd6ab7fe640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 0x7fd69407d730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005fd0 tx=0x7fd6a0005ea0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.218+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd69c0825c0 con 0x7fd6ac100690 2024-09-21T04:20:08.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.411+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ac19b480 con 0x7fd69407b270 2024-09-21T04:20:08.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.412+0000 7fd6a97fa640 1 -- 172.21.15.110:0/2927143224 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd6ac19b480 con 0x7fd69407b270 2024-09-21T04:20:08.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 msgr2=0x7fd69407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 0x7fd69407d730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005fd0 tx=0x7fd6a0005ea0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 msgr2=0x7fd6ac1955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005640 tx=0x7fd69c005670 comp rx=0 tx=0).stop 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 shutdown_connections 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd69407b270 0x7fd69407d730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6ac102fc0 0x7fd6ac195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 --2- 172.21.15.110:0/2927143224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6ac100690 0x7fd6ac1955d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 >> 172.21.15.110:0/2927143224 conn(0x7fd6ac0fa230 msgr2=0x7fd6ac0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 shutdown_connections 2024-09-21T04:20:08.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.414+0000 7fd6b29a8640 1 -- 172.21.15.110:0/2927143224 wait complete. 2024-09-21T04:20:08.427 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:20:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 -- 172.21.15.110:0/4201191190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 msgr2=0x7fbfbc102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 --2- 172.21.15.110:0/4201191190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc102a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098b0 tx=0x7fbfb802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 -- 172.21.15.110:0/4201191190 shutdown_connections 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 --2- 172.21.15.110:0/4201191190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 0x7fbfbc105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 --2- 172.21.15.110:0/4201191190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc102a20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 -- 172.21.15.110:0/4201191190 >> 172.21.15.110:0/4201191190 conn(0x7fbfbc0fa230 msgr2=0x7fbfbc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.498+0000 7fbfc4364640 1 -- 172.21.15.110:0/4201191190 shutdown_connections 2024-09-21T04:20:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 -- 172.21.15.110:0/4201191190 wait complete. 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 Processor -- start 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 -- start start 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 0x7fbfbc195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc196290 con 0x7fbfbc100630 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.499+0000 7fbfc4364640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc196400 con 0x7fbfbc102f60 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49880/0 (socket says 172.21.15.110:49880) 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 -- 172.21.15.110:0/2982436507 learned_addr learned my addr 172.21.15.110:0/2982436507 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc18ff640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 0x7fbfbc195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 -- 172.21.15.110:0/2982436507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 msgr2=0x7fbfbc195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 0x7fbfbc195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 -- 172.21.15.110:0/2982436507 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009560 con 0x7fbfbc100630 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc2100640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005610 tx=0x7fbfb8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb8004ba0 con 0x7fbfbc100630 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfb8004d00 con 0x7fbfbc100630 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc19adf0 con 0x7fbfbc100630 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb8003bd0 con 0x7fbfbc100630 2024-09-21T04:20:08.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.500+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc19b2d0 con 0x7fbfbc100630 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.501+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc108830 con 0x7fbfbc100630 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.502+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fbfb8031640 con 0x7fbfbc100630 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.502+0000 7fbfaf7fe640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 0x7fbfa807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.503+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbfb80bf590 con 0x7fbfbc100630 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.503+0000 7fbfc18ff640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 0x7fbfa807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.503+0000 7fbfc18ff640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 0x7fbfa807d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc196c50 tx=0x7fbfb0009110 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.507+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fbfb80825a0 con 0x7fbfbc100630 2024-09-21T04:20:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.705+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfbc19b4b0 con 0x7fbfa807b270 2024-09-21T04:20:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.706+0000 7fbfaf7fe640 1 -- 172.21.15.110:0/2982436507 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbfbc19b4b0 con 0x7fbfa807b270 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 msgr2=0x7fbfa807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 0x7fbfa807d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc196c50 tx=0x7fbfb0009110 comp rx=0 tx=0).stop 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 msgr2=0x7fbfbc195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005610 tx=0x7fbfb8005640 comp rx=0 tx=0).stop 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 shutdown_connections 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbfa807b270 0x7fbfa807d730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfbc102f60 0x7fbfbc195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 --2- 172.21.15.110:0/2982436507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbfbc100630 0x7fbfbc195780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 >> 172.21.15.110:0/2982436507 conn(0x7fbfbc0fa230 msgr2=0x7fbfbc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 shutdown_connections 2024-09-21T04:20:08.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.709+0000 7fbfc4364640 1 -- 172.21.15.110:0/2982436507 wait complete. 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 -- 172.21.15.110:0/3987904002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0ffca0 msgr2=0x7f87ec100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/3987904002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0ffca0 0x7f87ec100120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009880 tx=0x7f87dc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 -- 172.21.15.110:0/3987904002 shutdown_connections 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/3987904002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0ffca0 0x7f87ec100120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/3987904002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0feaa0 0x7f87ec0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 -- 172.21.15.110:0/3987904002 >> 172.21.15.110:0/3987904002 conn(0x7f87ec0fa230 msgr2=0x7f87ec0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:08.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 -- 172.21.15.110:0/3987904002 shutdown_connections 2024-09-21T04:20:08.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.792+0000 7f87f3d0b640 1 -- 172.21.15.110:0/3987904002 wait complete. 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 Processor -- start 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 -- start start 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0ffca0 0x7f87ec19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec19a8e0 con 0x7f87ec0feaa0 2024-09-21T04:20:08.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f3d0b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec19aa50 con 0x7f87ec0ffca0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49898/0 (socket says 172.21.15.110:49898) 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 -- 172.21.15.110:0/4252001729 learned_addr learned my addr 172.21.15.110:0/4252001729 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f12a6640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0ffca0 0x7f87ec19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 -- 172.21.15.110:0/4252001729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0ffca0 msgr2=0x7f87ec19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0ffca0 0x7f87ec19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.793+0000 7f87f1aa7640 1 -- 172.21.15.110:0/4252001729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009560 con 0x7f87ec0feaa0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87f1aa7640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87e800c8f0 tx=0x7f87e800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e80041f0 con 0x7f87ec0feaa0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e8007e70 con 0x7f87ec0feaa0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e80049d0 con 0x7f87ec0feaa0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec19ad30 con 0x7f87ec0feaa0 2024-09-21T04:20:08.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.794+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec19b230 con 0x7f87ec0feaa0 2024-09-21T04:20:08.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.795+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec18e640 con 0x7f87ec0feaa0 2024-09-21T04:20:08.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.795+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f87e8004350 con 0x7f87ec0feaa0 2024-09-21T04:20:08.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.796+0000 7f87e2ffd640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 0x7f87d807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:08.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.796+0000 7f87f12a6640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 0x7f87d807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:08.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.796+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f87e809e1c0 con 0x7f87ec0feaa0 2024-09-21T04:20:08.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.796+0000 7f87f12a6640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 0x7f87d807d730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.800+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f87e8061150 con 0x7f87ec0feaa0 2024-09-21T04:20:08.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.986+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ec02c5c0 con 0x7f87d807b270 2024-09-21T04:20:08.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.995+0000 7f87e2ffd640 1 -- 172.21.15.110:0/4252001729 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f87ec02c5c0 con 0x7f87d807b270 2024-09-21T04:20:08.998 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:20:08.998 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (11m) 10s ago 13m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (4m) 10s ago 13m 8400k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (4m) 109s ago 12m 7369k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (116s) 10s ago 13m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (112s) 109s ago 12m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (11m) 10s ago 13m 86.2M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (7m) 10s ago 7m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (7m) 10s ago 7m 16.4M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (7m) 109s ago 7m 17.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (7m) 109s ago 7m 18.8M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (4m) 10s ago 14m 516M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (3m) 109s ago 12m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:20:08.999 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (2m) 10s ago 14m 47.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (2m) 109s ago 12m 34.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (13m) 10s ago 13m 17.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (12m) 109s ago 12m 15.3M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (89s) 10s ago 11m 102M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (52s) 10s ago 10m 104M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (15s) 10s ago 10m 14.4M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (9m) 109s ago 9m 326M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e adae30c5c147 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (9m) 109s ago 9m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (8m) 109s ago 8m 254M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:20:09.000 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 10s ago 12m 54.1M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:20:09.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 msgr2=0x7f87d807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 0x7f87d807d730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0314e0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 msgr2=0x7f87ec068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f87e800c8f0 tx=0x7f87e800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 shutdown_connections 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f87d807b270 0x7f87d807d730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87ec0ffca0 0x7f87ec19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 --2- 172.21.15.110:0/4252001729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87ec0feaa0 0x7f87ec068f10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 >> 172.21.15.110:0/4252001729 conn(0x7f87ec0fa230 msgr2=0x7f87ec103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 shutdown_connections 2024-09-21T04:20:09.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:08.998+0000 7f87f3d0b640 1 -- 172.21.15.110:0/4252001729 wait complete. 2024-09-21T04:20:09.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 -- 172.21.15.110:0/260820716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 msgr2=0x7f62e4100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 --2- 172.21.15.110:0/260820716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e4100100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009880 tx=0x7f62d402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.081 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:08 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/222 objects degraded (9.459%), 6 pgs degraded) 2024-09-21T04:20:09.081 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:08 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:20:09.081 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:09.081 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 -- 172.21.15.110:0/260820716 shutdown_connections 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 --2- 172.21.15.110:0/260820716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e4100100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 --2- 172.21.15.110:0/260820716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 0x7f62e40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 -- 172.21.15.110:0/260820716 >> 172.21.15.110:0/260820716 conn(0x7f62e40fa230 msgr2=0x7f62e40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 -- 172.21.15.110:0/260820716 shutdown_connections 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.077+0000 7f62e8fda640 1 -- 172.21.15.110:0/260820716 wait complete. 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 Processor -- start 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 -- start start 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 0x7f62e4068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e419a950 con 0x7f62e40ffc80 2024-09-21T04:20:09.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.078+0000 7f62e8fda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e419aa90 con 0x7f62e40fea80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49904/0 (socket says 172.21.15.110:49904) 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 -- 172.21.15.110:0/1368180335 learned_addr learned my addr 172.21.15.110:0/1368180335 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e259c640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 0x7f62e4068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 -- 172.21.15.110:0/1368180335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 msgr2=0x7f62e4068f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 0x7f62e4068f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 -- 172.21.15.110:0/1368180335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4009560 con 0x7f62e40ffc80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e1d9b640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005610 tx=0x7f62d4005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62d4004ba0 con 0x7f62e40ffc80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62d4004d00 con 0x7f62e40ffc80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62d4003bd0 con 0x7f62e40ffc80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.079+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e419ace0 con 0x7f62e40ffc80 2024-09-21T04:20:09.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.080+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e419b250 con 0x7f62e40ffc80 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.081+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e418e620 con 0x7f62e40ffc80 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.081+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f62d4031640 con 0x7f62e40ffc80 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.082+0000 7f62db7fe640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 0x7f62d007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.082+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f62d40bf5a0 con 0x7f62e40ffc80 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.082+0000 7f62e259c640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 0x7f62d007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.082+0000 7f62e259c640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 0x7f62d007d730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0058c0 tx=0x7f62cc00a2d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.086+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f62d40826e0 con 0x7f62e40ffc80 2024-09-21T04:20:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:08 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/222 objects degraded (9.459%), 6 pgs degraded) 2024-09-21T04:20:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:08 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:20:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.333+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f62e4107910 con 0x7f62e40ffc80 2024-09-21T04:20:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.334+0000 7f62db7fe640 1 -- 172.21.15.110:0/1368180335 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f62d4085800 con 0x7f62e40ffc80 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 3, 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:20:09.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 7, 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 7 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:20:09.338 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 msgr2=0x7f62d007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 0x7f62d007d730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0058c0 tx=0x7f62cc00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 msgr2=0x7f62e419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005610 tx=0x7f62d4005640 comp rx=0 tx=0).stop 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 shutdown_connections 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f62d007b270 0x7f62d007d730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e40ffc80 0x7f62e419a380 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 --2- 172.21.15.110:0/1368180335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e40fea80 0x7f62e4068f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 >> 172.21.15.110:0/1368180335 conn(0x7f62e40fa230 msgr2=0x7f62e4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 shutdown_connections 2024-09-21T04:20:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.336+0000 7f62e8fda640 1 -- 172.21.15.110:0/1368180335 wait complete. 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 -- 172.21.15.110:0/2156714749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 msgr2=0x7f60ec101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 --2- 172.21.15.110:0/2156714749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec101680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009880 tx=0x7f60e002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 -- 172.21.15.110:0/2156714749 shutdown_connections 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 --2- 172.21.15.110:0/2156714749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 0x7f60ec0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 --2- 172.21.15.110:0/2156714749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec101680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 -- 172.21.15.110:0/2156714749 >> 172.21.15.110:0/2156714749 conn(0x7f60ec0fb330 msgr2=0x7f60ec0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 -- 172.21.15.110:0/2156714749 shutdown_connections 2024-09-21T04:20:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 -- 172.21.15.110:0/2156714749 wait complete. 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.416+0000 7f60f1014640 1 Processor -- start 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60f1014640 1 -- start start 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60f1014640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 0x7f60ec1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60f1014640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60f1014640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec196300 con 0x7f60ec101280 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60f1014640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec196470 con 0x7f60ec0ff7c0 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49918/0 (socket says 172.21.15.110:49918) 2024-09-21T04:20:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 -- 172.21.15.110:0/2673342255 learned_addr learned my addr 172.21.15.110:0/2673342255 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ead9d640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 0x7f60ec1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 -- 172.21.15.110:0/2673342255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 msgr2=0x7f60ec1957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 0x7f60ec1957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 -- 172.21.15.110:0/2673342255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009560 con 0x7f60ec101280 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.417+0000 7f60ea59c640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00b450 tx=0x7f60dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.418+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc011860 con 0x7f60ec101280 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.418+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec19af10 con 0x7f60ec101280 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.418+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60dc011e80 con 0x7f60ec101280 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.418+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc00f570 con 0x7f60ec101280 2024-09-21T04:20:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.418+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec19b4b0 con 0x7f60ec101280 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.419+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec0ffcc0 con 0x7f60ec101280 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.419+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f60dc0103e0 con 0x7f60ec101280 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.420+0000 7f60d3fff640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 0x7f60d407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.420+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f60dc09dff0 con 0x7f60ec101280 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.420+0000 7f60ead9d640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 0x7f60d407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.420+0000 7f60ead9d640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 0x7f60d407d730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009230 tx=0x7f60e0005b50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.424+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f60dc061080 con 0x7f60ec101280 2024-09-21T04:20:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.638+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f60ec108900 con 0x7f60ec101280 2024-09-21T04:20:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.638+0000 7f60d3fff640 1 -- 172.21.15.110:0/2673342255 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f60dc0641a0 con 0x7f60ec101280 2024-09-21T04:20:09.641 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:20:09.641 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:20:09.641 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:20:09.642 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:20:09.643 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:09.644 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 msgr2=0x7f60d407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 0x7f60d407d730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009230 tx=0x7f60e0005b50 comp rx=0 tx=0).stop 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 msgr2=0x7f60ec195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00b450 tx=0x7f60dc00b910 comp rx=0 tx=0).stop 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 shutdown_connections 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f60d407b270 0x7f60d407d730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec101280 0x7f60ec195d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 --2- 172.21.15.110:0/2673342255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec0ff7c0 0x7f60ec1957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.640+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 >> 172.21.15.110:0/2673342255 conn(0x7f60ec0fb330 msgr2=0x7f60ec103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.641+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 shutdown_connections 2024-09-21T04:20:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.641+0000 7f60f1014640 1 -- 172.21.15.110:0/2673342255 wait complete. 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 -- 172.21.15.110:0/3878666705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 msgr2=0x7f23d80fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/3878666705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d80fff90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009880 tx=0x7f23d402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 -- 172.21.15.110:0/3878666705 shutdown_connections 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/3878666705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/3878666705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d80fff90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 -- 172.21.15.110:0/3878666705 >> 172.21.15.110:0/3878666705 conn(0x7f23d80fb320 msgr2=0x7f23d80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 -- 172.21.15.110:0/3878666705 shutdown_connections 2024-09-21T04:20:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.725+0000 7f23dfbbe640 1 -- 172.21.15.110:0/3878666705 wait complete. 2024-09-21T04:20:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 Processor -- start 2024-09-21T04:20:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 -- start start 2024-09-21T04:20:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d8074790 con 0x7f23d80ffb90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dfbbe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d8072e10 con 0x7f23d8100d90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49938/0 (socket says 172.21.15.110:49938) 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 -- 172.21.15.110:0/636195482 learned_addr learned my addr 172.21.15.110:0/636195482 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd159640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 -- 172.21.15.110:0/636195482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 msgr2=0x7f23d80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 -- 172.21.15.110:0/636195482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4009560 con 0x7f23d80ffb90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd159640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23dd95a640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f23d402a9c0 tx=0x7f23d4005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.726+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d4004b20 con 0x7f23d80ffb90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.727+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23d4004c80 con 0x7f23d80ffb90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.727+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d8073090 con 0x7f23d80ffb90 2024-09-21T04:20:09.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.727+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d4003b80 con 0x7f23d80ffb90 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.727+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8073600 con 0x7f23d80ffb90 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.728+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d80fff90 con 0x7f23d80ffb90 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.728+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f23d4031640 con 0x7f23d80ffb90 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.729+0000 7f23ceffd640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 0x7f23c407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.729+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f23d40bf700 con 0x7f23d80ffb90 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.729+0000 7f23dd159640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 0x7f23c407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.729+0000 7f23dd159640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 0x7f23c407d730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23d8073fd0 tx=0x7f23c8005d90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:09.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.733+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f23d4082710 con 0x7f23d80ffb90 2024-09-21T04:20:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:09 smithi110 ceph-mon[92455]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:09 smithi110 ceph-mon[92455]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:09 smithi110 ceph-mon[92455]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:09 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1368180335' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:09.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:09 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2673342255' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:20:09.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.929+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d8105540 con 0x7f23c407b270 2024-09-21T04:20:09.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.930+0000 7f23ceffd640 1 -- 172.21.15.110:0/636195482 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f23d8105540 con 0x7f23c407b270 2024-09-21T04:20:09.933 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:20:09.933 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:20:09.933 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "9/23 daemons upgraded", 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:20:09.934 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.932+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 msgr2=0x7f23c407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.932+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 0x7f23c407d730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23d8073fd0 tx=0x7f23c8005d90 comp rx=0 tx=0).stop 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.932+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 msgr2=0x7f23d8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.932+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f23d402a9c0 tx=0x7f23d4005610 comp rx=0 tx=0).stop 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 shutdown_connections 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f23c407b270 0x7f23c407d730 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23d8100d90 0x7f23d80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 --2- 172.21.15.110:0/636195482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23d80ffb90 0x7f23d8074250 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 >> 172.21.15.110:0/636195482 conn(0x7f23d80fb320 msgr2=0x7f23d80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 shutdown_connections 2024-09-21T04:20:09.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:09.933+0000 7f23dfbbe640 1 -- 172.21.15.110:0/636195482 wait complete. 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 -- 172.21.15.110:0/3405838894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 msgr2=0x7f3db41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 --2- 172.21.15.110:0/3405838894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db41029b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009c50 tx=0x7f3dac02a9a0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 -- 172.21.15.110:0/3405838894 shutdown_connections 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 --2- 172.21.15.110:0/3405838894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 0x7f3db41052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 --2- 172.21.15.110:0/3405838894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db41029b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 -- 172.21.15.110:0/3405838894 >> 172.21.15.110:0/3405838894 conn(0x7f3db40fa1c0 msgr2=0x7f3db40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.013+0000 7f3dba169640 1 -- 172.21.15.110:0/3405838894 shutdown_connections 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 -- 172.21.15.110:0/3405838894 wait complete. 2024-09-21T04:20:10.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 Processor -- start 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 -- start start 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 0x7f3db4195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4196260 con 0x7f3db41005c0 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.014+0000 7f3dba169640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db41963d0 con 0x7f3db4102ef0 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49950/0 (socket says 172.21.15.110:49950) 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 -- 172.21.15.110:0/62268716 learned_addr learned my addr 172.21.15.110:0/62268716 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db2ffd640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 0x7f3db4195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 -- 172.21.15.110:0/62268716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 msgr2=0x7f3db4195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 0x7f3db4195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 -- 172.21.15.110:0/62268716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac009900 con 0x7f3db41005c0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db37fe640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0098d0 tx=0x7f3dac030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dac003970 con 0x7f3db41005c0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3dac004050 con 0x7f3db41005c0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db419ae10 con 0x7f3db41005c0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.015+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dac044700 con 0x7f3db41005c0 2024-09-21T04:20:10.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.016+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db419b2f0 con 0x7f3db41005c0 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.017+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db41087c0 con 0x7f3db41005c0 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.017+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f3dac00a040 con 0x7f3db41005c0 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.017+0000 7f3db0ff9640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 0x7f3da007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.018+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3dac0bf890 con 0x7f3db41005c0 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.018+0000 7f3db2ffd640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 0x7f3da007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:10.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.018+0000 7f3db2ffd640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 0x7f3da007d740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3db4196c70 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:10.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.022+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f3dac0828a0 con 0x7f3db41005c0 2024-09-21T04:20:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:09 smithi148 ceph-mon[78701]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:09 smithi148 ceph-mon[78701]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:09 smithi148 ceph-mon[78701]: pgmap v101: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:09 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1368180335' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:10.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:09 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2673342255' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:20:10.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.273+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3db41089a0 con 0x7f3db41005c0 2024-09-21T04:20:10.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.273+0000 7f3db0ff9640 1 -- 172.21.15.110:0/62268716 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3dac0859c0 con 0x7f3db41005c0 2024-09-21T04:20:10.276 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:20:10.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 msgr2=0x7f3da007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:10.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 0x7f3da007d740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3db4196c70 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 msgr2=0x7f3db4195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0098d0 tx=0x7f3dac030890 comp rx=0 tx=0).stop 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 shutdown_connections 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.275+0000 7f3dba169640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3da007b280 0x7f3da007d740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.276+0000 7f3dba169640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3db4102ef0 0x7f3db4195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.276+0000 7f3dba169640 1 --2- 172.21.15.110:0/62268716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db41005c0 0x7f3db4195750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.276+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 >> 172.21.15.110:0/62268716 conn(0x7f3db40fa1c0 msgr2=0x7f3db40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.276+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 shutdown_connections 2024-09-21T04:20:10.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:10.276+0000 7f3dba169640 1 -- 172.21.15.110:0/62268716 wait complete. 2024-09-21T04:20:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:10 smithi110 ceph-mon[92455]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:10 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/62268716' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:20:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:10 smithi148 ceph-mon[78701]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:10 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/62268716' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:20:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:11 smithi110 ceph-mon[92455]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:12.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:11 smithi110 ceph-mon[92455]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:12.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:11 smithi148 ceph-mon[78701]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:12.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:11 smithi148 ceph-mon[78701]: pgmap v102: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:12 smithi110 ceph-mon[92455]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:12 smithi148 ceph-mon[78701]: pgmap v103: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:14 smithi110 ceph-mon[92455]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:15.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:14 smithi148 ceph-mon[78701]: pgmap v104: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:17.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:16 smithi110 ceph-mon[92455]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:17.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:16 smithi148 ceph-mon[78701]: pgmap v105: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:18.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:17 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:20:18.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:17 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:20:18.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:17 smithi148 ceph-mon[78701]: Upgrade: osd.3 is safe to restart 2024-09-21T04:20:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:17 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:20:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:17 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:20:18.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:17 smithi110 ceph-mon[92455]: Upgrade: osd.3 is safe to restart 2024-09-21T04:20:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:19 smithi110 ceph-mon[92455]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:20:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:19.427 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:19.427 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:19 smithi148 ceph-mon[78701]: pgmap v106: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:19.427 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:19.427 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:20:19.427 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:20.402 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:20 smithi148 ceph-mon[78701]: Upgrade: Updating osd.3 2024-09-21T04:20:20.403 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:20 smithi148 ceph-mon[78701]: Deploying daemon osd.3 on smithi148 2024-09-21T04:20:20.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:20 smithi110 ceph-mon[92455]: Upgrade: Updating osd.3 2024-09-21T04:20:20.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:20 smithi110 ceph-mon[92455]: Deploying daemon osd.3 on smithi148 2024-09-21T04:20:21.358 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:21 smithi148 ceph-mon[78701]: pgmap v107: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:21.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:21 smithi110 ceph-mon[92455]: pgmap v107: 65 pgs: 65 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:22.243 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:21 smithi148 systemd[1]: Stopping Ceph osd.3 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:20:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:22 smithi110 ceph-mon[92455]: osd.3 marked itself down and dead 2024-09-21T04:20:22.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:22 smithi148 ceph-mon[78701]: osd.3 marked itself down and dead 2024-09-21T04:20:22.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:22 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:20:22.240+0000 7efcc9836640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:20:22.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:22 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:20:22.240+0000 7efcc9836640 -1 osd.3 64 *** Got signal Terminated *** 2024-09-21T04:20:22.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:22 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[40934]: 2024-09-21T04:20:22.240+0000 7efcc9836640 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:20:23.190 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:22 smithi148 podman[86618]: 2024-09-21 04:20:22.810799845 +0000 UTC m=+0.669589228 container died adae30c5c147cf3b25b48b35de59159d7f7a8a31d294fcc200a43ca47a18ef31 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-6a1bdf8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: osdmap e65: 6 total, 5 up, 6 in 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [0, 2]} : dispatch 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]} : dispatch 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]} : dispatch 2024-09-21T04:20:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 3]} : dispatch 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [1, 5]} : dispatch 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 0]} : dispatch 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]} : dispatch 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: pgmap v109: 65 pgs: 16 stale+active+clean, 49 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [0, 2]}]': finished 2024-09-21T04:20:23.671 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:23 smithi148 podman[86618]: 2024-09-21 04:20:23.418002176 +0000 UTC m=+1.276791587 container cleanup adae30c5c147cf3b25b48b35de59159d7f7a8a31d294fcc200a43ca47a18ef31 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=reef-6a1bdf8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T04:20:23.671 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:23 smithi148 bash[86618]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3 2024-09-21T04:20:23.671 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: osdmap e65: 6 total, 5 up, 6 in 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [0, 2]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 3]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [1, 5]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 0]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: pgmap v109: 65 pgs: 16 stale+active+clean, 49 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:23.672 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [0, 2]}]': finished 2024-09-21T04:20:23.940 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:23 smithi148 podman[86644]: 2024-09-21 04:20:23.669316564 +0000 UTC m=+0.851725418 container remove adae30c5c147cf3b25b48b35de59159d7f7a8a31d294fcc200a43ca47a18ef31 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=reef-6a1bdf8, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:20:24.294 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:23.928580162 +0000 UTC m=+0.015800005 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:20:24.295 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:24.056009364 +0000 UTC m=+0.143229192 container create 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:20:24.295 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:24.292627805 +0000 UTC m=+0.379847637 container init 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:20:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]': finished 2024-09-21T04:20:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]': finished 2024-09-21T04:20:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 3]}]': finished 2024-09-21T04:20:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [1, 5]}]': finished 2024-09-21T04:20:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 0]}]': finished 2024-09-21T04:20:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]}]': finished 2024-09-21T04:20:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: osdmap e66: 6 total, 5 up, 6 in 2024-09-21T04:20:24.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:24 smithi110 ceph-mon[92455]: osdmap e67: 6 total, 5 up, 6 in 2024-09-21T04:20:24.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]': finished 2024-09-21T04:20:24.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]': finished 2024-09-21T04:20:24.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [1, 3]}]': finished 2024-09-21T04:20:24.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.4", "id": [1, 5]}]': finished 2024-09-21T04:20:24.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 0]}]': finished 2024-09-21T04:20:24.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]}]': finished 2024-09-21T04:20:24.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: osdmap e66: 6 total, 5 up, 6 in 2024-09-21T04:20:24.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:24 smithi148 ceph-mon[78701]: osdmap e67: 6 total, 5 up, 6 in 2024-09-21T04:20:24.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:24.29614456 +0000 UTC m=+0.383364402 container start 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:20:24.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:24.400878114 +0000 UTC m=+0.488097946 container attach 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:20:24.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:24 smithi148 podman[86735]: 2024-09-21 04:20:24.488054956 +0000 UTC m=+0.575274787 container died 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T04:20:25.386 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86759]: 2024-09-21 04:20:25.129226566 +0000 UTC m=+0.635528153 container remove 99b7ac59461e61ec398b95c490f270fee13cd3c4d3c4d79aa4f89bb6d04db552 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:20:25.386 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.3.service: Deactivated successfully. 2024-09-21T04:20:25.386 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 systemd[1]: Stopped Ceph osd.3 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:20:25.386 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.3.service: Consumed 1min 17.039s CPU time. 2024-09-21T04:20:25.386 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:25 smithi148 ceph-mon[78701]: pgmap v112: 65 pgs: 10 active+undersized, 2 unknown, 8 peering, 3 stale+active+clean, 7 active+undersized+degraded, 35 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23/222 objects degraded (10.360%) 2024-09-21T04:20:25.386 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:25 smithi148 ceph-mon[78701]: osdmap e68: 6 total, 5 up, 6 in 2024-09-21T04:20:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:25 smithi110 ceph-mon[92455]: pgmap v112: 65 pgs: 10 active+undersized, 2 unknown, 8 peering, 3 stale+active+clean, 7 active+undersized+degraded, 35 active+clean; 195 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23/222 objects degraded (10.360%) 2024-09-21T04:20:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:25 smithi110 ceph-mon[92455]: osdmap e68: 6 total, 5 up, 6 in 2024-09-21T04:20:25.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 systemd[1]: Starting Ceph osd.3 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:20:25.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86860]: 2024-09-21 04:20:25.49772258 +0000 UTC m=+0.016020190 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:20:25.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86860]: 2024-09-21 04:20:25.613538157 +0000 UTC m=+0.131835760 container create 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T04:20:26.029 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86860]: 2024-09-21 04:20:25.876091536 +0000 UTC m=+0.394389141 container init 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T04:20:26.029 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86860]: 2024-09-21 04:20:25.880484569 +0000 UTC m=+0.398782178 container start 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T04:20:26.029 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:25 smithi148 podman[86860]: 2024-09-21 04:20:25.976025823 +0000 UTC m=+0.494323438 container attach 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:20:26.317 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:26 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:26.317 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:26 smithi148 bash[86860]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:26.317 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:26 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:26.317 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:26 smithi148 bash[86860]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:26.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:26 smithi110 ceph-mon[92455]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T04:20:26.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:26 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 23/222 objects degraded (10.360%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:26.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:26 smithi148 ceph-mon[78701]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T04:20:26.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:26 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 23/222 objects degraded (10.360%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:27 smithi110 ceph-mon[92455]: osdmap e69: 6 total, 5 up, 6 in 2024-09-21T04:20:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:27 smithi110 ceph-mon[92455]: osdmap e70: 6 total, 5 up, 6 in 2024-09-21T04:20:27.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:27 smithi110 ceph-mon[92455]: pgmap v116: 65 pgs: 1 active+clean+remapped, 14 active+undersized, 3 remapped+peering, 2 unknown, 8 peering, 13 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/222 objects degraded (15.315%); 0 B/s, 192 objects/s recovering 2024-09-21T04:20:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:27 smithi148 ceph-mon[78701]: osdmap e69: 6 total, 5 up, 6 in 2024-09-21T04:20:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:27 smithi148 ceph-mon[78701]: osdmap e70: 6 total, 5 up, 6 in 2024-09-21T04:20:27.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:27 smithi148 ceph-mon[78701]: pgmap v116: 65 pgs: 1 active+clean+remapped, 14 active+undersized, 3 remapped+peering, 2 unknown, 8 peering, 13 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/222 objects degraded (15.315%); 0 B/s, 192 objects/s recovering 2024-09-21T04:20:28.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:28 smithi110 ceph-mon[92455]: osdmap e71: 6 total, 5 up, 6 in 2024-09-21T04:20:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:28 smithi148 ceph-mon[78701]: osdmap e71: 6 total, 5 up, 6 in 2024-09-21T04:20:29.312 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:20:29.312 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 bash[86860]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 bash[86860]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:28 smithi148 bash[86860]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:20:29.313 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:20:29.648 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-mon[78701]: osdmap e72: 6 total, 5 up, 6 in 2024-09-21T04:20:29.648 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-mon[78701]: pgmap v119: 65 pgs: 4 active+clean+remapped, 18 active+undersized, 3 remapped+peering, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47 MiB/s rd, 76 MiB/s wr, 585 op/s; 42/222 objects degraded (18.919%); 0 B/s, 403 objects/s recovering 2024-09-21T04:20:29.648 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:20:29.648 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:20:29.648 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:20:29.648 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:20:29.648 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:20:29.649 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:20:29.649 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate[86875]: --> ceph-volume lvm activate successful for osd ID: 3 2024-09-21T04:20:29.649 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 bash[86860]: --> ceph-volume lvm activate successful for osd ID: 3 2024-09-21T04:20:29.649 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:29 smithi148 podman[86860]: 2024-09-21 04:20:29.33841194 +0000 UTC m=+3.856709550 container died 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True) 2024-09-21T04:20:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:29 smithi110 ceph-mon[92455]: osdmap e72: 6 total, 5 up, 6 in 2024-09-21T04:20:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:29 smithi110 ceph-mon[92455]: pgmap v119: 65 pgs: 4 active+clean+remapped, 18 active+undersized, 3 remapped+peering, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47 MiB/s rd, 76 MiB/s wr, 585 op/s; 42/222 objects degraded (18.919%); 0 B/s, 403 objects/s recovering 2024-09-21T04:20:30.337 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 podman[87077]: 2024-09-21 04:20:30.013099452 +0000 UTC m=+0.668533522 container remove 9c56081c02ca96309b56a4d647837b5f937eb7cb99d31edc9926eefbc6f070ab (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:20:30.337 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 podman[87136]: 2024-09-21 04:20:30.122395137 +0000 UTC m=+0.016132670 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:20:30.337 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 podman[87136]: 2024-09-21 04:20:30.222102856 +0000 UTC m=+0.115840384 container create 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T04:20:30.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:30 smithi148 ceph-mon[78701]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-21T04:20:30.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 podman[87136]: 2024-09-21 04:20:30.460175116 +0000 UTC m=+0.353912650 container init 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T04:20:30.690 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 podman[87136]: 2024-09-21 04:20:30.463647753 +0000 UTC m=+0.357385280 container start 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T04:20:30.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 ceph-osd[87155]: -- 172.21.15.148:0/2079248822 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652af8c41e0 con 0x5652af8aa000 2024-09-21T04:20:30.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 bash[87136]: 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 2024-09-21T04:20:30.691 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:30 smithi148 systemd[1]: Started Ceph osd.3 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:20:30.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:30 smithi110 ceph-mon[92455]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-21T04:20:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:31 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:31.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:31 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:31.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:31 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:31.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:31 smithi148 ceph-mon[78701]: pgmap v120: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 37 MiB/s rd, 59 MiB/s wr, 458 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 3.5 MiB/s, 329 objects/s recovering 2024-09-21T04:20:31.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:31 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 42/222 objects degraded (18.919%), 16 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:31 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:31 smithi110 ceph-mon[92455]: pgmap v120: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 37 MiB/s rd, 59 MiB/s wr, 458 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 3.5 MiB/s, 329 objects/s recovering 2024-09-21T04:20:31.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:31 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 42/222 objects degraded (18.919%), 16 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:32.190 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:32 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:20:32.104+0000 7f5996ef3640 -1 Falling back to public interface 2024-09-21T04:20:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:32 smithi110 ceph-mon[92455]: pgmap v121: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 42 MiB/s wr, 324 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 2.5 MiB/s, 232 objects/s recovering 2024-09-21T04:20:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:32 smithi148 ceph-mon[78701]: pgmap v121: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 42 MiB/s wr, 324 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 2.5 MiB/s, 232 objects/s recovering 2024-09-21T04:20:34.475 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:34 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.475 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:34 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.475 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:34 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.475 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:34 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:34 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:34 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:34 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:34.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:34 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:35 smithi110 ceph-mon[92455]: pgmap v122: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 34 MiB/s wr, 261 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 2.0 MiB/s, 101 objects/s recovering 2024-09-21T04:20:35.550 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:35 smithi148 ceph-mon[78701]: pgmap v122: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 34 MiB/s wr, 261 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 2.0 MiB/s, 101 objects/s recovering 2024-09-21T04:20:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: pgmap v123: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 29 MiB/s wr, 221 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 1.7 MiB/s, 85 objects/s recovering 2024-09-21T04:20:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:37.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:20:37.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:36 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: pgmap v123: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 29 MiB/s wr, 221 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 1.7 MiB/s, 85 objects/s recovering 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:20:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:36 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:20:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:37 smithi110 ceph-mon[92455]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:20:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:37 smithi148 ceph-mon[78701]: Upgrade: unsafe to stop osd(s) at this time (17 PGs are or would become offline) 2024-09-21T04:20:39.147 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:38 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:20:38.877+0000 7f5996ef3640 -1 osd.3 64 log_to_monitors true 2024-09-21T04:20:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:39 smithi110 ceph-mon[92455]: from='osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:20:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:39 smithi110 ceph-mon[92455]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:20:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:39 smithi110 ceph-mon[92455]: pgmap v124: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 26 MiB/s wr, 200 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 1.5 MiB/s, 77 objects/s recovering 2024-09-21T04:20:39.440 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:20:39.162+0000 7f598ac95640 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:20:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-mon[78701]: from='osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:20:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-mon[78701]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:20:39.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:39 smithi148 ceph-mon[78701]: pgmap v124: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 16 active+undersized+degraded, 24 active+clean; 195 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 26 MiB/s wr, 200 op/s; 42/222 objects degraded (18.919%); 8/222 objects misplaced (3.604%); 1.5 MiB/s, 77 objects/s recovering 2024-09-21T04:20:40.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 -- 172.21.15.110:0/2791363194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64440ff7c0 msgr2=0x7f64440ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 --2- 172.21.15.110:0/2791363194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64440ff7c0 0x7f64440ffc40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6430009880 tx=0x7f643002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 -- 172.21.15.110:0/2791363194 shutdown_connections 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 --2- 172.21.15.110:0/2791363194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64440ff7c0 0x7f64440ffc40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 --2- 172.21.15.110:0/2791363194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6444101280 0x7f6444101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 -- 172.21.15.110:0/2791363194 >> 172.21.15.110:0/2791363194 conn(0x7f64440fb330 msgr2=0x7f64440fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 -- 172.21.15.110:0/2791363194 shutdown_connections 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.366+0000 7f644ad65640 1 -- 172.21.15.110:0/2791363194 wait complete. 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 Processor -- start 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 -- start start 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 0x7f6444195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444196270 con 0x7f6444101280 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f644ad65640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64441963e0 con 0x7f64440ff7c0 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f6443fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f6443fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55716/0 (socket says 172.21.15.110:55716) 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.367+0000 7f6443fff640 1 -- 172.21.15.110:0/1118590879 learned_addr learned my addr 172.21.15.110:0/1118590879 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6448b01640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 0x7f6444195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6443fff640 1 -- 172.21.15.110:0/1118590879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 msgr2=0x7f6444195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6443fff640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 0x7f6444195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6443fff640 1 -- 172.21.15.110:0/1118590879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009560 con 0x7f6444101280 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6448b01640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 0x7f6444195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6443fff640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6430005520 tx=0x7f6430005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6430004aa0 con 0x7f6444101280 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644419ae20 con 0x7f6444101280 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6430004c00 con 0x7f6444101280 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6430003b10 con 0x7f6444101280 2024-09-21T04:20:40.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.368+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644419b360 con 0x7f6444101280 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.369+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64440ffc40 con 0x7f6444101280 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.370+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f6430003d40 con 0x7f6444101280 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.371+0000 7f6441ffb640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 0x7f643407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.371+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6934+0+0 (secure 0 0 0) 0x7f64300bf6f0 con 0x7f6444101280 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.371+0000 7f6448b01640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 0x7f643407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.371+0000 7f6448b01640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 0x7f643407d730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64380058c0 tx=0x7f6438005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.375+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f64300825b0 con 0x7f6444101280 2024-09-21T04:20:40.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:40 smithi110 ceph-mon[92455]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:20:40.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:40 smithi110 ceph-mon[92455]: osdmap e73: 6 total, 5 up, 6 in 2024-09-21T04:20:40.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:40 smithi110 ceph-mon[92455]: from='osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:20:40.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:40 smithi110 ceph-mon[92455]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:20:40.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:40 smithi148 ceph-mon[78701]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:20:40.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:40 smithi148 ceph-mon[78701]: osdmap e73: 6 total, 5 up, 6 in 2024-09-21T04:20:40.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:40 smithi148 ceph-mon[78701]: from='osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:20:40.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:40 smithi148 ceph-mon[78701]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:20:40.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.572+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f644402c450 con 0x7f643407b270 2024-09-21T04:20:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.574+0000 7f6441ffb640 1 -- 172.21.15.110:0/1118590879 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f644402c450 con 0x7f643407b270 2024-09-21T04:20:40.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 msgr2=0x7f643407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 0x7f643407d730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64380058c0 tx=0x7f6438005850 comp rx=0 tx=0).stop 2024-09-21T04:20:40.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 msgr2=0x7f6444195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6430005520 tx=0x7f6430005550 comp rx=0 tx=0).stop 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 shutdown_connections 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f643407b270 0x7f643407d730 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6444101280 0x7f6444195ca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 --2- 172.21.15.110:0/1118590879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440ff7c0 0x7f6444195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 >> 172.21.15.110:0/1118590879 conn(0x7f64440fb330 msgr2=0x7f6444104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 shutdown_connections 2024-09-21T04:20:40.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.576+0000 7f644ad65640 1 -- 172.21.15.110:0/1118590879 wait complete. 2024-09-21T04:20:40.587 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 -- 172.21.15.110:0/1488056805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140fea30 msgr2=0x7f26140fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 --2- 172.21.15.110:0/1488056805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140fea30 0x7f26140fee30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2604009880 tx=0x7f260402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 -- 172.21.15.110:0/1488056805 shutdown_connections 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 --2- 172.21.15.110:0/1488056805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140ffc30 0x7f26141000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 --2- 172.21.15.110:0/1488056805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140fea30 0x7f26140fee30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 -- 172.21.15.110:0/1488056805 >> 172.21.15.110:0/1488056805 conn(0x7f26140fa1c0 msgr2=0x7f26140fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 -- 172.21.15.110:0/1488056805 shutdown_connections 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.657+0000 7f261a5d9640 1 -- 172.21.15.110:0/1488056805 wait complete. 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 Processor -- start 2024-09-21T04:20:40.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 -- start start 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140fea30 0x7f2614195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614196080 con 0x7f26140ffc30 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f261a5d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26141961f0 con 0x7f26140fea30 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.658+0000 7f26137fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55742/0 (socket says 172.21.15.110:55742) 2024-09-21T04:20:40.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 -- 172.21.15.110:0/2969665678 learned_addr learned my addr 172.21.15.110:0/2969665678 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f2613fff640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140fea30 0x7f2614195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 -- 172.21.15.110:0/2969665678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140fea30 msgr2=0x7f2614195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140fea30 0x7f2614195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 -- 172.21.15.110:0/2969665678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604009560 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26137fe640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f260800c8a0 tx=0x7f260800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2608007b80 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2608007ce0 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261419acd0 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2608018810 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.659+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261419b250 con 0x7f26140ffc30 2024-09-21T04:20:40.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.660+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2614107620 con 0x7f26140ffc30 2024-09-21T04:20:40.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.661+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f2608018a80 con 0x7f26140ffc30 2024-09-21T04:20:40.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.661+0000 7f26117fa640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 0x7f25fc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.661+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6934+0+0 (secure 0 0 0) 0x7f260809e600 con 0x7f26140ffc30 2024-09-21T04:20:40.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.662+0000 7f2613fff640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 0x7f25fc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.662+0000 7f2613fff640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 0x7f25fc07d740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2604009230 tx=0x7f26040314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.666+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f2608061460 con 0x7f26140ffc30 2024-09-21T04:20:40.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.860+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26140682f0 con 0x7f25fc07b280 2024-09-21T04:20:40.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.862+0000 7f26117fa640 1 -- 172.21.15.110:0/2969665678 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f26140682f0 con 0x7f25fc07b280 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 msgr2=0x7f25fc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 0x7f25fc07d740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2604009230 tx=0x7f26040314e0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 msgr2=0x7f2614195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f260800c8a0 tx=0x7f260800cd60 comp rx=0 tx=0).stop 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 shutdown_connections 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f25fc07b280 0x7f25fc07d740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26140ffc30 0x7f2614195b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 --2- 172.21.15.110:0/2969665678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26140fea30 0x7f2614195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 >> 172.21.15.110:0/2969665678 conn(0x7f26140fa1c0 msgr2=0x7f2614067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:40.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 shutdown_connections 2024-09-21T04:20:40.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.864+0000 7f261a5d9640 1 -- 172.21.15.110:0/2969665678 wait complete. 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.944+0000 7f1c03b69640 1 -- 172.21.15.110:0/1130937810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 msgr2=0x7f1bfc0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.944+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1130937810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc0fee30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009880 tx=0x7f1bf802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 -- 172.21.15.110:0/1130937810 shutdown_connections 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1130937810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1130937810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc0fee30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 -- 172.21.15.110:0/1130937810 >> 172.21.15.110:0/1130937810 conn(0x7f1bfc0fa1c0 msgr2=0x7f1bfc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 -- 172.21.15.110:0/1130937810 shutdown_connections 2024-09-21T04:20:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 -- 172.21.15.110:0/1130937810 wait complete. 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 Processor -- start 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 -- start start 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.945+0000 7f1c03b69640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c03b69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c03b69640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc101550 con 0x7f1bfc0fea30 2024-09-21T04:20:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c03b69640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1016c0 con 0x7f1bfc0ffc30 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55754/0 (socket says 172.21.15.110:55754) 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 -- 172.21.15.110:0/1040941342 learned_addr learned my addr 172.21.15.110:0/1040941342 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01104640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 -- 172.21.15.110:0/1040941342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 msgr2=0x7f1bfc101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 -- 172.21.15.110:0/1040941342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8009560 con 0x7f1bfc0fea30 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01104640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c01905640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802a9c0 tx=0x7f1bf80054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf8004aa0 con 0x7f1bfc0fea30 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bf8004c00 con 0x7f1bfc0fea30 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf8003a80 con 0x7f1bfc0fea30 2024-09-21T04:20:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc101940 con 0x7f1bfc0fea30 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.946+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc19d680 con 0x7f1bfc0fea30 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.947+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc107640 con 0x7f1bfc0fea30 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.948+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1bf8003cf0 con 0x7f1bfc0fea30 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.948+0000 7f1beeffd640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 0x7f1be807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.949+0000 7f1c01104640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 0x7f1be807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.949+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6934+0+0 (secure 0 0 0) 0x7f1bf80bf640 con 0x7f1bfc0fea30 2024-09-21T04:20:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.949+0000 7f1c01104640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 0x7f1be807d740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc102740 tx=0x7f1bf0005e50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:40.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:40.953+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1bf8082520 con 0x7f1bfc0fea30 2024-09-21T04:20:41.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.137+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bfc0fc3d0 con 0x7f1be807b280 2024-09-21T04:20:41.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.147+0000 7f1beeffd640 1 -- 172.21.15.110:0/1040941342 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1bfc0fc3d0 con 0x7f1be807b280 2024-09-21T04:20:41.150 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:20:41.150 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (12m) 42s ago 13m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (5m) 42s ago 14m 8400k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (4m) 7s ago 13m 7541k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (2m) 42s ago 14m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (2m) 7s ago 13m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (11m) 42s ago 13m 86.2M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (7m) 42s ago 7m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (7m) 42s ago 7m 16.4M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (7m) 7s ago 7m 17.9M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (7m) 7s ago 7m 19.7M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (4m) 42s ago 15m 516M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (3m) 7s ago 13m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (3m) 42s ago 15m 47.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:20:41.151 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (3m) 7s ago 12m 39.8M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (14m) 42s ago 14m 17.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (13m) 7s ago 13m 14.9M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (2m) 42s ago 11m 102M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (84s) 42s ago 11m 104M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (47s) 42s ago 10m 14.4M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (11s) 7s ago 10m 11.4M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (9m) 7s ago 9m 260M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e 1cff5418b8ee 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (9m) 7s ago 9m 259M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:20:41.152 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 42s ago 13m 54.1M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 msgr2=0x7f1be807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 0x7f1be807d740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc102740 tx=0x7f1bf0005e50 comp rx=0 tx=0).stop 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 msgr2=0x7f1bfc1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802a9c0 tx=0x7f1bf80054f0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 shutdown_connections 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1be807b280 0x7f1be807d740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bfc0ffc30 0x7f1bfc101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 --2- 172.21.15.110:0/1040941342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bfc0fea30 0x7f1bfc1029c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 >> 172.21.15.110:0/1040941342 conn(0x7f1bfc0fa1c0 msgr2=0x7f1bfc0fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 shutdown_connections 2024-09-21T04:20:41.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.151+0000 7f1c03b69640 1 -- 172.21.15.110:0/1040941342 wait complete. 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- 172.21.15.110:0/2154467941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 msgr2=0x7f5648101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 --2- 172.21.15.110:0/2154467941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648101680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5644009880 tx=0x7f564402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- 172.21.15.110:0/2154467941 shutdown_connections 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 --2- 172.21.15.110:0/2154467941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f56480ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 --2- 172.21.15.110:0/2154467941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648101680 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- 172.21.15.110:0/2154467941 >> 172.21.15.110:0/2154467941 conn(0x7f56480fb330 msgr2=0x7f56480fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- 172.21.15.110:0/2154467941 shutdown_connections 2024-09-21T04:20:41.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- 172.21.15.110:0/2154467941 wait complete. 2024-09-21T04:20:41.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 Processor -- start 2024-09-21T04:20:41.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 -- start start 2024-09-21T04:20:41.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.247+0000 7f564f713640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f5648195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564f713640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564f713640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648196220 con 0x7f5648101280 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564f713640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648196390 con 0x7f56480ff7c0 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55768/0 (socket says 172.21.15.110:55768) 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 -- 172.21.15.110:0/444452222 learned_addr learned my addr 172.21.15.110:0/444452222 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564d4af640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f5648195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 -- 172.21.15.110:0/444452222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 msgr2=0x7f5648195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f5648195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 -- 172.21.15.110:0/444452222 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009560 con 0x7f5648101280 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564d4af640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f5648195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564ccae640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f563800d720 tx=0x7f563800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56380051f0 con 0x7f5648101280 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5638009ce0 con 0x7f5648101280 2024-09-21T04:20:41.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564819ae30 con 0x7f5648101280 2024-09-21T04:20:41.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5638010460 con 0x7f5648101280 2024-09-21T04:20:41.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.248+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564819b3d0 con 0x7f5648101280 2024-09-21T04:20:41.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.249+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56480ffc40 con 0x7f5648101280 2024-09-21T04:20:41.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.250+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f5638005350 con 0x7f5648101280 2024-09-21T04:20:41.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.251+0000 7f563e7fc640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 0x7f563407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.251+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f563809dfd0 con 0x7f5648101280 2024-09-21T04:20:41.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.251+0000 7f564d4af640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 0x7f563407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.251+0000 7f564d4af640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 0x7f563407d730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5644009230 tx=0x7f56440314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.255+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f5638060ec0 con 0x7f5648101280 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486] boot 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: osdmap e74: 6 total, 6 up, 6 in 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:41.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: pgmap v127: 65 pgs: 2 remapped+peering, 5 peering, 5 active+clean+remapped, 16 active+undersized, 13 active+undersized+degraded, 24 active+clean; 195 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:20:41.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:41 smithi110 ceph-mon[92455]: osdmap e75: 6 total, 6 up, 6 in 2024-09-21T04:20:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:20:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: osd.3 [v2:172.21.15.148:6800/3301372486,v1:172.21.15.148:6801/3301372486] boot 2024-09-21T04:20:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: osdmap e74: 6 total, 6 up, 6 in 2024-09-21T04:20:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:20:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:41.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:41.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: pgmap v127: 65 pgs: 2 remapped+peering, 5 peering, 5 active+clean+remapped, 16 active+undersized, 13 active+undersized+degraded, 24 active+clean; 195 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:20:41.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:41 smithi148 ceph-mon[78701]: osdmap e75: 6 total, 6 up, 6 in 2024-09-21T04:20:41.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.506+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f56481088a0 con 0x7f5648101280 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.507+0000 7f563e7fc640 1 -- 172.21.15.110:0/444452222 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f5638063fe0 con 0x7f5648101280 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:20:41.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2, 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:20:41.511 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 6, 2024-09-21T04:20:41.512 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 8 2024-09-21T04:20:41.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:20:41.512 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:20:41.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 msgr2=0x7f563407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 0x7f563407d730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5644009230 tx=0x7f56440314e0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 msgr2=0x7f5648195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f563800d720 tx=0x7f563800dbe0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 shutdown_connections 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f563407b270 0x7f563407d730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5648101280 0x7f5648195c50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 --2- 172.21.15.110:0/444452222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56480ff7c0 0x7f5648195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.509+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 >> 172.21.15.110:0/444452222 conn(0x7f56480fb330 msgr2=0x7f5648104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.510+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 shutdown_connections 2024-09-21T04:20:41.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.510+0000 7f564f713640 1 -- 172.21.15.110:0/444452222 wait complete. 2024-09-21T04:20:41.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1980426299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 msgr2=0x7f2fd40ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1980426299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd40ffc40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009880 tx=0x7f2fd002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1980426299 shutdown_connections 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1980426299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd40ffc40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1980426299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 0x7f2fd4101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1980426299 >> 172.21.15.110:0/1980426299 conn(0x7f2fd40fb330 msgr2=0x7f2fd40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1980426299 shutdown_connections 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.590+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1980426299 wait complete. 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 Processor -- start 2024-09-21T04:20:41.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 -- start start 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 0x7f2fd4195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4196200 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fdbb4f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4196370 con 0x7f2fd4101280 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55776/0 (socket says 172.21.15.110:55776) 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 -- 172.21.15.110:0/1013313588 learned_addr learned my addr 172.21.15.110:0/1013313588 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd90ea640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 0x7f2fd4195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 -- 172.21.15.110:0/1013313588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 msgr2=0x7f2fd4195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 0x7f2fd4195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 -- 172.21.15.110:0/1013313588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009560 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.591+0000 7f2fd98eb640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800d720 tx=0x7f2fc800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.592+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fc80041f0 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.592+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fc8004350 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.592+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd419ae10 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.592+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fc8003df0 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.592+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd419b350 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.593+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd40ffc40 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.593+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f2fc80035f0 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.594+0000 7f2fc6ffd640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 0x7f2fc007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.594+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f2fc809dd30 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.594+0000 7f2fd90ea640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 0x7f2fc007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.594+0000 7f2fd90ea640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 0x7f2fc007d730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd00314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.598+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f2fc8060d50 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.813+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2fd402c570 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.813+0000 7f2fc6ffd640 1 -- 172.21.15.110:0/1013313588 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f2fc8017790 con 0x7f2fd40ff7c0 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:20:41.817 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:20:41.818 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:20:41.819 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:41.820 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 msgr2=0x7f2fc007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 0x7f2fc007d730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd00314e0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 msgr2=0x7f2fd41956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800d720 tx=0x7f2fc800dbe0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 shutdown_connections 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2fc007b270 0x7f2fc007d730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fd4101280 0x7f2fd4195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 --2- 172.21.15.110:0/1013313588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fd40ff7c0 0x7f2fd41956f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 >> 172.21.15.110:0/1013313588 conn(0x7f2fd40fb330 msgr2=0x7f2fd4104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 shutdown_connections 2024-09-21T04:20:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.816+0000 7f2fdbb4f640 1 -- 172.21.15.110:0/1013313588 wait complete. 2024-09-21T04:20:41.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 -- 172.21.15.110:0/1727979429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 msgr2=0x7f68780fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 --2- 172.21.15.110:0/1727979429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68780fee80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6860009880 tx=0x7f686002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 -- 172.21.15.110:0/1727979429 shutdown_connections 2024-09-21T04:20:41.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 --2- 172.21.15.110:0/1727979429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 0x7f6878100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 --2- 172.21.15.110:0/1727979429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68780fee80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 -- 172.21.15.110:0/1727979429 >> 172.21.15.110:0/1727979429 conn(0x7f68780fa230 msgr2=0x7f68780fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 -- 172.21.15.110:0/1727979429 shutdown_connections 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 -- 172.21.15.110:0/1727979429 wait complete. 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.897+0000 7f687d229640 1 Processor -- start 2024-09-21T04:20:41.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687d229640 1 -- start start 2024-09-21T04:20:41.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687d229640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687d229640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 0x7f6878194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687d229640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781950f0 con 0x7f68780fea80 2024-09-21T04:20:41.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687d229640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878195260 con 0x7f68780ffc80 2024-09-21T04:20:41.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55792/0 (socket says 172.21.15.110:55792) 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 -- 172.21.15.110:0/2948635409 learned_addr learned my addr 172.21.15.110:0/2948635409 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f687659c640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 0x7f6878194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 -- 172.21.15.110:0/2948635409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 msgr2=0x7f6878194b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 0x7f6878194b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 -- 172.21.15.110:0/2948635409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009560 con 0x7f68780fea80 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.898+0000 7f6876d9d640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6860005610 tx=0x7f6860005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.899+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6860004ba0 con 0x7f68780fea80 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.899+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6860004d00 con 0x7f68780fea80 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.899+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6860003bd0 con 0x7f68780fea80 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.899+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6878199ca0 con 0x7f68780fea80 2024-09-21T04:20:41.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.899+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687819a140 con 0x7f68780fea80 2024-09-21T04:20:41.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.900+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68781076f0 con 0x7f68780fea80 2024-09-21T04:20:41.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.900+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f6860031640 con 0x7f68780fea80 2024-09-21T04:20:41.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.901+0000 7f686ffff640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 0x7f686407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:41.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.901+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f68600bf7f0 con 0x7f68780fea80 2024-09-21T04:20:41.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.901+0000 7f687659c640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 0x7f686407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:41.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.901+0000 7f687659c640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 0x7f686407d730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6878195b00 tx=0x7f6868008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:41.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:41.905+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f68600826e0 con 0x7f68780fea80 2024-09-21T04:20:42.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.102+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f687819a3b0 con 0x7f686407b270 2024-09-21T04:20:42.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.103+0000 7f686ffff640 1 -- 172.21.15.110:0/2948635409 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f687819a3b0 con 0x7f686407b270 2024-09-21T04:20:42.259 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:20:42.259 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "10/23 daemons upgraded", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:20:42.260 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:20:42.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 msgr2=0x7f686407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 0x7f686407d730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6878195b00 tx=0x7f6868008040 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 msgr2=0x7f68781945e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6860005610 tx=0x7f6860005640 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 shutdown_connections 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f686407b270 0x7f686407d730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68780ffc80 0x7f6878194b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 --2- 172.21.15.110:0/2948635409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68780fea80 0x7f68781945e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 >> 172.21.15.110:0/2948635409 conn(0x7f68780fa230 msgr2=0x7f6878102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 shutdown_connections 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.105+0000 7f687d229640 1 -- 172.21.15.110:0/2948635409 wait complete. 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 -- 172.21.15.110:0/3179627579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780fea30 msgr2=0x7f15780fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3179627579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780fea30 0x7f15780fee30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1574009880 tx=0x7f157402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 -- 172.21.15.110:0/3179627579 shutdown_connections 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3179627579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780ffc30 0x7f15781000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3179627579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780fea30 0x7f15780fee30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 -- 172.21.15.110:0/3179627579 >> 172.21.15.110:0/3179627579 conn(0x7f15780fa1c0 msgr2=0x7f15780fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 -- 172.21.15.110:0/3179627579 shutdown_connections 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.194+0000 7f157f8ee640 1 -- 172.21.15.110:0/3179627579 wait complete. 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 Processor -- start 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 -- start start 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 0x7f1578194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578195040 con 0x7f15780ffc30 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157f8ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781951b0 con 0x7f15780fea30 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57168/0 (socket says 172.21.15.110:57168) 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 -- 172.21.15.110:0/3294246764 learned_addr learned my addr 172.21.15.110:0/3294246764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:20:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157d68a640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 0x7f1578194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 -- 172.21.15.110:0/3294246764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 msgr2=0x7f1578194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 0x7f1578194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 -- 172.21.15.110:0/3294246764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574009560 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157d68a640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 0x7f1578194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.195+0000 7f157ce89640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f156800d720 tx=0x7f156800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.196+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15680051f0 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.196+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1568009ce0 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.196+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578199c50 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.196+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1568010460 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.196+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157819a220 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.197+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1578107640 con 0x7f15780ffc30 2024-09-21T04:20:42.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.198+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1568005350 con 0x7f15780ffc30 2024-09-21T04:20:42.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.198+0000 7f156e7fc640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 0x7f156407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:20:42.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.198+0000 7f157d68a640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 0x7f156407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:20:42.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.198+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f156809dfd0 con 0x7f15780ffc30 2024-09-21T04:20:42.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.199+0000 7f157d68a640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 0x7f156407d740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1574009230 tx=0x7f15740314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:20:42.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.203+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1568060ec0 con 0x7f15780ffc30 2024-09-21T04:20:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:42 smithi110 ceph-mon[92455]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:42 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 36/222 objects degraded (16.216%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:42 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/444452222' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:42 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1013313588' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:20:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:42 smithi110 ceph-mon[92455]: osdmap e76: 6 total, 6 up, 6 in 2024-09-21T04:20:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:42 smithi148 ceph-mon[78701]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:42 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 36/222 objects degraded (16.216%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:20:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:42 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/444452222' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:20:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:42 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1013313588' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:20:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:42 smithi148 ceph-mon[78701]: osdmap e76: 6 total, 6 up, 6 in 2024-09-21T04:20:42.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.462+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f15781077d0 con 0x7f15780ffc30 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.463+0000 7f156e7fc640 1 -- 172.21.15.110:0/3294246764 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+906 (secure 0 0 0) 0x7f1568063fe0 con 0x7f15780ffc30 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN Degraded data redundancy: 36/222 objects degraded (16.216%), 13 pgs degraded 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/222 objects degraded (16.216%), 13 pgs degraded 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-09-21T04:20:42.466 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-09-21T04:20:42.467 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 msgr2=0x7f156407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 0x7f156407d740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1574009230 tx=0x7f15740314e0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 msgr2=0x7f1578194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f156800d720 tx=0x7f156800dbe0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 shutdown_connections 2024-09-21T04:20:42.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f156407b280 0x7f156407d740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15780ffc30 0x7f1578194a70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 --2- 172.21.15.110:0/3294246764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15780fea30 0x7f1578194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:20:42.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 >> 172.21.15.110:0/3294246764 conn(0x7f15780fa1c0 msgr2=0x7f1578102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:20:42.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 shutdown_connections 2024-09-21T04:20:42.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:20:42.465+0000 7f157f8ee640 1 -- 172.21.15.110:0/3294246764 wait complete. 2024-09-21T04:20:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:43 smithi110 ceph-mon[92455]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:43 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3294246764' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:20:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:43 smithi110 ceph-mon[92455]: pgmap v130: 65 pgs: 3 remapped+peering, 5 peering, 4 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 27 active+clean; 195 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:20:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:43 smithi148 ceph-mon[78701]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:20:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:43 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3294246764' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:20:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:43 smithi148 ceph-mon[78701]: pgmap v130: 65 pgs: 3 remapped+peering, 5 peering, 4 active+clean+remapped, 15 active+undersized, 11 active+undersized+degraded, 27 active+clean; 195 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 32/222 objects degraded (14.414%) 2024-09-21T04:20:44.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:44 smithi148 ceph-mon[78701]: osdmap e77: 6 total, 6 up, 6 in 2024-09-21T04:20:44.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:44 smithi110 ceph-mon[92455]: osdmap e77: 6 total, 6 up, 6 in 2024-09-21T04:20:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:45 smithi110 ceph-mon[92455]: osdmap e78: 6 total, 6 up, 6 in 2024-09-21T04:20:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:45 smithi110 ceph-mon[92455]: pgmap v133: 65 pgs: 1 active+undersized+remapped, 3 remapped+peering, 5 peering, 56 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:45 smithi148 ceph-mon[78701]: osdmap e78: 6 total, 6 up, 6 in 2024-09-21T04:20:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:45 smithi148 ceph-mon[78701]: pgmap v133: 65 pgs: 1 active+undersized+remapped, 3 remapped+peering, 5 peering, 56 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail 2024-09-21T04:20:46.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:46 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/222 objects degraded (14.414%), 11 pgs degraded) 2024-09-21T04:20:46.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:46 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:20:46.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:46 smithi148 ceph-mon[78701]: osdmap e79: 6 total, 6 up, 6 in 2024-09-21T04:20:46.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:46 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/222 objects degraded (14.414%), 11 pgs degraded) 2024-09-21T04:20:46.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:46 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:20:46.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:46 smithi110 ceph-mon[92455]: osdmap e79: 6 total, 6 up, 6 in 2024-09-21T04:20:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:47 smithi110 ceph-mon[92455]: osdmap e80: 6 total, 6 up, 6 in 2024-09-21T04:20:47.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:47 smithi110 ceph-mon[92455]: pgmap v136: 65 pgs: 1 active+undersized+remapped, 1 remapped+peering, 63 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 0 objects/s recovering 2024-09-21T04:20:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:47 smithi148 ceph-mon[78701]: osdmap e80: 6 total, 6 up, 6 in 2024-09-21T04:20:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:47 smithi148 ceph-mon[78701]: pgmap v136: 65 pgs: 1 active+undersized+remapped, 1 remapped+peering, 63 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 0 objects/s recovering 2024-09-21T04:20:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:48 smithi110 ceph-mon[92455]: pgmap v137: 65 pgs: 1 remapped+peering, 64 active+clean; 195 MiB data, 961 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s, 0 objects/s recovering 2024-09-21T04:20:49.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:48 smithi148 ceph-mon[78701]: pgmap v137: 65 pgs: 1 remapped+peering, 64 active+clean; 195 MiB data, 961 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s, 0 objects/s recovering 2024-09-21T04:20:51.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:50 smithi110 ceph-mon[92455]: pgmap v138: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 117 objects/s recovering 2024-09-21T04:20:51.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:50 smithi148 ceph-mon[78701]: pgmap v138: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 117 objects/s recovering 2024-09-21T04:20:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:20:52.421 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:20:53.379 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:53 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:20:53.379 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:53 smithi148 ceph-mon[78701]: Upgrade: osd.4 is safe to restart 2024-09-21T04:20:53.379 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:53 smithi148 ceph-mon[78701]: pgmap v139: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 1024 KiB/s, 98 objects/s recovering 2024-09-21T04:20:53.416 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:53 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:20:53.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:53 smithi110 ceph-mon[92455]: Upgrade: osd.4 is safe to restart 2024-09-21T04:20:53.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:53 smithi110 ceph-mon[92455]: pgmap v139: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 1024 KiB/s, 98 objects/s recovering 2024-09-21T04:20:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: Upgrade: Updating osd.4 2024-09-21T04:20:54.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:54.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:20:54.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:54.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:54 smithi110 ceph-mon[92455]: Deploying daemon osd.4 on smithi148 2024-09-21T04:20:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:20:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: Upgrade: Updating osd.4 2024-09-21T04:20:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:54.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:20:54.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:20:54.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:54 smithi148 ceph-mon[78701]: Deploying daemon osd.4 on smithi148 2024-09-21T04:20:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:55 smithi110 ceph-mon[92455]: pgmap v140: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 0 B/s, 77 objects/s recovering 2024-09-21T04:20:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:55 smithi148 ceph-mon[78701]: pgmap v140: 65 pgs: 65 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 0 B/s, 77 objects/s recovering 2024-09-21T04:20:56.541 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:56 smithi148 systemd[1]: Stopping Ceph osd.4 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:20:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:56 smithi110 ceph-mon[92455]: osd.4 marked itself down and dead 2024-09-21T04:20:56.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:56 smithi148 ceph-mon[78701]: osd.4 marked itself down and dead 2024-09-21T04:20:56.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:56 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[47367]: 2024-09-21T04:20:56.537+0000 7fb3d7a42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:20:56.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:56 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[47367]: 2024-09-21T04:20:56.537+0000 7fb3d7a42640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-21T04:20:56.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:56 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[47367]: 2024-09-21T04:20:56.537+0000 7fb3d7a42640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:20:57.422 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 podman[91349]: 2024-09-21 04:20:57.129998015 +0000 UTC m=+0.709278999 container died 1cff5418b8ee10f64b4e70268578c2931305c2cb4c0c1e02042bc9200b854105 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-6a1bdf8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:20:57.423 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 podman[91349]: 2024-09-21 04:20:57.420931953 +0000 UTC m=+1.000212928 container cleanup 1cff5418b8ee10f64b4e70268578c2931305c2cb4c0c1e02042bc9200b854105 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=reef-6a1bdf8) 2024-09-21T04:20:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:57 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:20:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:57 smithi148 ceph-mon[78701]: osdmap e81: 6 total, 5 up, 6 in 2024-09-21T04:20:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:57 smithi148 ceph-mon[78701]: pgmap v142: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 0 B/s, 74 objects/s recovering 2024-09-21T04:20:57.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:57 smithi148 ceph-mon[78701]: osdmap e82: 6 total, 5 up, 6 in 2024-09-21T04:20:57.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 bash[91349]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4 2024-09-21T04:20:57.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 podman[91366]: 2024-09-21 04:20:57.639156858 +0000 UTC m=+0.512729289 container remove 1cff5418b8ee10f64b4e70268578c2931305c2cb4c0c1e02042bc9200b854105 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-6a1bdf8, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:20:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:57 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:20:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:57 smithi110 ceph-mon[92455]: osdmap e81: 6 total, 5 up, 6 in 2024-09-21T04:20:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:57 smithi110 ceph-mon[92455]: pgmap v142: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 0 B/s, 74 objects/s recovering 2024-09-21T04:20:57.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:57 smithi110 ceph-mon[92455]: osdmap e82: 6 total, 5 up, 6 in 2024-09-21T04:20:58.278 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 podman[91457]: 2024-09-21 04:20:57.876961578 +0000 UTC m=+0.016091350 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:20:58.278 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:57 smithi148 podman[91457]: 2024-09-21 04:20:57.981788372 +0000 UTC m=+0.120918179 container create decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T04:20:58.278 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 podman[91457]: 2024-09-21 04:20:58.201009489 +0000 UTC m=+0.340139251 container init decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:20:58.278 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 podman[91457]: 2024-09-21 04:20:58.204392565 +0000 UTC m=+0.343522327 container start decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T04:20:58.279 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 podman[91457]: 2024-09-21 04:20:58.275687833 +0000 UTC m=+0.414817612 container attach decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T04:20:58.615 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 podman[91457]: 2024-09-21 04:20:58.393059564 +0000 UTC m=+0.532189334 container died decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD) 2024-09-21T04:20:58.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:58 smithi110 ceph-mon[92455]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T04:20:58.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:58.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:58 smithi148 ceph-mon[78701]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T04:20:58.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:20:59.400 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 podman[91480]: 2024-09-21 04:20:58.945693786 +0000 UTC m=+0.545044243 container remove decd715ae5f660a8d78165a1b1e7c718f2f3b78ef4510389d80a930f020a52e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:20:59.400 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.4.service: Deactivated successfully. 2024-09-21T04:20:59.400 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 systemd[1]: Stopped Ceph osd.4 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:20:59.400 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:58 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.4.service: Consumed 1min 2.828s CPU time. 2024-09-21T04:20:59.400 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 systemd[1]: Starting Ceph osd.4 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:20:59.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:20:59 smithi148 ceph-mon[78701]: pgmap v144: 65 pgs: 7 active+undersized, 2 peering, 2 stale+active+clean, 5 active+undersized+degraded, 49 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 13/222 objects degraded (5.856%) 2024-09-21T04:20:59.690 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 podman[91578]: 2024-09-21 04:20:59.397326807 +0000 UTC m=+0.110244599 container create ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T04:20:59.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 podman[91578]: 2024-09-21 04:20:59.303327435 +0000 UTC m=+0.016245240 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:20:59.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 podman[91578]: 2024-09-21 04:20:59.599736872 +0000 UTC m=+0.312654664 container init ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T04:20:59.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 podman[91578]: 2024-09-21 04:20:59.603117977 +0000 UTC m=+0.316035778 container start ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T04:20:59.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:20:59 smithi110 ceph-mon[92455]: pgmap v144: 65 pgs: 7 active+undersized, 2 peering, 2 stale+active+clean, 5 active+undersized+degraded, 49 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 13/222 objects degraded (5.856%) 2024-09-21T04:21:00.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 podman[91578]: 2024-09-21 04:20:59.691622999 +0000 UTC m=+0.404540800 container attach ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T04:21:00.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:00.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 bash[91578]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:00.191 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:00.191 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:20:59 smithi148 bash[91578]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:00 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 13/222 objects degraded (5.856%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:00.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:00 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 13/222 objects degraded (5.856%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:01 smithi110 ceph-mon[92455]: pgmap v145: 65 pgs: 16 active+undersized, 2 peering, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:01.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:01 smithi148 ceph-mon[78701]: pgmap v145: 65 pgs: 16 active+undersized, 2 peering, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T04:21:02.941 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T04:21:03.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-mon[78701]: pgmap v146: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:03.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-mon[78701]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate[91593]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:02 smithi148 bash[91578]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T04:21:03.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:03 smithi148 podman[91578]: 2024-09-21 04:21:03.016263034 +0000 UTC m=+3.729180846 container died ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:21:03.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:02 smithi110 ceph-mon[92455]: pgmap v146: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 965 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:03.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:02 smithi110 ceph-mon[92455]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T04:21:03.690 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:03 smithi148 podman[91810]: 2024-09-21 04:21:03.545145295 +0000 UTC m=+0.519772872 container remove ceced1077e7ec3d89e22c13a4ecda65ec7f4cb0aee49a9ea0e5e1c0bf21c840c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-activate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T04:21:04.025 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:03 smithi148 podman[91867]: 2024-09-21 04:21:03.65469395 +0000 UTC m=+0.016277021 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:21:04.025 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:03 smithi148 podman[91867]: 2024-09-21 04:21:03.787698048 +0000 UTC m=+0.149281119 container create 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:21:04.025 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:03 smithi148 podman[91867]: 2024-09-21 04:21:03.998401767 +0000 UTC m=+0.359984838 container init 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:21:04.025 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:04 smithi148 podman[91867]: 2024-09-21 04:21:04.00146693 +0000 UTC m=+0.363050001 container start 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T04:21:04.025 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:04 smithi148 ceph-osd[91886]: -- 172.21.15.148:0/774497453 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560dd905c1e0 con 0x560dd8fec000 2024-09-21T04:21:04.440 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:04 smithi148 bash[91867]: 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 2024-09-21T04:21:04.440 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:04 smithi148 systemd[1]: Started Ceph osd.4 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:21:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:05 smithi110 ceph-mon[92455]: pgmap v147: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:05 smithi148 ceph-mon[78701]: pgmap v147: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:05.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:05 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:21:05.641+0000 7f6dff746640 -1 Falling back to public interface 2024-09-21T04:21:06.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:06 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 36/222 objects degraded (16.216%), 14 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:06 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 36/222 objects degraded (16.216%), 14 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:08.013 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:07 smithi148 ceph-mon[78701]: pgmap v148: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:08.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:07 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:07 smithi110 ceph-mon[92455]: pgmap v148: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:07 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:09 smithi110 ceph-mon[92455]: pgmap v149: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:09.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:09.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:09.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:09 smithi148 ceph-mon[78701]: pgmap v149: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:11.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:11.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-09-21T04:21:11.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:11 smithi110 ceph-mon[92455]: pgmap v150: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:11.690 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:21:11.354+0000 7f6dff746640 -1 osd.4 80 log_to_monitors true 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-09-21T04:21:11.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:11 smithi148 ceph-mon[78701]: pgmap v150: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 969 MiB used, 535 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:12.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- 172.21.15.110:0/2153153658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c102f60 msgr2=0x7f863c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 --2- 172.21.15.110:0/2153153658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c102f60 0x7f863c105350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86300098b0 tx=0x7f863002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- 172.21.15.110:0/2153153658 shutdown_connections 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 --2- 172.21.15.110:0/2153153658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c102f60 0x7f863c105350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 --2- 172.21.15.110:0/2153153658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c100630 0x7f863c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- 172.21.15.110:0/2153153658 >> 172.21.15.110:0/2153153658 conn(0x7f863c0fa230 msgr2=0x7f863c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- 172.21.15.110:0/2153153658 shutdown_connections 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- 172.21.15.110:0/2153153658 wait complete. 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 Processor -- start 2024-09-21T04:21:12.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.560+0000 7f8642512640 1 -- start start 2024-09-21T04:21:12.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f8642512640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f8642512640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c102f60 0x7f863c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f8642512640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c196150 con 0x7f863c100630 2024-09-21T04:21:12.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f8642512640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1962c0 con 0x7f863c102f60 2024-09-21T04:21:12.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52796/0 (socket says 172.21.15.110:52796) 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 -- 172.21.15.110:0/1954274270 learned_addr learned my addr 172.21.15.110:0/1954274270 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863b7fe640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c102f60 0x7f863c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 -- 172.21.15.110:0/1954274270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c102f60 msgr2=0x7f863c195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c102f60 0x7f863c195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 -- 172.21.15.110:0/1954274270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009560 con 0x7f863c100630 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.561+0000 7f863bfff640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f862c00d900 tx=0x7f862c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.562+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f862c009c00 con 0x7f863c100630 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.562+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c19ad60 con 0x7f863c100630 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.562+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f862c00b660 con 0x7f863c100630 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.562+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f862c00f680 con 0x7f863c100630 2024-09-21T04:21:12.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.562+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c19b2a0 con 0x7f863c100630 2024-09-21T04:21:12.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.563+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c108790 con 0x7f863c100630 2024-09-21T04:21:12.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.563+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f862c010ab0 con 0x7f863c100630 2024-09-21T04:21:12.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.564+0000 7f86397fa640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 0x7f862407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.564+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f862c09e110 con 0x7f863c100630 2024-09-21T04:21:12.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.564+0000 7f863b7fe640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 0x7f862407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.565+0000 7f863b7fe640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 0x7f862407d730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8630009880 tx=0x7f86300314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:12.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.568+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f862c061110 con 0x7f863c100630 2024-09-21T04:21:12.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:12 smithi110 ceph-mon[92455]: from='osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:21:12.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:12 smithi110 ceph-mon[92455]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:21:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:12 smithi148 ceph-mon[78701]: from='osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:21:12.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:12 smithi148 ceph-mon[78701]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:21:12.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:21:12 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:21:12.390+0000 7f6df34e8640 -1 osd.4 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:21:12.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.766+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f863c19b4c0 con 0x7f862407b270 2024-09-21T04:21:12.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.767+0000 7f86397fa640 1 -- 172.21.15.110:0/1954274270 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f863c19b4c0 con 0x7f862407b270 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 msgr2=0x7f862407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 0x7f862407d730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8630009880 tx=0x7f86300314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 msgr2=0x7f863c195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f862c00d900 tx=0x7f862c00ddc0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 shutdown_connections 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f862407b270 0x7f862407d730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f863c102f60 0x7f863c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 --2- 172.21.15.110:0/1954274270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863c100630 0x7f863c195640 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 >> 172.21.15.110:0/1954274270 conn(0x7f863c0fa230 msgr2=0x7f863c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 shutdown_connections 2024-09-21T04:21:12.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.770+0000 7f8642512640 1 -- 172.21.15.110:0/1954274270 wait complete. 2024-09-21T04:21:12.781 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:21:12.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- 172.21.15.110:0/2127681407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91300ffb70 msgr2=0x7f91300fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 --2- 172.21.15.110:0/2127681407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91300ffb70 0x7f91300fff70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9124009880 tx=0x7f912402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- 172.21.15.110:0/2127681407 shutdown_connections 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 --2- 172.21.15.110:0/2127681407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130100d70 0x7f91301011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 --2- 172.21.15.110:0/2127681407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91300ffb70 0x7f91300fff70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- 172.21.15.110:0/2127681407 >> 172.21.15.110:0/2127681407 conn(0x7f91300fb2e0 msgr2=0x7f91300fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- 172.21.15.110:0/2127681407 shutdown_connections 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- 172.21.15.110:0/2127681407 wait complete. 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 Processor -- start 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.851+0000 7f9138031640 1 -- start start 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91300ffb70 0x7f9130195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130196180 con 0x7f9130100d70 2024-09-21T04:21:12.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91301962f0 con 0x7f91300ffb70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52816/0 (socket says 172.21.15.110:52816) 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 -- 172.21.15.110:0/3716299474 learned_addr learned my addr 172.21.15.110:0/3716299474 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9135dcd640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91300ffb70 0x7f9130195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 -- 172.21.15.110:0/3716299474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91300ffb70 msgr2=0x7f9130195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91300ffb70 0x7f9130195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 -- 172.21.15.110:0/3716299474 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009560 con 0x7f9130100d70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f91355cc640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f912c00d720 tx=0x7f912c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c0051f0 con 0x7f9130100d70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913019ad90 con 0x7f9130100d70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912c009ce0 con 0x7f9130100d70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c010460 con 0x7f9130100d70 2024-09-21T04:21:12.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.852+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913019b330 con 0x7f9130100d70 2024-09-21T04:21:12.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.853+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91300fff70 con 0x7f9130100d70 2024-09-21T04:21:12.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.854+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f912c005350 con 0x7f9130100d70 2024-09-21T04:21:12.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.854+0000 7f9122ffd640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 0x7f911c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:12.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.855+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f912c09dec0 con 0x7f9130100d70 2024-09-21T04:21:12.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.855+0000 7f9135dcd640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 0x7f911c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:12.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.855+0000 7f9135dcd640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 0x7f911c07d740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9124009230 tx=0x7f91240314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:12.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:12.859+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f912c060ec0 con 0x7f9130100d70 2024-09-21T04:21:13.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.053+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913002c540 con 0x7f911c07b280 2024-09-21T04:21:13.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.054+0000 7f9122ffd640 1 -- 172.21.15.110:0/3716299474 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f913002c540 con 0x7f911c07b280 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 msgr2=0x7f911c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 0x7f911c07d740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9124009230 tx=0x7f91240314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 msgr2=0x7f9130195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f912c00d720 tx=0x7f912c00dbe0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 shutdown_connections 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f911c07b280 0x7f911c07d740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9130100d70 0x7f9130195bb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 --2- 172.21.15.110:0/3716299474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91300ffb70 0x7f9130195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 >> 172.21.15.110:0/3716299474 conn(0x7f91300fb2e0 msgr2=0x7f91301044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 shutdown_connections 2024-09-21T04:21:13.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.056+0000 7f9138031640 1 -- 172.21.15.110:0/3716299474 wait complete. 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 -- 172.21.15.110:0/1319389582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00100630 msgr2=0x7f7c00102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 --2- 172.21.15.110:0/1319389582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00100630 0x7f7c00102a20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7be80098b0 tx=0x7f7be802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 -- 172.21.15.110:0/1319389582 shutdown_connections 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 --2- 172.21.15.110:0/1319389582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00102f60 0x7f7c00105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 --2- 172.21.15.110:0/1319389582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00100630 0x7f7c00102a20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 -- 172.21.15.110:0/1319389582 >> 172.21.15.110:0/1319389582 conn(0x7f7c000fa230 msgr2=0x7f7c000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 -- 172.21.15.110:0/1319389582 shutdown_connections 2024-09-21T04:21:13.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 -- 172.21.15.110:0/1319389582 wait complete. 2024-09-21T04:21:13.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.137+0000 7f7c050f3640 1 Processor -- start 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7c050f3640 1 -- start start 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7c050f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00100630 0x7f7c00195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7c050f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7c050f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00196170 con 0x7f7c00102f60 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7c050f3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001962e0 con 0x7f7c00100630 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52824/0 (socket says 172.21.15.110:52824) 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 -- 172.21.15.110:0/4072105030 learned_addr learned my addr 172.21.15.110:0/4072105030 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfed9d640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00100630 0x7f7c00195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 -- 172.21.15.110:0/4072105030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00100630 msgr2=0x7f7c00195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00100630 0x7f7c00195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 -- 172.21.15.110:0/4072105030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8009560 con 0x7f7c00102f60 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.138+0000 7f7bfe59c640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c920 tx=0x7f7bf000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.139+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0007b80 con 0x7f7c00102f60 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.139+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf0007ce0 con 0x7f7c00102f60 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.139+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf00187d0 con 0x7f7c00102f60 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.139+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0019ad80 con 0x7f7c00102f60 2024-09-21T04:21:13.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.139+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0019b350 con 0x7f7c00102f60 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.140+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c001087e0 con 0x7f7c00102f60 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.140+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f7bf000f450 con 0x7f7c00102f60 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.141+0000 7f7bf7fff640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 0x7f7bec07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.141+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f7bf009e460 con 0x7f7c00102f60 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.141+0000 7f7bfed9d640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 0x7f7bec07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.141+0000 7f7bfed9d640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 0x7f7bec07d730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7be8006010 tx=0x7f7be80314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.145+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f7bf00613b0 con 0x7f7c00102f60 2024-09-21T04:21:13.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.330+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c001038e0 con 0x7f7bec07b270 2024-09-21T04:21:13.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.340+0000 7f7bf7fff640 1 -- 172.21.15.110:0/4072105030 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7c001038e0 con 0x7f7bec07b270 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (12m) 74s ago 14m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (5m) 74s ago 14m 8400k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (5m) 6s ago 13m 8066k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (3m) 74s ago 14m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (2m) 6s ago 13m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (12m) 74s ago 14m 86.2M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (8m) 74s ago 8m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (8m) 74s ago 8m 16.4M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:21:13.343 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (8m) 6s ago 8m 18.1M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (8m) 6s ago 8m 20.0M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (5m) 74s ago 15m 516M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (4m) 6s ago 13m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (3m) 74s ago 15m 47.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (3m) 6s ago 13m 41.3M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (14m) 74s ago 14m 17.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (13m) 6s ago 13m 14.8M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (2m) 74s ago 12m 102M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (116s) 74s ago 11m 104M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (79s) 74s ago 11m 14.4M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (43s) 6s ago 10m 136M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (9s) 6s ago 10m 14.5M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (10m) 6s ago 9m 283M 2240M 18.2.4-871-g6a1bdf85 0adab276c97e b5919b7c9b5d 2024-09-21T04:21:13.344 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 74s ago 14m 54.1M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:21:13.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 msgr2=0x7f7bec07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 0x7f7bec07d730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7be8006010 tx=0x7f7be80314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 msgr2=0x7f7c00195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c920 tx=0x7f7bf000cde0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 shutdown_connections 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bec07b270 0x7f7bec07d730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c00102f60 0x7f7c00195ba0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 --2- 172.21.15.110:0/4072105030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00100630 0x7f7c00195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.342+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 >> 172.21.15.110:0/4072105030 conn(0x7f7c000fa230 msgr2=0x7f7c000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.343+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 shutdown_connections 2024-09-21T04:21:13.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.343+0000 7f7c050f3640 1 -- 172.21.15.110:0/4072105030 wait complete. 2024-09-21T04:21:13.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 -- 172.21.15.110:0/3294184924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 msgr2=0x7fc138101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 --2- 172.21.15.110:0/3294184924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc138101680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009880 tx=0x7fc12c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 -- 172.21.15.110:0/3294184924 shutdown_connections 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 --2- 172.21.15.110:0/3294184924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 0x7fc1380ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 --2- 172.21.15.110:0/3294184924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc138101680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 -- 172.21.15.110:0/3294184924 >> 172.21.15.110:0/3294184924 conn(0x7fc1380fb330 msgr2=0x7fc1380fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 -- 172.21.15.110:0/3294184924 shutdown_connections 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 -- 172.21.15.110:0/3294184924 wait complete. 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.431+0000 7fc13d808640 1 Processor -- start 2024-09-21T04:21:13.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13d808640 1 -- start start 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13d808640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 0x7fc138074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13d808640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13d808640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc138072de0 con 0x7fc138101280 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13d808640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc138072f50 con 0x7fc1380ff7c0 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52838/0 (socket says 172.21.15.110:52838) 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 -- 172.21.15.110:0/1148820457 learned_addr learned my addr 172.21.15.110:0/1148820457 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 -- 172.21.15.110:0/1148820457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 msgr2=0x7fc138074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc136ffd640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 0x7fc138074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 0x7fc138074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 -- 172.21.15.110:0/1148820457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c009560 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc1367fc640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c8f0 tx=0x7fc12800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.432+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc128007b80 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.433+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1280054e0 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.433+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc138073230 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.433+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc12800f450 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.433+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13819d770 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.434+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1380ffc40 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.434+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fc128004f60 con 0x7fc138101280 2024-09-21T04:21:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.435+0000 7fc13c806640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 0x7fc12007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.435+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fc12809e210 con 0x7fc138101280 2024-09-21T04:21:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.435+0000 7fc136ffd640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 0x7fc12007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.435+0000 7fc136ffd640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 0x7fc12007d730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc12c031540 tx=0x7fc12c000bc0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.439+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fc128061160 con 0x7fc138101280 2024-09-21T04:21:13.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:21:13.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: osdmap e83: 6 total, 5 up, 6 in 2024-09-21T04:21:13.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: from='osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:13.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:13.660 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:13.660 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:13 smithi110 ceph-mon[92455]: pgmap v152: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:21:13.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: osdmap e83: 6 total, 5 up, 6 in 2024-09-21T04:21:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: from='osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:13.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:13 smithi148 ceph-mon[78701]: pgmap v152: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 36/222 objects degraded (16.216%) 2024-09-21T04:21:13.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.696+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc13802c3e0 con 0x7fc138101280 2024-09-21T04:21:13.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.696+0000 7fc13c806640 1 -- 172.21.15.110:0/1148820457 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fc128064280 con 0x7fc138101280 2024-09-21T04:21:13.699 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:21:13.699 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:21:13.699 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 1, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 5 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 5, 2024-09-21T04:21:13.700 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 9 2024-09-21T04:21:13.701 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:21:13.701 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:21:13.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 msgr2=0x7fc12007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 0x7fc12007d730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc12c031540 tx=0x7fc12c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 msgr2=0x7fc1380728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c8f0 tx=0x7fc12800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 shutdown_connections 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc12007b270 0x7fc12007d730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc138101280 0x7fc1380728a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 --2- 172.21.15.110:0/1148820457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1380ff7c0 0x7fc138074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.698+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 >> 172.21.15.110:0/1148820457 conn(0x7fc1380fb330 msgr2=0x7fc138104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.699+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 shutdown_connections 2024-09-21T04:21:13.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.699+0000 7fc13d808640 1 -- 172.21.15.110:0/1148820457 wait complete. 2024-09-21T04:21:13.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 -- 172.21.15.110:0/3507479691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 msgr2=0x7f02e40ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 --2- 172.21.15.110:0/3507479691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e40ffc40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009880 tx=0x7f02cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 -- 172.21.15.110:0/3507479691 shutdown_connections 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 --2- 172.21.15.110:0/3507479691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e40ffc40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 --2- 172.21.15.110:0/3507479691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 -- 172.21.15.110:0/3507479691 >> 172.21.15.110:0/3507479691 conn(0x7f02e40fb330 msgr2=0x7f02e40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 -- 172.21.15.110:0/3507479691 shutdown_connections 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.782+0000 7f02e9242640 1 -- 172.21.15.110:0/3507479691 wait complete. 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 Processor -- start 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 -- start start 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4196350 con 0x7f02e40ff7c0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e9242640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41964c0 con 0x7f02e4101280 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52862/0 (socket says 172.21.15.110:52862) 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.783+0000 7f02e2d9d640 1 -- 172.21.15.110:0/3696578133 learned_addr learned my addr 172.21.15.110:0/3696578133 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02dbfff640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e2d9d640 1 -- 172.21.15.110:0/3696578133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 msgr2=0x7f02e4195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e2d9d640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e2d9d640 1 -- 172.21.15.110:0/3696578133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc009560 con 0x7f02e40ff7c0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02dbfff640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e2d9d640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b400 tx=0x7f02d400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d4011800 con 0x7f02e40ff7c0 2024-09-21T04:21:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02d4011e20 con 0x7f02e40ff7c0 2024-09-21T04:21:13.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e419af60 con 0x7f02e40ff7c0 2024-09-21T04:21:13.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d400f2e0 con 0x7f02e40ff7c0 2024-09-21T04:21:13.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.784+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e419b4a0 con 0x7f02e40ff7c0 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.785+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40ffcc0 con 0x7f02e40ff7c0 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.786+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f02d4010380 con 0x7f02e40ff7c0 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.786+0000 7f02e0d99640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 0x7f02d007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.786+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f02d409dff0 con 0x7f02e40ff7c0 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.787+0000 7f02dbfff640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 0x7f02d007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:13.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.787+0000 7f02dbfff640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 0x7f02d007d730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000c00 tx=0x7f02cc0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:13.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:13.791+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f02d4060f40 con 0x7f02e40ff7c0 2024-09-21T04:21:14.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.010+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f02e402c480 con 0x7f02e40ff7c0 2024-09-21T04:21:14.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.010+0000 7f02e0d99640 1 -- 172.21.15.110:0/3696578133 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f02d4064060 con 0x7f02e40ff7c0 2024-09-21T04:21:14.013 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:21:14.013 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:21:14.014 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:21:14.015 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:14.016 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:14.017 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:14.017 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 msgr2=0x7f02d007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 0x7f02d007d730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000c00 tx=0x7f02cc0314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 msgr2=0x7f02e4195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b400 tx=0x7f02d400b8c0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 shutdown_connections 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f02d007b270 0x7f02d007d730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e4101280 0x7f02e4195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 --2- 172.21.15.110:0/3696578133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e40ff7c0 0x7f02e4195840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 >> 172.21.15.110:0/3696578133 conn(0x7f02e40fb330 msgr2=0x7f02e41044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 shutdown_connections 2024-09-21T04:21:14.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.013+0000 7f02e9242640 1 -- 172.21.15.110:0/3696578133 wait complete. 2024-09-21T04:21:14.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 -- 172.21.15.110:0/2986304681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43680ffb90 msgr2=0x7f43680fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 --2- 172.21.15.110:0/2986304681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43680ffb90 0x7f43680fff90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4358009880 tx=0x7f435802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 -- 172.21.15.110:0/2986304681 shutdown_connections 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 --2- 172.21.15.110:0/2986304681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4368100d90 0x7f4368101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 --2- 172.21.15.110:0/2986304681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43680ffb90 0x7f43680fff90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 -- 172.21.15.110:0/2986304681 >> 172.21.15.110:0/2986304681 conn(0x7f43680fb320 msgr2=0x7f43680fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 -- 172.21.15.110:0/2986304681 shutdown_connections 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.094+0000 7f43705d1640 1 -- 172.21.15.110:0/2986304681 wait complete. 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 Processor -- start 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 -- start start 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43680ffb90 0x7f43680741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368074720 con 0x7f4368100d90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f43705d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368072de0 con 0x7f43680ffb90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52886/0 (socket says 172.21.15.110:52886) 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 -- 172.21.15.110:0/2942704717 learned_addr learned my addr 172.21.15.110:0/2942704717 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436e36d640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43680ffb90 0x7f43680741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 -- 172.21.15.110:0/2942704717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43680ffb90 msgr2=0x7f43680741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43680ffb90 0x7f43680741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 -- 172.21.15.110:0/2942704717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4358009560 con 0x7f4368100d90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.095+0000 7f436db6c640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f436400c8a0 tx=0x7f436400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.096+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4364007b80 con 0x7f4368100d90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.096+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4364007ce0 con 0x7f4368100d90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.096+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4368073060 con 0x7f4368100d90 2024-09-21T04:21:14.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.096+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4364018810 con 0x7f4368100d90 2024-09-21T04:21:14.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.096+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4368073630 con 0x7f4368100d90 2024-09-21T04:21:14.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.097+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436818f710 con 0x7f4368100d90 2024-09-21T04:21:14.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.097+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4364018970 con 0x7f4368100d90 2024-09-21T04:21:14.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.098+0000 7f435f7fe640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 0x7f435407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.098+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f436409e5e0 con 0x7f4368100d90 2024-09-21T04:21:14.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.098+0000 7f436e36d640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 0x7f435407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.098+0000 7f436e36d640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 0x7f435407d730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4358006010 tx=0x7f4358000bc0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:14.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.102+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4364061510 con 0x7f4368100d90 2024-09-21T04:21:14.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.298+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f436802c4f0 con 0x7f435407b270 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.299+0000 7f435f7fe640 1 -- 172.21.15.110:0/2942704717 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f436802c4f0 con 0x7f435407b270 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:21:14.302 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "11/23 daemons upgraded", 2024-09-21T04:21:14.303 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:21:14.303 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:21:14.303 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:21:14.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 msgr2=0x7f435407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 0x7f435407d730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4358006010 tx=0x7f4358000bc0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 msgr2=0x7f43680728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f436400c8a0 tx=0x7f436400cd60 comp rx=0 tx=0).stop 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 shutdown_connections 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f435407b270 0x7f435407d730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4368100d90 0x7f43680728a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 --2- 172.21.15.110:0/2942704717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43680ffb90 0x7f43680741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 >> 172.21.15.110:0/2942704717 conn(0x7f43680fb320 msgr2=0x7f4368104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 shutdown_connections 2024-09-21T04:21:14.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.301+0000 7f43705d1640 1 -- 172.21.15.110:0/2942704717 wait complete. 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.391+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/1991251770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 msgr2=0x7f1df81000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.391+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/1991251770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81000b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009880 tx=0x7f1dec02a4b0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/1991251770 shutdown_connections 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/1991251770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81000b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/1991251770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 0x7f1df80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/1991251770 >> 172.21.15.110:0/1991251770 conn(0x7f1df80fa1c0 msgr2=0x7f1df80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/1991251770 shutdown_connections 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.392+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/1991251770 wait complete. 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.393+0000 7f1dfe8dd640 1 Processor -- start 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.393+0000 7f1dfe8dd640 1 -- start start 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.393+0000 7f1dfe8dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 0x7f1df81a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.393+0000 7f1dfe8dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.394+0000 7f1dfe8dd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df81a4ff0 con 0x7f1df80ffc30 2024-09-21T04:21:14.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.394+0000 7f1dfe8dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df81a9950 con 0x7f1df80fea30 2024-09-21T04:21:14.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.394+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.394+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52904/0 (socket says 172.21.15.110:52904) 2024-09-21T04:21:14.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.394+0000 7f1df77fe640 1 -- 172.21.15.110:0/4194127795 learned_addr learned my addr 172.21.15.110:0/4194127795 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.395+0000 7f1df7fff640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 0x7f1df81a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.396+0000 7f1df77fe640 1 -- 172.21.15.110:0/4194127795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 msgr2=0x7f1df81a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.396+0000 7f1df77fe640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 0x7f1df81a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.396+0000 7f1df77fe640 1 -- 172.21.15.110:0/4194127795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009560 con 0x7f1df80ffc30 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.396+0000 7f1df77fe640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006010 tx=0x7f1dec0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:14.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.396+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dec004aa0 con 0x7f1df80ffc30 2024-09-21T04:21:14.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.397+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dec004c00 con 0x7f1df80ffc30 2024-09-21T04:21:14.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.397+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dec003ac0 con 0x7f1df80ffc30 2024-09-21T04:21:14.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.397+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df81a9af0 con 0x7f1df80ffc30 2024-09-21T04:21:14.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.397+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df81a9fa0 con 0x7f1df80ffc30 2024-09-21T04:21:14.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.400+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1dec003c20 con 0x7f1df80ffc30 2024-09-21T04:21:14.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.400+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df81071e0 con 0x7f1df80ffc30 2024-09-21T04:21:14.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.401+0000 7f1df57fa640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 0x7f1de007d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:14.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.401+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f1dec0bf540 con 0x7f1df80ffc30 2024-09-21T04:21:14.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.401+0000 7f1df7fff640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 0x7f1de007d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:14.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.402+0000 7f1df7fff640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 0x7f1de007d850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005fd0 tx=0x7f1de8005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:14.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.408+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1dec0824f0 con 0x7f1df80ffc30 2024-09-21T04:21:14.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.656+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1df81aa510 con 0x7f1df80ffc30 2024-09-21T04:21:14.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:14.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:14.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:21:14.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631] boot 2024-09-21T04:21:14.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: osdmap e84: 6 total, 6 up, 6 in 2024-09-21T04:21:14.660 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:21:14.660 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1148820457' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:14.660 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:14 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3696578133' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:21:14.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.656+0000 7f1df57fa640 1 -- 172.21.15.110:0/4194127795 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+961 (secure 0 0 0) 0x7f1dec085610 con 0x7f1df80ffc30 2024-09-21T04:21:14.660 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN Degraded data redundancy: 36/222 objects degraded (16.216%), 14 pgs degraded 2024-09-21T04:21:14.660 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/222 objects degraded (16.216%), 14 pgs degraded 2024-09-21T04:21:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-09-21T04:21:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.9 is active+undersized+degraded, acting [1,3] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-09-21T04:21:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-09-21T04:21:14.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 msgr2=0x7f1de007d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 0x7f1de007d850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005fd0 tx=0x7f1de8005850 comp rx=0 tx=0).stop 2024-09-21T04:21:14.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 msgr2=0x7f1df81a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:14.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006010 tx=0x7f1dec0055a0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 shutdown_connections 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1de007b390 0x7f1de007d850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1df80ffc30 0x7f1df81a49a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 --2- 172.21.15.110:0/4194127795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1df80fea30 0x7f1df81a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 >> 172.21.15.110:0/4194127795 conn(0x7f1df80fa1c0 msgr2=0x7f1df8102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.658+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 shutdown_connections 2024-09-21T04:21:14.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:14.659+0000 7f1dfe8dd640 1 -- 172.21.15.110:0/4194127795 wait complete. 2024-09-21T04:21:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:21:14.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: osd.4 [v2:172.21.15.148:6808/193056631,v1:172.21.15.148:6809/193056631] boot 2024-09-21T04:21:14.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: osdmap e84: 6 total, 6 up, 6 in 2024-09-21T04:21:14.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:21:14.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1148820457' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:14.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:14 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3696578133' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:21:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:15 smithi110 ceph-mon[92455]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:15 smithi110 ceph-mon[92455]: osdmap e85: 6 total, 6 up, 6 in 2024-09-21T04:21:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:15 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4194127795' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:21:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:15 smithi110 ceph-mon[92455]: pgmap v155: 65 pgs: 13 peering, 11 active+undersized, 8 active+undersized+degraded, 33 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 15/222 objects degraded (6.757%) 2024-09-21T04:21:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:15 smithi148 ceph-mon[78701]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:15 smithi148 ceph-mon[78701]: osdmap e85: 6 total, 6 up, 6 in 2024-09-21T04:21:15.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:15 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4194127795' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:21:15.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:15 smithi148 ceph-mon[78701]: pgmap v155: 65 pgs: 13 peering, 11 active+undersized, 8 active+undersized+degraded, 33 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 15/222 objects degraded (6.757%) 2024-09-21T04:21:16.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:16 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 8 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:16.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:16 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 8 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:17.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:17 smithi110 ceph-mon[92455]: pgmap v156: 65 pgs: 13 peering, 8 active+undersized, 4 active+undersized+degraded, 40 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 9/222 objects degraded (4.054%) 2024-09-21T04:21:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:17 smithi148 ceph-mon[78701]: pgmap v156: 65 pgs: 13 peering, 8 active+undersized, 4 active+undersized+degraded, 40 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 9/222 objects degraded (4.054%) 2024-09-21T04:21:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:18 smithi110 ceph-mon[92455]: pgmap v157: 65 pgs: 10 peering, 55 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:18 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/222 objects degraded (4.054%), 4 pgs degraded) 2024-09-21T04:21:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:18 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:21:19.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:18 smithi148 ceph-mon[78701]: pgmap v157: 65 pgs: 10 peering, 55 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:19.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:18 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/222 objects degraded (4.054%), 4 pgs degraded) 2024-09-21T04:21:19.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:18 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:21:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:20 smithi110 ceph-mon[92455]: pgmap v158: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:21.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:20 smithi148 ceph-mon[78701]: pgmap v158: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:22 smithi110 ceph-mon[92455]: pgmap v159: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:22 smithi148 ceph-mon[78701]: pgmap v159: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:24.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:24.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:25 smithi110 ceph-mon[92455]: pgmap v160: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:25.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:25 smithi148 ceph-mon[78701]: pgmap v160: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:26 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:21:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:26 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:21:26.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:26 smithi110 ceph-mon[92455]: Upgrade: osd.5 is safe to restart 2024-09-21T04:21:26.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:26 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:21:26.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:26 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:21:26.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:26 smithi148 ceph-mon[78701]: Upgrade: osd.5 is safe to restart 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: Upgrade: Updating osd.5 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: Deploying daemon osd.5 on smithi148 2024-09-21T04:21:28.085 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:27 smithi148 ceph-mon[78701]: pgmap v161: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: Upgrade: Updating osd.5 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: Deploying daemon osd.5 on smithi148 2024-09-21T04:21:28.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:27 smithi110 ceph-mon[92455]: pgmap v161: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:29 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:29 smithi110 ceph-mon[92455]: pgmap v162: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:29.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:29 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:29.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:29 smithi148 ceph-mon[78701]: pgmap v162: 65 pgs: 65 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:29.708 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:29 smithi148 systemd[1]: Stopping Ceph osd.5 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:21:30.185 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[54279]: 2024-09-21T04:21:29.704+0000 7f4feb93e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:21:30.185 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[54279]: 2024-09-21T04:21:29.704+0000 7f4feb93e640 -1 osd.5 85 *** Got signal Terminated *** 2024-09-21T04:21:30.185 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:29 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[54279]: 2024-09-21T04:21:29.704+0000 7f4feb93e640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:21:30.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:30 smithi148 ceph-mon[78701]: osd.5 marked itself down and dead 2024-09-21T04:21:30.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:30 smithi110 ceph-mon[92455]: osd.5 marked itself down and dead 2024-09-21T04:21:30.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:30 smithi148 podman[96224]: 2024-09-21 04:21:30.614595485 +0000 UTC m=+1.006213899 container died b5919b7c9b5d12af3f4b4e7c9283010601a3847d5fa2a7683a71edc829f6f766 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-6a1bdf8, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:21:31.249 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:31 smithi148 ceph-mon[78701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:21:31.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:31 smithi148 ceph-mon[78701]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-21T04:21:31.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:31 smithi148 ceph-mon[78701]: osdmap e86: 6 total, 5 up, 6 in 2024-09-21T04:21:31.250 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:31 smithi148 ceph-mon[78701]: pgmap v164: 65 pgs: 14 stale+active+clean, 51 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:31.250 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:30 smithi148 podman[96224]: 2024-09-21 04:21:30.963635671 +0000 UTC m=+1.355254072 container cleanup b5919b7c9b5d12af3f4b4e7c9283010601a3847d5fa2a7683a71edc829f6f766 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-6a1bdf8) 2024-09-21T04:21:31.250 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:30 smithi148 bash[96224]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5 2024-09-21T04:21:31.573 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96239]: 2024-09-21 04:21:31.247695602 +0000 UTC m=+0.629585360 container remove b5919b7c9b5d12af3f4b4e7c9283010601a3847d5fa2a7683a71edc829f6f766 (image=quay.ceph.io/ceph-ci/ceph@sha256:de4fa798d334aaeb6a820ab2dad2fbcb86bcd7f118d94ed3a02ce126be31cec6, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=reef-6a1bdf8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:21:31.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:31 smithi110 ceph-mon[92455]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:21:31.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:31 smithi110 ceph-mon[92455]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-21T04:21:31.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:31 smithi110 ceph-mon[92455]: osdmap e86: 6 total, 5 up, 6 in 2024-09-21T04:21:31.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:31 smithi110 ceph-mon[92455]: pgmap v164: 65 pgs: 14 stale+active+clean, 51 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:31.870 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.47396242 +0000 UTC m=+0.015990134 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:21:31.870 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.582096411 +0000 UTC m=+0.124124152 container create c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T04:21:31.870 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.784303698 +0000 UTC m=+0.326331397 container init c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T04:21:31.870 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.787590715 +0000 UTC m=+0.329618414 container start c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T04:21:31.870 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.867458181 +0000 UTC m=+0.409485898 container attach c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T04:21:32.190 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:31 smithi148 podman[96330]: 2024-09-21 04:21:31.978670917 +0000 UTC m=+0.520698641 container died c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T04:21:32.606 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:32 smithi148 ceph-mon[78701]: osdmap e87: 6 total, 5 up, 6 in 2024-09-21T04:21:32.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:32 smithi110 ceph-mon[92455]: osdmap e87: 6 total, 5 up, 6 in 2024-09-21T04:21:32.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:32 smithi148 podman[96354]: 2024-09-21 04:21:32.604546261 +0000 UTC m=+0.616753001 container remove c2b524e1a2498db5661699e352c99e52c47da0db521e391313b821c30222a1ae (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:21:32.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:32 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.5.service: Deactivated successfully. 2024-09-21T04:21:32.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:32 smithi148 systemd[1]: Stopped Ceph osd.5 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:21:32.941 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:32 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.5.service: Consumed 1min 6.671s CPU time. 2024-09-21T04:21:32.941 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:32 smithi148 systemd[1]: Starting Ceph osd.5 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:21:33.368 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:33 smithi148 ceph-mon[78701]: pgmap v166: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 11/222 objects degraded (4.955%) 2024-09-21T04:21:33.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 podman[96453]: 2024-09-21 04:21:32.950993556 +0000 UTC m=+0.015916315 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:21:33.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 podman[96453]: 2024-09-21 04:21:33.055659755 +0000 UTC m=+0.120582558 container create 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:21:33.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 podman[96453]: 2024-09-21 04:21:33.291521296 +0000 UTC m=+0.356444065 container init 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:21:33.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 podman[96453]: 2024-09-21 04:21:33.29492424 +0000 UTC m=+0.359847008 container start 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:21:33.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 podman[96453]: 2024-09-21 04:21:33.366142512 +0000 UTC m=+0.431065285 container attach 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:21:33.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:33 smithi110 ceph-mon[92455]: pgmap v166: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 11/222 objects degraded (4.955%) 2024-09-21T04:21:33.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:33.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 bash[96453]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:33.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:33.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:33 smithi148 bash[96453]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:34.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:34 smithi110 ceph-mon[92455]: Health check failed: Degraded data redundancy: 11/222 objects degraded (4.955%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:34.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:34 smithi148 ceph-mon[78701]: Health check failed: Degraded data redundancy: 11/222 objects degraded (4.955%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:35 smithi110 ceph-mon[92455]: pgmap v167: 65 pgs: 16 active+undersized, 1 stale+active+clean, 10 active+undersized+degraded, 38 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 28/222 objects degraded (12.613%) 2024-09-21T04:21:35.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:35 smithi148 ceph-mon[78701]: pgmap v167: 65 pgs: 16 active+undersized, 1 stale+active+clean, 10 active+undersized+degraded, 38 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 28/222 objects degraded (12.613%) 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:21:36.691 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:21:36.692 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-09-21T04:21:36.692 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-09-21T04:21:37.103 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-mon[78701]: pgmap v168: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:21:37.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:21:37.105 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate[96468]: --> ceph-volume lvm activate successful for osd ID: 5 2024-09-21T04:21:37.105 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 bash[96453]: --> ceph-volume lvm activate successful for osd ID: 5 2024-09-21T04:21:37.105 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:36 smithi148 podman[96453]: 2024-09-21 04:21:36.766083734 +0000 UTC m=+3.831006493 container died 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True) 2024-09-21T04:21:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:36 smithi110 ceph-mon[92455]: pgmap v168: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:37.639 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 podman[96691]: 2024-09-21 04:21:37.386165042 +0000 UTC m=+0.610333888 container remove 838900df6f9f66a5d1f5b5377c40e3c3b91354320ee88993d21ad8c6b33f5dca (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-21T04:21:37.639 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 podman[96750]: 2024-09-21 04:21:37.493758167 +0000 UTC m=+0.015764030 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:21:37.639 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 podman[96750]: 2024-09-21 04:21:37.637254041 +0000 UTC m=+0.159259889 container create c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T04:21:37.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 podman[96750]: 2024-09-21 04:21:37.847742201 +0000 UTC m=+0.369748063 container init c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:21:37.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 podman[96750]: 2024-09-21 04:21:37.850757995 +0000 UTC m=+0.372763850 container start c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T04:21:37.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 ceph-osd[96770]: -- 172.21.15.148:0/4073776846 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed82ec81e0 con 0x55ed82067c00 2024-09-21T04:21:38.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 bash[96750]: c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e 2024-09-21T04:21:38.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:37 smithi148 systemd[1]: Started Ceph osd.5 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:39.364 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:38 smithi148 ceph-mon[78701]: pgmap v169: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:38 smithi110 ceph-mon[92455]: pgmap v169: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:40.038 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:39 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:21:39.718+0000 7f003f85b640 -1 Falling back to public interface 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: pgmap v170: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:41 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 30/222 objects degraded (13.514%), 12 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:41.937 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.938 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.938 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: pgmap v170: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:41.938 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.938 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:41.938 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:41 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 30/222 objects degraded (13.514%), 12 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:43.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:42 smithi148 ceph-mon[78701]: pgmap v171: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:42 smithi110 ceph-mon[92455]: pgmap v171: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:44.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 -- 172.21.15.110:0/1710157464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0ffc30 msgr2=0x7f0e7c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:44.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 --2- 172.21.15.110:0/1710157464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c1000b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009880 tx=0x7f0e6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 -- 172.21.15.110:0/1710157464 shutdown_connections 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 --2- 172.21.15.110:0/1710157464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c1000b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 --2- 172.21.15.110:0/1710157464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 -- 172.21.15.110:0/1710157464 >> 172.21.15.110:0/1710157464 conn(0x7f0e7c0fa1c0 msgr2=0x7f0e7c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.741+0000 7f0e839b5640 1 -- 172.21.15.110:0/1710157464 shutdown_connections 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 -- 172.21.15.110:0/1710157464 wait complete. 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 Processor -- start 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 -- start start 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:44.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c196160 con 0x7f0e7c0fea30 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e839b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1962d0 con 0x7f0e7c0ffc30 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e81751640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e81751640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58282/0 (socket says 172.21.15.110:58282) 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e81751640 1 -- 172.21.15.110:0/693948551 learned_addr learned my addr 172.21.15.110:0/693948551 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.742+0000 7f0e80f50640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e81751640 1 -- 172.21.15.110:0/693948551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0ffc30 msgr2=0x7f0e7c195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e81751640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e81751640 1 -- 172.21.15.110:0/693948551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009560 con 0x7f0e7c0fea30 2024-09-21T04:21:44.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e81751640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c920 tx=0x7f0e7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:44.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e780041f0 con 0x7f0e7c0fea30 2024-09-21T04:21:44.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c19ad70 con 0x7f0e7c0fea30 2024-09-21T04:21:44.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e78007e70 con 0x7f0e7c0fea30 2024-09-21T04:21:44.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e780049d0 con 0x7f0e7c0fea30 2024-09-21T04:21:44.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.743+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c19b2b0 con 0x7f0e7c0fea30 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.744+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c107640 con 0x7f0e7c0fea30 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.745+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f0e78004350 con 0x7f0e7c0fea30 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.745+0000 7f0e727fc640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 0x7f0e6807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.745+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f0e7809e100 con 0x7f0e7c0fea30 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.745+0000 7f0e80f50640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 0x7f0e6807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:44.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.746+0000 7f0e80f50640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 0x7f0e6807d740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c000c00 tx=0x7f0e6c0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:44.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.750+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f0e78061100 con 0x7f0e7c0fea30 2024-09-21T04:21:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.945+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e7c0683a0 con 0x7f0e6807b280 2024-09-21T04:21:44.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.947+0000 7f0e727fc640 1 -- 172.21.15.110:0/693948551 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0e7c0683a0 con 0x7f0e6807b280 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 msgr2=0x7f0e6807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 0x7f0e6807d740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c000c00 tx=0x7f0e6c0314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 msgr2=0x7f0e7c195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c920 tx=0x7f0e7800cde0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 shutdown_connections 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0e6807b280 0x7f0e6807d740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7c0ffc30 0x7f0e7c195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 --2- 172.21.15.110:0/693948551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c0fea30 0x7f0e7c195650 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 >> 172.21.15.110:0/693948551 conn(0x7f0e7c0fa1c0 msgr2=0x7f0e7c067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 shutdown_connections 2024-09-21T04:21:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:44.949+0000 7f0e839b5640 1 -- 172.21.15.110:0/693948551 wait complete. 2024-09-21T04:21:44.961 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 -- 172.21.15.110:0/2835557803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c4100d90 msgr2=0x7f10c4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/2835557803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c4100d90 0x7f10c4101210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009880 tx=0x7f10b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 -- 172.21.15.110:0/2835557803 shutdown_connections 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/2835557803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c4100d90 0x7f10c4101210 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/2835557803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c40ffb90 0x7f10c40fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 -- 172.21.15.110:0/2835557803 >> 172.21.15.110:0/2835557803 conn(0x7f10c40fb320 msgr2=0x7f10c40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 -- 172.21.15.110:0/2835557803 shutdown_connections 2024-09-21T04:21:45.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.030+0000 7f10c9cd1640 1 -- 172.21.15.110:0/2835557803 wait complete. 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 Processor -- start 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 -- start start 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4100d90 0x7f10c4195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4196160 con 0x7f10c40ffb90 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c9cd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41962d0 con 0x7f10c4100d90 2024-09-21T04:21:45.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58300/0 (socket says 172.21.15.110:58300) 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 -- 172.21.15.110:0/1632046876 learned_addr learned my addr 172.21.15.110:0/1632046876 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c2ffd640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4100d90 0x7f10c4195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 -- 172.21.15.110:0/1632046876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4100d90 msgr2=0x7f10c4195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4100d90 0x7f10c4195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.031+0000 7f10c37fe640 1 -- 172.21.15.110:0/1632046876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4009560 con 0x7f10c40ffb90 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c37fe640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10b800d720 tx=0x7f10b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b80041f0 con 0x7f10c40ffb90 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c419ad70 con 0x7f10c40ffb90 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c419b2b0 con 0x7f10c40ffb90 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10b8004350 con 0x7f10c40ffb90 2024-09-21T04:21:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.032+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b8003df0 con 0x7f10c40ffb90 2024-09-21T04:21:45.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.033+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0005280 con 0x7f10c40ffb90 2024-09-21T04:21:45.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.034+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f10b80035f0 con 0x7f10c40ffb90 2024-09-21T04:21:45.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.034+0000 7f10c0ff9640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 0x7f10ac07d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.034+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f10b809ddb0 con 0x7f10c40ffb90 2024-09-21T04:21:45.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.034+0000 7f10c2ffd640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 0x7f10ac07d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.035+0000 7f10c2ffd640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 0x7f10ac07d6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009230 tx=0x7f10b40058d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.039+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f10b8060db0 con 0x7f10c40ffb90 2024-09-21T04:21:45.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.235+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10b0000bf0 con 0x7f10ac07b220 2024-09-21T04:21:45.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.236+0000 7f10c0ff9640 1 -- 172.21.15.110:0/1632046876 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f10b0000bf0 con 0x7f10ac07b220 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 msgr2=0x7f10ac07d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 0x7f10ac07d6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009230 tx=0x7f10b40058d0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 msgr2=0x7f10c4195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f10b800d720 tx=0x7f10b800dbe0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 shutdown_connections 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f10ac07b220 0x7f10ac07d6e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4100d90 0x7f10c4195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 --2- 172.21.15.110:0/1632046876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10c40ffb90 0x7f10c4195650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 >> 172.21.15.110:0/1632046876 conn(0x7f10c40fb320 msgr2=0x7f10c4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 shutdown_connections 2024-09-21T04:21:45.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.239+0000 7f10c9cd1640 1 -- 172.21.15.110:0/1632046876 wait complete. 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 -- 172.21.15.110:0/772733756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 msgr2=0x7f0a48100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/772733756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48100120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009880 tx=0x7f0a3002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 -- 172.21.15.110:0/772733756 shutdown_connections 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/772733756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48100120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/772733756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 0x7f0a480feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 -- 172.21.15.110:0/772733756 >> 172.21.15.110:0/772733756 conn(0x7f0a480fa230 msgr2=0x7f0a480fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 -- 172.21.15.110:0/772733756 shutdown_connections 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.320+0000 7f0a4d3da640 1 -- 172.21.15.110:0/772733756 wait complete. 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 Processor -- start 2024-09-21T04:21:45.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 -- start start 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 0x7f0a48195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48196140 con 0x7f0a480ffca0 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a4d3da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481962b0 con 0x7f0a480feaa0 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.321+0000 7f0a3e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 0x7f0a48195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58326/0 (socket says 172.21.15.110:58326) 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 -- 172.21.15.110:0/4118342931 learned_addr learned my addr 172.21.15.110:0/4118342931 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 -- 172.21.15.110:0/4118342931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 msgr2=0x7f0a48195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 0x7f0a48195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 -- 172.21.15.110:0/4118342931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009560 con 0x7f0a480ffca0 2024-09-21T04:21:45.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a3e7fc640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005520 tx=0x7f0a30005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a30004aa0 con 0x7f0a480ffca0 2024-09-21T04:21:45.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a30004c00 con 0x7f0a480ffca0 2024-09-21T04:21:45.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4819acf0 con 0x7f0a480ffca0 2024-09-21T04:21:45.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a30003b10 con 0x7f0a480ffca0 2024-09-21T04:21:45.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.322+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4819b260 con 0x7f0a480ffca0 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.323+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a48107690 con 0x7f0a480ffca0 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.324+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f0a30031d20 con 0x7f0a480ffca0 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.324+0000 7f0a44ff9640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 0x7f0a3407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.324+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f0a300bf4e0 con 0x7f0a480ffca0 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.324+0000 7f0a46ffd640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 0x7f0a3407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.325+0000 7f0a46ffd640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 0x7f0a3407d730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ffb00 tx=0x7f0a3800b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.329+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f0a30082560 con 0x7f0a480ffca0 2024-09-21T04:21:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.525+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a4819b4a0 con 0x7f0a3407b270 2024-09-21T04:21:45.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.534+0000 7f0a44ff9640 1 -- 172.21.15.110:0/4118342931 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0a4819b4a0 con 0x7f0a3407b270 2024-09-21T04:21:45.537 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:21:45.537 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (13m) 106s ago 14m 25.4M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:21:45.537 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (6m) 106s ago 15m 8400k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:21:45.537 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (5m) 4s ago 14m 8376k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (3m) 106s ago 15m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (3m) 4s ago 14m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (12m) 106s ago 14m 86.2M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (8m) 106s ago 8m 153M - 18.2.4-871-g6a1bdf85 0adab276c97e e007af4e5715 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (9m) 106s ago 9m 16.4M - 18.2.4-871-g6a1bdf85 0adab276c97e 59c62942d14a 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (8m) 4s ago 8m 18.3M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (9m) 4s ago 9m 20.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (5m) 106s ago 16m 516M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (4m) 4s ago 14m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (4m) 106s ago 16m 47.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (4m) 4s ago 14m 38.2M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (15m) 106s ago 15m 17.1M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:21:45.538 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (14m) 4s ago 14m 15.2M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (3m) 106s ago 12m 102M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (2m) 106s ago 12m 104M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (112s) 106s ago 11m 14.4M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (75s) 4s ago 11m 137M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (41s) 4s ago 10m 99.7M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (8s) 4s ago 10m 11.7M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 106s ago 14m 54.1M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:21:45.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.536+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 msgr2=0x7f0a3407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.536+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 0x7f0a3407d730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ffb00 tx=0x7f0a3800b390 comp rx=0 tx=0).stop 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.536+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 msgr2=0x7f0a48195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.536+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005520 tx=0x7f0a30005550 comp rx=0 tx=0).stop 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.536+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 shutdown_connections 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f0a3407b270 0x7f0a3407d730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a480ffca0 0x7f0a48195b70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 --2- 172.21.15.110:0/4118342931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a480feaa0 0x7f0a48195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 >> 172.21.15.110:0/4118342931 conn(0x7f0a480fa230 msgr2=0x7f0a480684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 shutdown_connections 2024-09-21T04:21:45.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.537+0000 7f0a4d3da640 1 -- 172.21.15.110:0/4118342931 wait complete. 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 -- 172.21.15.110:0/331892996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf00100d60 msgr2=0x7fdf001011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 --2- 172.21.15.110:0/331892996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf00100d60 0x7fdf001011e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009880 tx=0x7fdef402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 -- 172.21.15.110:0/331892996 shutdown_connections 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 --2- 172.21.15.110:0/331892996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf00100d60 0x7fdf001011e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 --2- 172.21.15.110:0/331892996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf000ffb60 0x7fdf000fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.619+0000 7fdf06448640 1 -- 172.21.15.110:0/331892996 >> 172.21.15.110:0/331892996 conn(0x7fdf000fb330 msgr2=0x7fdf000fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.620+0000 7fdf06448640 1 -- 172.21.15.110:0/331892996 shutdown_connections 2024-09-21T04:21:45.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.620+0000 7fdf06448640 1 -- 172.21.15.110:0/331892996 wait complete. 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all osd 2024-09-21T04:21:45.624 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: Upgrade: Setting require_osd_release to 19 squid 2024-09-21T04:21:45.625 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd require-osd-release", "release": "squid"} : dispatch 2024-09-21T04:21:45.626 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:45 smithi110 ceph-mon[92455]: pgmap v172: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.620+0000 7fdf06448640 1 Processor -- start 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.620+0000 7fdf06448640 1 -- start start 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdf06448640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 0x7fdf0019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdf06448640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdf06448640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0019c740 con 0x7fdf000ffb60 2024-09-21T04:21:45.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdf06448640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0019c8b0 con 0x7fdf00100d60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdeff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdefffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 0x7fdf0019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdeff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.110:49836/0 (socket says 172.21.15.110:49836) 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.621+0000 7fdeff7fe640 1 -- 172.21.15.110:0/856477184 learned_addr learned my addr 172.21.15.110:0/856477184 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefffff640 1 -- 172.21.15.110:0/856477184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 msgr2=0x7fdf0019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefffff640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefffff640 1 -- 172.21.15.110:0/856477184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4009560 con 0x7fdf000ffb60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefffff640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 0x7fdf0019bc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e940 tx=0x7fdef000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdeff7fe640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef000cc90 con 0x7fdf000ffb60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf001a1340 con 0x7fdf000ffb60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdef00044f0 con 0x7fdf000ffb60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.622+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef0010430 con 0x7fdf000ffb60 2024-09-21T04:21:45.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.623+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001a1910 con 0x7fdf000ffb60 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.624+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fdef0004030 con 0x7fdf000ffb60 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.624+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf00195d00 con 0x7fdf000ffb60 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.624+0000 7fdefd7fa640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 0x7fdee807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.625+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fdef0014070 con 0x7fdf000ffb60 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.625+0000 7fdeff7fe640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 0x7fdee807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.625+0000 7fdeff7fe640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 0x7fdee807d740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdef4000c00 tx=0x7fdef40314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.631+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fdef0061730 con 0x7fdf000ffb60 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all osd 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-21T04:21:45.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-21T04:21:45.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: Upgrade: Setting require_osd_release to 19 squid 2024-09-21T04:21:45.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd require-osd-release", "release": "squid"} : dispatch 2024-09-21T04:21:45.693 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-mon[78701]: pgmap v172: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:45.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.891+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf0002c480 con 0x7fdf000ffb60 2024-09-21T04:21:45.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.892+0000 7fdefd7fa640 1 -- 172.21.15.110:0/856477184 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+728 (secure 0 0 0) 0x7fdef0064850 con 0x7fdf000ffb60 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 5 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:21:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 4, 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 9 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:21:45.897 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 msgr2=0x7fdee807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 0x7fdee807d740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdef4000c00 tx=0x7fdef40314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 msgr2=0x7fdf0019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 0x7fdf0019bc30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e940 tx=0x7fdef000ee00 comp rx=0 tx=0).stop 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 shutdown_connections 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdee807b280 0x7fdee807d740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf00100d60 0x7fdf0019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 --2- 172.21.15.110:0/856477184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf000ffb60 0x7fdf0019bc30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 >> 172.21.15.110:0/856477184 conn(0x7fdf000fb330 msgr2=0x7fdf00104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 shutdown_connections 2024-09-21T04:21:45.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.894+0000 7fdf06448640 1 -- 172.21.15.110:0/856477184 wait complete. 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/3019422426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 msgr2=0x7f6e34102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/3019422426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34102a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009c50 tx=0x7f6e2402a9a0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/3019422426 shutdown_connections 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/3019422426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/3019422426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34102a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/3019422426 >> 172.21.15.110:0/3019422426 conn(0x7f6e340fa230 msgr2=0x7f6e340fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/3019422426 shutdown_connections 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.975+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/3019422426 wait complete. 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 Processor -- start 2024-09-21T04:21:45.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 -- start start 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34196130 con 0x7f6e34100610 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e3a0c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e341962a0 con 0x7f6e34102f40 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e337fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e337fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58350/0 (socket says 172.21.15.110:58350) 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e337fe640 1 -- 172.21.15.110:0/1517323984 learned_addr learned my addr 172.21.15.110:0/1517323984 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.976+0000 7f6e32ffd640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e337fe640 1 -- 172.21.15.110:0/1517323984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 msgr2=0x7f6e34195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e337fe640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e337fe640 1 -- 172.21.15.110:0/1517323984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009900 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e32ffd640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e337fe640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006010 tx=0x7f6e24030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e24003b40 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e24030cc0 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3419ace0 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e24044700 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.977+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3419b180 con 0x7f6e34100610 2024-09-21T04:21:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.978+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e341087a0 con 0x7f6e34100610 2024-09-21T04:21:45.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.979+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f6e2400a040 con 0x7f6e34100610 2024-09-21T04:21:45.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.979+0000 7f6e30ff9640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 0x7f6e1c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:45.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.979+0000 7f6e32ffd640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 0x7f6e1c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:45.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.979+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f6e240bf8c0 con 0x7f6e34100610 2024-09-21T04:21:45.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.980+0000 7f6e32ffd640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 0x7f6e1c07d730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e28005d00 tx=0x7f6e28005c70 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:45.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:45.983+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f6e24082810 con 0x7f6e34100610 2024-09-21T04:21:46.190 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:45 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:21:45.748+0000 7f003f85b640 -1 osd.5 85 log_to_monitors true 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.206+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e3419b370 con 0x7f6e34100610 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.206+0000 7f6e30ff9640 1 -- 172.21.15.110:0/1517323984 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f6e24085930 con 0x7f6e34100610 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:e15 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:21:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 15 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:12:57.204430+0000 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:21:46.210 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 46 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14562} 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:21:46.211 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{0:14562} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.110:6828/1332815334,v1:172.21.15.110:6829/1332815334] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:14574} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2135055413,v1:172.21.15.110:6827/2135055413] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:24295} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:46.212 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 msgr2=0x7f6e1c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 0x7f6e1c07d730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e28005d00 tx=0x7f6e28005c70 comp rx=0 tx=0).stop 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 msgr2=0x7f6e34195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006010 tx=0x7f6e24030890 comp rx=0 tx=0).stop 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 shutdown_connections 2024-09-21T04:21:46.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6e1c07b270 0x7f6e1c07d730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e34102f40 0x7f6e34195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 --2- 172.21.15.110:0/1517323984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e34100610 0x7f6e34195620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.208+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 >> 172.21.15.110:0/1517323984 conn(0x7f6e340fa230 msgr2=0x7f6e340fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:46.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.209+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 shutdown_connections 2024-09-21T04:21:46.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.209+0000 7f6e3a0c8640 1 -- 172.21.15.110:0/1517323984 wait complete. 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 -- 172.21.15.110:0/2293699583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 msgr2=0x7f3654102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 --2- 172.21.15.110:0/2293699583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f3654102a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f36480098b0 tx=0x7f364802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 -- 172.21.15.110:0/2293699583 shutdown_connections 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 --2- 172.21.15.110:0/2293699583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 0x7f3654105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 --2- 172.21.15.110:0/2293699583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f3654102a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 -- 172.21.15.110:0/2293699583 >> 172.21.15.110:0/2293699583 conn(0x7f36540fa230 msgr2=0x7f36540fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:46.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 -- 172.21.15.110:0/2293699583 shutdown_connections 2024-09-21T04:21:46.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.292+0000 7f365adee640 1 -- 172.21.15.110:0/2293699583 wait complete. 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 Processor -- start 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 -- start start 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 0x7f36540fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540feb20 con 0x7f3654100630 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f365adee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540fec90 con 0x7f3654102f60 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58366/0 (socket says 172.21.15.110:58366) 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 -- 172.21.15.110:0/208839627 learned_addr learned my addr 172.21.15.110:0/208839627 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3653fff640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 0x7f36540fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 -- 172.21.15.110:0/208839627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 msgr2=0x7f36540fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 0x7f36540fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 -- 172.21.15.110:0/208839627 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648009560 con 0x7f3654100630 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3658b8a640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f364802a9f0 tx=0x7f36480055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.293+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3648004aa0 con 0x7f3654100630 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.294+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3648005d50 con 0x7f3654100630 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.294+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36540fef10 con 0x7f3654100630 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.294+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36480039b0 con 0x7f3654100630 2024-09-21T04:21:46.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.294+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365419d6c0 con 0x7f3654100630 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.295+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36541087e0 con 0x7f3654100630 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.295+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f3648031c00 con 0x7f3654100630 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.296+0000 7f3651ffb640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 0x7f364407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.296+0000 7f3653fff640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 0x7f364407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.296+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f36480bf490 con 0x7f3654100630 2024-09-21T04:21:46.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.296+0000 7f3653fff640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 0x7f364407d730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36540ffd10 tx=0x7f3640005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:46.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.300+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f3648082460 con 0x7f3654100630 2024-09-21T04:21:46.508 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: osdmap e88: 6 total, 5 up, 6 in 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.trhyxr"]} : dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: Upgrade: It appears safe to stop mds.cephfs.smithi110.trhyxr 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/856477184' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:46.509 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:46 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1517323984' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.505+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36541038e0 con 0x7f364407b270 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.506+0000 7f3651ffb640 1 -- 172.21.15.110:0/208839627 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f36541038e0 con 0x7f364407b270 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:21:46.510 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "12/23 daemons upgraded", 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:21:46.511 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.508+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 msgr2=0x7f364407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.508+0000 7f365adee640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 0x7f364407d730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36540ffd10 tx=0x7f3640005f90 comp rx=0 tx=0).stop 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.508+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 msgr2=0x7f36540fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.508+0000 7f365adee640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f364802a9f0 tx=0x7f36480055a0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 shutdown_connections 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f364407b270 0x7f364407d730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3654102f60 0x7f36540fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 --2- 172.21.15.110:0/208839627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3654100630 0x7f36540fff90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 >> 172.21.15.110:0/208839627 conn(0x7f36540fa230 msgr2=0x7f36540fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 shutdown_connections 2024-09-21T04:21:46.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.509+0000 7f365adee640 1 -- 172.21.15.110:0/208839627 wait complete. 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 -- 172.21.15.110:0/151014305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980fea80 msgr2=0x7fb0980fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/151014305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980fea80 0x7fb0980fee80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb088009880 tx=0x7fb08802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 -- 172.21.15.110:0/151014305 shutdown_connections 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/151014305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980ffc80 0x7fb098100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/151014305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980fea80 0x7fb0980fee80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 -- 172.21.15.110:0/151014305 >> 172.21.15.110:0/151014305 conn(0x7fb0980fa230 msgr2=0x7fb0980fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 -- 172.21.15.110:0/151014305 shutdown_connections 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.590+0000 7fb09e9ae640 1 -- 172.21.15.110:0/151014305 wait complete. 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 Processor -- start 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 -- start start 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 0x7fb098194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098195040 con 0x7fb0980ffc80 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb09e9ae640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981951b0 con 0x7fb0980fea80 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb0977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb0977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58390/0 (socket says 172.21.15.110:58390) 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb0977fe640 1 -- 172.21.15.110:0/2210900554 learned_addr learned my addr 172.21.15.110:0/2210900554 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.591+0000 7fb097fff640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 0x7fb098194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0977fe640 1 -- 172.21.15.110:0/2210900554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 msgr2=0x7fb098194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0977fe640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 0x7fb098194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0977fe640 1 -- 172.21.15.110:0/2210900554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088009560 con 0x7fb0980ffc80 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb097fff640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 0x7fb098194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0977fe640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d720 tx=0x7fb08c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08c0051f0 con 0x7fb0980ffc80 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb08c009ce0 con 0x7fb0980ffc80 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb098199c00 con 0x7fb0980ffc80 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08c010460 con 0x7fb0980ffc80 2024-09-21T04:21:46.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.592+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09819a1d0 con 0x7fb0980ffc80 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.593+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09818e620 con 0x7fb0980ffc80 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.594+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb08c005350 con 0x7fb0980ffc80 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.594+0000 7fb0957fa640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 0x7fb08007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.594+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fb08c09dea0 con 0x7fb0980ffc80 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.594+0000 7fb097fff640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 0x7fb08007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:21:46.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.595+0000 7fb097fff640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 0x7fb08007d730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb088009230 tx=0x7fb0880314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:21:46.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.599+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb08c060e70 con 0x7fb0980ffc80 2024-09-21T04:21:46.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:21:46.434+0000 7f0037605640 -1 osd.5 85 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:21:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-21T04:21:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-21T04:21:46.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: osdmap e88: 6 total, 5 up, 6 in 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.trhyxr"]} : dispatch 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: Upgrade: It appears safe to stop mds.cephfs.smithi110.trhyxr 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/856477184' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:21:46.692 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:46 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1517323984' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:21:46.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.851+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb0981078b0 con 0x7fb0980ffc80 2024-09-21T04:21:46.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.851+0000 7fb0957fa640 1 -- 172.21.15.110:0/2210900554 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+942 (secure 0 0 0) 0x7fb08c0173c0 con 0x7fb0980ffc80 2024-09-21T04:21:46.855 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 30/222 objects degraded (13.514%), 12 pgs degraded 2024-09-21T04:21:46.855 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-21T04:21:46.855 INFO:teuthology.orchestra.run.smithi110.stdout: osd.5 (root=default,host=smithi148) is down 2024-09-21T04:21:46.855 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 30/222 objects degraded (13.514%), 12 pgs degraded 2024-09-21T04:21:46.855 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-09-21T04:21:46.856 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-09-21T04:21:46.857 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-09-21T04:21:46.857 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-09-21T04:21:46.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 msgr2=0x7fb08007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 0x7fb08007d730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb088009230 tx=0x7fb0880314e0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 msgr2=0x7fb098194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d720 tx=0x7fb08c00dbe0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 shutdown_connections 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb08007b270 0x7fb08007d730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0980ffc80 0x7fb098194a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 --2- 172.21.15.110:0/2210900554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0980fea80 0x7fb098194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:21:46.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 >> 172.21.15.110:0/2210900554 conn(0x7fb0980fa230 msgr2=0x7fb098103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:21:46.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 shutdown_connections 2024-09-21T04:21:46.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:21:46.854+0000 7fb09e9ae640 1 -- 172.21.15.110:0/2210900554 wait complete. 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: osdmap e89: 6 total, 5 up, 6 in 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2210900554' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: pgmap v175: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:21:47.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:47.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: osdmap e89: 6 total, 5 up, 6 in 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]} : dispatch 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2210900554' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: pgmap v175: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.trhyxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:21:47.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:21:48.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: Upgrade: Updating mds.cephfs.smithi110.trhyxr 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: Deploying daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482] boot 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: osdmap e90: 6 total, 6 up, 6 in 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:21:48.688 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:48 smithi110 ceph-mon[92455]: osdmap e91: 6 total, 6 up, 6 in 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: Upgrade: Updating mds.cephfs.smithi110.trhyxr 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: Deploying daemon mds.cephfs.smithi110.trhyxr on smithi110 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: osd.5 [v2:172.21.15.148:6816/3334621482,v1:172.21.15.148:6817/3334621482] boot 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: osdmap e90: 6 total, 6 up, 6 in 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:21:48.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:48 smithi148 ceph-mon[78701]: osdmap e91: 6 total, 6 up, 6 in 2024-09-21T04:21:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:49 smithi110 ceph-mon[92455]: pgmap v178: 65 pgs: 1 peering, 16 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:49 smithi148 ceph-mon[78701]: pgmap v178: 65 pgs: 1 peering, 16 active+undersized, 12 active+undersized+degraded, 36 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 30/222 objects degraded (13.514%) 2024-09-21T04:21:50.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:50 smithi110 ceph-mon[92455]: osdmap e92: 6 total, 6 up, 6 in 2024-09-21T04:21:50.631 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:50 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 2 up:standby 2024-09-21T04:21:50.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:50 smithi148 ceph-mon[78701]: osdmap e92: 6 total, 6 up, 6 in 2024-09-21T04:21:50.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:50 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 2 up:standby 2024-09-21T04:21:51.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:51 smithi110 ceph-mon[92455]: pgmap v180: 65 pgs: 1 peering, 12 active+undersized, 7 active+undersized+degraded, 45 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 15/222 objects degraded (6.757%) 2024-09-21T04:21:51.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:51 smithi148 ceph-mon[78701]: pgmap v180: 65 pgs: 1 peering, 12 active+undersized, 7 active+undersized+degraded, 45 active+clean; 195 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 15/222 objects degraded (6.757%) 2024-09-21T04:21:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:52 smithi148 ceph-mon[78701]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:52.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:52 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:52.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:52 smithi110 ceph-mon[92455]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:21:52.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:52.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:52.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:52 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:21:53.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:boot 2024-09-21T04:21:53.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:21:53.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:21:53.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: pgmap v181: 65 pgs: 1 peering, 64 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:53.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:53.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:53.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:boot 2024-09-21T04:21:53.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi110.faxwju=up:active} 3 up:standby 2024-09-21T04:21:53.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:21:53.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: pgmap v181: 65 pgs: 1 peering, 64 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:53.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:53.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:21:54.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:54 smithi148 ceph-mon[78701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/222 objects degraded (6.757%), 7 pgs degraded) 2024-09-21T04:21:54.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:54 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:21:54.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:54 smithi110 ceph-mon[92455]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/222 objects degraded (6.757%), 7 pgs degraded) 2024-09-21T04:21:54.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:54 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:21:55.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:55 smithi110 ceph-mon[92455]: pgmap v182: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:55.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:55 smithi148 ceph-mon[78701]: pgmap v182: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:56 smithi110 ceph-mon[92455]: pgmap v183: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:57.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:56 smithi148 ceph-mon[78701]: pgmap v183: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:58.254 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.254 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:58.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:21:59.296 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:21:59 smithi110 ceph-mon[92455]: pgmap v184: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:21:59.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:21:59 smithi148 ceph-mon[78701]: pgmap v184: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:22:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:02.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: pgmap v185: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.faxwju"]} : dispatch 2024-09-21T04:22:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: pgmap v185: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.faxwju"]} : dispatch 2024-09-21T04:22:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:02 smithi110 ceph-mon[92455]: Upgrade: It appears safe to stop mds.cephfs.smithi110.faxwju 2024-09-21T04:22:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:03.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:02 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:02 smithi148 ceph-mon[78701]: Upgrade: It appears safe to stop mds.cephfs.smithi110.faxwju 2024-09-21T04:22:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.faxwju", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:02 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:04.039 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:03 smithi110 ceph-mon[92455]: Upgrade: Updating mds.cephfs.smithi110.faxwju 2024-09-21T04:22:04.096 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:03 smithi110 ceph-mon[92455]: Deploying daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:22:04.096 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:03 smithi110 ceph-mon[92455]: pgmap v186: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:22:04.096 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:03 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:03 smithi148 ceph-mon[78701]: Upgrade: Updating mds.cephfs.smithi110.faxwju 2024-09-21T04:22:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:03 smithi148 ceph-mon[78701]: Deploying daemon mds.cephfs.smithi110.faxwju on smithi110 2024-09-21T04:22:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:03 smithi148 ceph-mon[78701]: pgmap v186: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:22:04.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:03 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:04.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[92451]: 2024-09-21T04:22:04.026+0000 7f34a828c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: osdmap e93: 6 total, 6 up, 6 in 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: Standby daemon mds.cephfs.smithi148.tpueww assigned to filesystem cephfs as rank 0 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:05.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:05.168 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:04 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:replay} 2 up:standby 2024-09-21T04:22:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: osdmap e93: 6 total, 6 up, 6 in 2024-09-21T04:22:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: Standby daemon mds.cephfs.smithi148.tpueww assigned to filesystem cephfs as rank 0 2024-09-21T04:22:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:05.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:04 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:replay} 2 up:standby 2024-09-21T04:22:06.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:05 smithi110 ceph-mon[92455]: pgmap v188: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-09-21T04:22:06.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:05 smithi148 ceph-mon[78701]: pgmap v188: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-09-21T04:22:07.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:06 smithi110 ceph-mon[92455]: pgmap v189: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 3 op/s 2024-09-21T04:22:07.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:06 smithi148 ceph-mon[78701]: pgmap v189: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 3 op/s 2024-09-21T04:22:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:09 smithi110 ceph-mon[92455]: pgmap v190: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 3 op/s 2024-09-21T04:22:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:09 smithi148 ceph-mon[78701]: pgmap v190: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 3 op/s 2024-09-21T04:22:11.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:10 smithi148 ceph-mon[78701]: pgmap v191: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-09-21T04:22:11.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:10 smithi110 ceph-mon[92455]: pgmap v191: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-09-21T04:22:12.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:11 smithi110 ceph-mon[92455]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0 2024-09-21T04:22:12.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:11 smithi110 ceph-mon[92455]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0 2024-09-21T04:22:12.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:11 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:reconnect 2024-09-21T04:22:12.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:11 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:reconnect} 2 up:standby 2024-09-21T04:22:12.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:11 smithi148 ceph-mon[78701]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0 2024-09-21T04:22:12.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:11 smithi148 ceph-mon[78701]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0 2024-09-21T04:22:12.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:11 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:reconnect 2024-09-21T04:22:12.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:11 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:reconnect} 2 up:standby 2024-09-21T04:22:13.313 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:13 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:rejoin 2024-09-21T04:22:13.314 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:13 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:rejoin} 2 up:standby 2024-09-21T04:22:13.314 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:13 smithi110 ceph-mon[92455]: daemon mds.cephfs.smithi148.tpueww is now active in filesystem cephfs as rank 0 2024-09-21T04:22:13.314 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:13 smithi110 ceph-mon[92455]: pgmap v192: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-09-21T04:22:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:13 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:rejoin 2024-09-21T04:22:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:13 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.tpueww=up:rejoin} 2 up:standby 2024-09-21T04:22:13.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:13 smithi148 ceph-mon[78701]: daemon mds.cephfs.smithi148.tpueww is now active in filesystem cephfs as rank 0 2024-09-21T04:22:13.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:13 smithi148 ceph-mon[78701]: pgmap v192: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-09-21T04:22:14.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:14 smithi148 ceph-mon[78701]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:14.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:14 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:22:14.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:14 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:active 2024-09-21T04:22:14.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:14 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi148.tpueww=up:active} 2 up:standby 2024-09-21T04:22:14.348 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:14 smithi110 ceph-mon[92455]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:14.349 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:14 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:22:14.349 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:14 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] up:active 2024-09-21T04:22:14.349 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:14 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi148.tpueww=up:active} 2 up:standby 2024-09-21T04:22:15.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:15.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:15.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:15.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: pgmap v193: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 94 B/s wr, 6 op/s 2024-09-21T04:22:15.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] up:boot 2024-09-21T04:22:15.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi148.tpueww=up:active} 3 up:standby 2024-09-21T04:22:15.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:22:15.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:15.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:15.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:15.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: pgmap v193: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 94 B/s wr, 6 op/s 2024-09-21T04:22:15.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] up:boot 2024-09-21T04:22:15.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi148.tpueww=up:active} 3 up:standby 2024-09-21T04:22:15.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 -- 172.21.15.110:0/753680451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 msgr2=0x7f19a4105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 --2- 172.21.15.110:0/753680451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a4105150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f19980098b0 tx=0x7f199802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 -- 172.21.15.110:0/753680451 shutdown_connections 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 --2- 172.21.15.110:0/753680451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a4105150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 --2- 172.21.15.110:0/753680451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 0x7f19a4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 -- 172.21.15.110:0/753680451 >> 172.21.15.110:0/753680451 conn(0x7f19a40fa030 msgr2=0x7f19a40fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 -- 172.21.15.110:0/753680451 shutdown_connections 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 -- 172.21.15.110:0/753680451 wait complete. 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.939+0000 7f19a3fff640 1 Processor -- start 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a3fff640 1 -- start start 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 0x7f19a40ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:16.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a3fff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41002d0 con 0x7f19a4102d60 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a3fff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a40fe950 con 0x7f19a4100430 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33778/0 (socket says 172.21.15.110:33778) 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 -- 172.21.15.110:0/546207387 learned_addr learned my addr 172.21.15.110:0/546207387 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a2ffd640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 0x7f19a40ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 -- 172.21.15.110:0/546207387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 msgr2=0x7f19a40ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 0x7f19a40ffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 -- 172.21.15.110:0/546207387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009560 con 0x7f19a4102d60 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.940+0000 7f19a27fc640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f199802a9f0 tx=0x7f1998005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.941+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1998004b20 con 0x7f19a4102d60 2024-09-21T04:22:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.941+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1998004c80 con 0x7f19a4102d60 2024-09-21T04:22:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.941+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a40febd0 con 0x7f19a4102d60 2024-09-21T04:22:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.941+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1998003bc0 con 0x7f19a4102d60 2024-09-21T04:22:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.941+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a40ff140 con 0x7f19a4102d60 2024-09-21T04:22:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.942+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a4108590 con 0x7f19a4102d60 2024-09-21T04:22:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.942+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1998031640 con 0x7f19a4102d60 2024-09-21T04:22:16.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.943+0000 7f198bfff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 0x7f198c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:16.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.943+0000 7f19a2ffd640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 0x7f198c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:16.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.943+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f19980bf790 con 0x7f19a4102d60 2024-09-21T04:22:16.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.944+0000 7f19a2ffd640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 0x7f198c07d730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1994005e10 tx=0x7f1994005da0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:16.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:16.947+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1998082680 con 0x7f19a4102d60 2024-09-21T04:22:17.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.148+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a41037a0 con 0x7f198c07b270 2024-09-21T04:22:17.151 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:16 smithi110 ceph-mon[92455]: pgmap v194: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-21T04:22:17.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.149+0000 7f198bfff640 1 -- 172.21.15.110:0/546207387 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f19a41037a0 con 0x7f198c07b270 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 msgr2=0x7f198c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 0x7f198c07d730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1994005e10 tx=0x7f1994005da0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 msgr2=0x7f19a40fe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f199802a9f0 tx=0x7f1998005610 comp rx=0 tx=0).stop 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 shutdown_connections 2024-09-21T04:22:17.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f198c07b270 0x7f198c07d730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a4102d60 0x7f19a40fe3e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 --2- 172.21.15.110:0/546207387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a4100430 0x7f19a40ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 >> 172.21.15.110:0/546207387 conn(0x7f19a40fa030 msgr2=0x7f19a40fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 shutdown_connections 2024-09-21T04:22:17.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.151+0000 7f19a3fff640 1 -- 172.21.15.110:0/546207387 wait complete. 2024-09-21T04:22:17.163 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.233+0000 7f8317771640 1 -- 172.21.15.110:0/2202019980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 msgr2=0x7f83101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.233+0000 7f8317771640 1 --2- 172.21.15.110:0/2202019980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f83101029b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83000098b0 tx=0x7f830002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- 172.21.15.110:0/2202019980 shutdown_connections 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 --2- 172.21.15.110:0/2202019980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 0x7f83101052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 --2- 172.21.15.110:0/2202019980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f83101029b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- 172.21.15.110:0/2202019980 >> 172.21.15.110:0/2202019980 conn(0x7f83100fa1c0 msgr2=0x7f83100fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- 172.21.15.110:0/2202019980 shutdown_connections 2024-09-21T04:22:17.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- 172.21.15.110:0/2202019980 wait complete. 2024-09-21T04:22:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 Processor -- start 2024-09-21T04:22:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- start start 2024-09-21T04:22:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 0x7f8310195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310196170 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f8317771640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101962e0 con 0x7f8310102ef0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f831550d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f831550d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33808/0 (socket says 172.21.15.110:33808) 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.234+0000 7f831550d640 1 -- 172.21.15.110:0/4021589211 learned_addr learned my addr 172.21.15.110:0/4021589211 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f8314d0c640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 0x7f8310195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f831550d640 1 -- 172.21.15.110:0/4021589211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 msgr2=0x7f8310195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f831550d640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 0x7f8310195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f831550d640 1 -- 172.21.15.110:0/4021589211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009560 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f831550d640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83000054f0 tx=0x7f8300005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8300004aa0 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8300004c00 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831019ad20 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8300003b10 con 0x7f83101005c0 2024-09-21T04:22:17.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.235+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831019b1c0 con 0x7f83101005c0 2024-09-21T04:22:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.236+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8310108750 con 0x7f83101005c0 2024-09-21T04:22:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.237+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8300003d80 con 0x7f83101005c0 2024-09-21T04:22:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.237+0000 7f83067fc640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 0x7f82fc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.238+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f83000bf5e0 con 0x7f83101005c0 2024-09-21T04:22:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.238+0000 7f8314d0c640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 0x7f82fc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.238+0000 7f8314d0c640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 0x7f82fc07d740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f830c005d00 tx=0x7f830c005c50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.242+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8300082550 con 0x7f83101005c0 2024-09-21T04:22:17.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:16 smithi148 ceph-mon[78701]: pgmap v194: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-09-21T04:22:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.457+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f831019b400 con 0x7f82fc07b280 2024-09-21T04:22:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.458+0000 7f83067fc640 1 -- 172.21.15.110:0/4021589211 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f831019b400 con 0x7f82fc07b280 2024-09-21T04:22:17.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 msgr2=0x7f82fc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 0x7f82fc07d740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f830c005d00 tx=0x7f830c005c50 comp rx=0 tx=0).stop 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 msgr2=0x7f8310195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83000054f0 tx=0x7f8300005520 comp rx=0 tx=0).stop 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 shutdown_connections 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f82fc07b280 0x7f82fc07d740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310102ef0 0x7f8310195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 --2- 172.21.15.110:0/4021589211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83101005c0 0x7f8310195660 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 >> 172.21.15.110:0/4021589211 conn(0x7f83100fa1c0 msgr2=0x7f83100fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 shutdown_connections 2024-09-21T04:22:17.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.460+0000 7f8317771640 1 -- 172.21.15.110:0/4021589211 wait complete. 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- 172.21.15.110:0/886472288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 msgr2=0x7f76481011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 --2- 172.21.15.110:0/886472288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f76481011b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f763c009880 tx=0x7f763c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- 172.21.15.110:0/886472288 shutdown_connections 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 --2- 172.21.15.110:0/886472288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f76481011b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 --2- 172.21.15.110:0/886472288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f76480fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- 172.21.15.110:0/886472288 >> 172.21.15.110:0/886472288 conn(0x7f76480fb2c0 msgr2=0x7f76480fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- 172.21.15.110:0/886472288 shutdown_connections 2024-09-21T04:22:17.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- 172.21.15.110:0/886472288 wait complete. 2024-09-21T04:22:17.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 Processor -- start 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.542+0000 7f764de2f640 1 -- start start 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f764de2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f7648195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f764de2f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f764de2f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648196120 con 0x7f7648100d30 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f764de2f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648196290 con 0x7f76480ffb30 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33828/0 (socket says 172.21.15.110:33828) 2024-09-21T04:22:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 -- 172.21.15.110:0/2748774012 learned_addr learned my addr 172.21.15.110:0/2748774012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f76477fe640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f7648195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 -- 172.21.15.110:0/2748774012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 msgr2=0x7f7648195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f7648195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 -- 172.21.15.110:0/2748774012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c009560 con 0x7f7648100d30 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f76477fe640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f7648195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7646ffd640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f763c0055a0 tx=0x7f763c0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c004b20 con 0x7f7648100d30 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763c004c80 con 0x7f7648100d30 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764819acd0 con 0x7f7648100d30 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.543+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c003bc0 con 0x7f7648100d30 2024-09-21T04:22:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.544+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764819b240 con 0x7f7648100d30 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.544+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764818f6b0 con 0x7f7648100d30 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.545+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f763c031640 con 0x7f7648100d30 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.546+0000 7f7644ff9640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 0x7f763007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.546+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f763c0bf750 con 0x7f7648100d30 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.546+0000 7f76477fe640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 0x7f763007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.546+0000 7f76477fe640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 0x7f763007d740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7648100b90 tx=0x7f7638005e70 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.550+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f763c082640 con 0x7f7648100d30 2024-09-21T04:22:17.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.751+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f764802c4f0 con 0x7f763007b280 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.760+0000 7f7644ff9640 1 -- 172.21.15.110:0/2748774012 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f764802c4f0 con 0x7f763007b280 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (13m) 20s ago 15m 25.8M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (6m) 20s ago 15m 8799k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (6m) 37s ago 14m 8376k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:22:17.763 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (4m) 20s ago 15m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (4m) 37s ago 14m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (13m) 20s ago 15m 86.4M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 starting - - - - 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (26s) 20s ago 9m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (9m) 37s ago 9m 18.3M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (9m) 37s ago 9m 20.2M - 18.2.4-871-g6a1bdf85 0adab276c97e a3c0e9bec880 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (6m) 20s ago 16m 520M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (5m) 37s ago 14m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (5m) 20s ago 16m 50.7M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (4m) 37s ago 14m 38.2M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (15m) 20s ago 15m 16.9M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (14m) 37s ago 14m 15.2M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:22:17.764 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (3m) 20s ago 13m 115M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (3m) 20s ago 12m 145M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 20s ago 12m 182M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (107s) 37s ago 11m 137M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (74s) 37s ago 11m 99.7M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (40s) 37s ago 11m 11.7M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:22:17.765 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 20s ago 15m 61.0M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 msgr2=0x7f763007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 0x7f763007d740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7648100b90 tx=0x7f7638005e70 comp rx=0 tx=0).stop 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 msgr2=0x7f7648195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f763c0055a0 tx=0x7f763c0055d0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 shutdown_connections 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f763007b280 0x7f763007d740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7648100d30 0x7f7648195b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 --2- 172.21.15.110:0/2748774012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ffb30 0x7f7648195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 >> 172.21.15.110:0/2748774012 conn(0x7f76480fb2c0 msgr2=0x7f7648104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 shutdown_connections 2024-09-21T04:22:17.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.763+0000 7f764de2f640 1 -- 172.21.15.110:0/2748774012 wait complete. 2024-09-21T04:22:17.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 -- 172.21.15.110:0/2282565345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 msgr2=0x7f849c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 --2- 172.21.15.110:0/2282565345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c105390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8490009880 tx=0x7f849002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 -- 172.21.15.110:0/2282565345 shutdown_connections 2024-09-21T04:22:17.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 --2- 172.21.15.110:0/2282565345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c105390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 --2- 172.21.15.110:0/2282565345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 0x7f849c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 -- 172.21.15.110:0/2282565345 >> 172.21.15.110:0/2282565345 conn(0x7f849c0fa230 msgr2=0x7f849c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:17.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 -- 172.21.15.110:0/2282565345 shutdown_connections 2024-09-21T04:22:17.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.843+0000 7f84a0a55640 1 -- 172.21.15.110:0/2282565345 wait complete. 2024-09-21T04:22:17.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 Processor -- start 2024-09-21T04:22:17.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 -- start start 2024-09-21T04:22:17.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 0x7f849c1955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1960c0 con 0x7f849c102fa0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f84a0a55640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c196230 con 0x7f849c100670 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f8499d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f8499d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33848/0 (socket says 172.21.15.110:33848) 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f8499d9b640 1 -- 172.21.15.110:0/4069695403 learned_addr learned my addr 172.21.15.110:0/4069695403 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f849a59c640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 0x7f849c1955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f8499d9b640 1 -- 172.21.15.110:0/4069695403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 msgr2=0x7f849c1955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.844+0000 7f8499d9b640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 0x7f849c1955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f8499d9b640 1 -- 172.21.15.110:0/4069695403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490009560 con 0x7f849c102fa0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f8499d9b640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84900054f0 tx=0x7f8490005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8490004aa0 con 0x7f849c102fa0 2024-09-21T04:22:17.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8490004c00 con 0x7f849c102fa0 2024-09-21T04:22:17.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c19acd0 con 0x7f849c102fa0 2024-09-21T04:22:17.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8490003ac0 con 0x7f849c102fa0 2024-09-21T04:22:17.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.845+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c19b2a0 con 0x7f849c102fa0 2024-09-21T04:22:17.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.846+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c108720 con 0x7f849c102fa0 2024-09-21T04:22:17.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.847+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8490003d80 con 0x7f849c102fa0 2024-09-21T04:22:17.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.847+0000 7f848b7fe640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 0x7f848407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:17.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.847+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f84900bf600 con 0x7f849c102fa0 2024-09-21T04:22:17.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.847+0000 7f849a59c640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 0x7f848407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:17.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.848+0000 7f849a59c640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 0x7f848407d730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f848c005fd0 tx=0x7f848c00b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:17.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:17.851+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f84900825a0 con 0x7f849c102fa0 2024-09-21T04:22:18.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.123+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f849c19b490 con 0x7f849c102fa0 2024-09-21T04:22:18.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.123+0000 7f848b7fe640 1 -- 172.21.15.110:0/4069695403 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f849c19b490 con 0x7f849c102fa0 2024-09-21T04:22:18.126 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:22:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2, 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:22:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 18.2.4-871-g6a1bdf85 (6a1bdf854bb5803efd58957907442aa01f41a8fc) reef (stable)": 2, 2024-09-21T04:22:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 12 2024-09-21T04:22:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:22:18.128 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:22:18.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.125+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 msgr2=0x7f848407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.125+0000 7f84a0a55640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 0x7f848407d730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f848c005fd0 tx=0x7f848c00b390 comp rx=0 tx=0).stop 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.125+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 msgr2=0x7f849c195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.125+0000 7f84a0a55640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84900054f0 tx=0x7f8490005520 comp rx=0 tx=0).stop 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.125+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 shutdown_connections 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f848407b270 0x7f848407d730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f849c102fa0 0x7f849c195af0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 --2- 172.21.15.110:0/4069695403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c100670 0x7f849c1955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 >> 172.21.15.110:0/4069695403 conn(0x7f849c0fa230 msgr2=0x7f849c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 shutdown_connections 2024-09-21T04:22:18.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.126+0000 7f84a0a55640 1 -- 172.21.15.110:0/4069695403 wait complete. 2024-09-21T04:22:18.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:17 smithi148 ceph-mon[78701]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:17 smithi148 ceph-mon[78701]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:17 smithi148 ceph-mon[78701]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 -- 172.21.15.110:0/1481505783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4100630 msgr2=0x7f5bc4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 --2- 172.21.15.110:0/1481505783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4100630 0x7f5bc4102a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098b0 tx=0x7f5bac02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 -- 172.21.15.110:0/1481505783 shutdown_connections 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 --2- 172.21.15.110:0/1481505783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4102f60 0x7f5bc4105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 --2- 172.21.15.110:0/1481505783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4100630 0x7f5bc4102a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 -- 172.21.15.110:0/1481505783 >> 172.21.15.110:0/1481505783 conn(0x7f5bc40fa230 msgr2=0x7f5bc40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 -- 172.21.15.110:0/1481505783 shutdown_connections 2024-09-21T04:22:18.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 -- 172.21.15.110:0/1481505783 wait complete. 2024-09-21T04:22:18.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.216+0000 7f5bca174640 1 Processor -- start 2024-09-21T04:22:18.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bca174640 1 -- start start 2024-09-21T04:22:18.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bca174640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4100630 0x7f5bc4195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bca174640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bca174640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4196290 con 0x7f5bc4102f60 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bca174640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4196400 con 0x7f5bc4100630 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33860/0 (socket says 172.21.15.110:33860) 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 -- 172.21.15.110:0/350985056 learned_addr learned my addr 172.21.15.110:0/350985056 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc37fe640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4100630 0x7f5bc4195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 -- 172.21.15.110:0/350985056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4100630 msgr2=0x7f5bc4195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4100630 0x7f5bc4195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 -- 172.21.15.110:0/350985056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009560 con 0x7f5bc4102f60 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.217+0000 7f5bc2ffd640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400b400 tx=0x7f5bb400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.218+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb4011800 con 0x7f5bc4102f60 2024-09-21T04:22:18.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.218+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bb4011e20 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.218+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc419ae50 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.218+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb400f2e0 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.218+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc419b3f0 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.219+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc4100a90 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.219+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f5bb4010380 con 0x7f5bc4102f60 2024-09-21T04:22:18.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.220+0000 7f5bc0ff9640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 0x7f5bb007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.220+0000 7f5bc37fe640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 0x7f5bb007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.220+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f5bb409df20 con 0x7f5bc4102f60 2024-09-21T04:22:18.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.221+0000 7f5bc37fe640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 0x7f5bb007d730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bac006010 tx=0x7f5bac0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.224+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f5bb40d5710 con 0x7f5bc4102f60 2024-09-21T04:22:18.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:17 smithi110 ceph-mon[92455]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:17 smithi110 ceph-mon[92455]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:17 smithi110 ceph-mon[92455]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:18.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.447+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5bc4108830 con 0x7f5bc4102f60 2024-09-21T04:22:18.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.448+0000 7f5bc0ff9640 1 -- 172.21.15.110:0/350985056 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1933 (secure 0 0 0) 0x7f5bb4064590 con 0x7f5bc4102f60 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:e23 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:15:001098+0000 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:22:18.451 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 22 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:12.995969+0000 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 93 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:22:18.452 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=24295} 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 24295 members: 24295 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{0:24295} state up:active seq 145 join_fscid=1 addr [v2:172.21.15.148:6824/509765299,v1:172.21.15.148:6825/509765299] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:22:18.453 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:24303} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] compat {c=[1],r=[1],i=[7ff]}] 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:22:18.454 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 23 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 msgr2=0x7f5bb007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 0x7f5bb007d730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bac006010 tx=0x7f5bac0314e0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 msgr2=0x7f5bc4195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400b400 tx=0x7f5bb400b8c0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 shutdown_connections 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5bb007b270 0x7f5bb007d730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bc4102f60 0x7f5bc4195cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 --2- 172.21.15.110:0/350985056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bc4100630 0x7f5bc4195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 >> 172.21.15.110:0/350985056 conn(0x7f5bc40fa230 msgr2=0x7f5bc40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 shutdown_connections 2024-09-21T04:22:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.450+0000 7f5bca174640 1 -- 172.21.15.110:0/350985056 wait complete. 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.532+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/3279233526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f580ffb00 msgr2=0x7f8f580fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.532+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/3279233526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f580ffb00 0x7f8f580fff00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009880 tx=0x7f8f4c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/3279233526 shutdown_connections 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/3279233526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f58100d00 0x7f8f58101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/3279233526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f580ffb00 0x7f8f580fff00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/3279233526 >> 172.21.15.110:0/3279233526 conn(0x7f8f580fb290 msgr2=0x7f8f580fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/3279233526 shutdown_connections 2024-09-21T04:22:18.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/3279233526 wait complete. 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 Processor -- start 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- start start 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 0x7f8f581955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f581960b0 con 0x7f8f58100d00 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.533+0000 7f8f5f0f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58196220 con 0x7f8f580ffb00 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33888/0 (socket says 172.21.15.110:33888) 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 -- 172.21.15.110:0/1881839320 learned_addr learned my addr 172.21.15.110:0/1881839320 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f5ce90640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 0x7f8f581955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 -- 172.21.15.110:0/1881839320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 msgr2=0x7f8f581955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 0x7f8f581955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 -- 172.21.15.110:0/1881839320 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c009560 con 0x7f8f58100d00 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f5ce90640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 0x7f8f581955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f57fff640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c920 tx=0x7f8f4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f44007b80 con 0x7f8f58100d00 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f440054e0 con 0x7f8f58100d00 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4400f450 con 0x7f8f58100d00 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.534+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5819acc0 con 0x7f8f58100d00 2024-09-21T04:22:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.535+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5819b240 con 0x7f8f58100d00 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.536+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58108710 con 0x7f8f58100d00 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.536+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8f44004f60 con 0x7f8f58100d00 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.537+0000 7f8f55ffb640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 0x7f8f4807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.537+0000 7f8f5ce90640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 0x7f8f4807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.537+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f8f4409e2a0 con 0x7f8f58100d00 2024-09-21T04:22:18.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.538+0000 7f8f5ce90640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 0x7f8f4807d740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c006010 tx=0x7f8f4c004040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.541+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8f44061190 con 0x7f8f58100d00 2024-09-21T04:22:18.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.743+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f5819b4f0 con 0x7f8f4807b280 2024-09-21T04:22:18.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.744+0000 7f8f55ffb640 1 -- 172.21.15.110:0/1881839320 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8f5819b4f0 con 0x7f8f4807b280 2024-09-21T04:22:18.747 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:22:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:22:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "13/23 daemons upgraded", 2024-09-21T04:22:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:22:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:22:18.749 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 msgr2=0x7f8f4807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 0x7f8f4807d740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c006010 tx=0x7f8f4c004040 comp rx=0 tx=0).stop 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 msgr2=0x7f8f58195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400c920 tx=0x7f8f4400cde0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 shutdown_connections 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f4807b280 0x7f8f4807d740 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f58100d00 0x7f8f58195ae0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 --2- 172.21.15.110:0/1881839320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f580ffb00 0x7f8f581955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 >> 172.21.15.110:0/1881839320 conn(0x7f8f580fb290 msgr2=0x7f8f58103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 shutdown_connections 2024-09-21T04:22:18.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.746+0000 7f8f5f0f4640 1 -- 172.21.15.110:0/1881839320 wait complete. 2024-09-21T04:22:18.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 -- 172.21.15.110:0/2334404312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 msgr2=0x7fa5a40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2334404312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a40feea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa594009880 tx=0x7fa59402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 -- 172.21.15.110:0/2334404312 shutdown_connections 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2334404312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a40ffca0 0x7fa5a4100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2334404312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a40feea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 -- 172.21.15.110:0/2334404312 >> 172.21.15.110:0/2334404312 conn(0x7fa5a40fa230 msgr2=0x7fa5a40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 -- 172.21.15.110:0/2334404312 shutdown_connections 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.826+0000 7fa5aa759640 1 -- 172.21.15.110:0/2334404312 wait complete. 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 Processor -- start 2024-09-21T04:22:18.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 -- start start 2024-09-21T04:22:18.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a40ffca0 0x7fa5a4101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41015c0 con 0x7fa5a40feaa0 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5aa759640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4101730 con 0x7fa5a40ffca0 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33914/0 (socket says 172.21.15.110:33914) 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 -- 172.21.15.110:0/2175395815 learned_addr learned my addr 172.21.15.110:0/2175395815 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 -- 172.21.15.110:0/2175395815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a40ffca0 msgr2=0x7fa5a4101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a40ffca0 0x7fa5a4101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.827+0000 7fa5a3fff640 1 -- 172.21.15.110:0/2175395815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594009560 con 0x7fa5a40feaa0 2024-09-21T04:22:18.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5a3fff640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa59402a9c0 tx=0x7fa594005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa594004b20 con 0x7fa5a40feaa0 2024-09-21T04:22:18.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a41019b0 con 0x7fa5a40feaa0 2024-09-21T04:22:18.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa594004c80 con 0x7fa5a40feaa0 2024-09-21T04:22:18.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa594003b80 con 0x7fa5a40feaa0 2024-09-21T04:22:18.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.828+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a419d7a0 con 0x7fa5a40feaa0 2024-09-21T04:22:18.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.829+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a40fef20 con 0x7fa5a40feaa0 2024-09-21T04:22:18.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.830+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fa594031640 con 0x7fa5a40feaa0 2024-09-21T04:22:18.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.830+0000 7fa5a17fa640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 0x7fa58c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:18.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.830+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa5940bf820 con 0x7fa5a40feaa0 2024-09-21T04:22:18.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.830+0000 7fa5a37fe640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 0x7fa58c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:18.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.831+0000 7fa5a37fe640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 0x7fa58c07d730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5a41027b0 tx=0x7fa598006c90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:18.835+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fa594082710 con 0x7fa5a40feaa0 2024-09-21T04:22:19.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.135+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa5a4107710 con 0x7fa5a40feaa0 2024-09-21T04:22:19.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.136+0000 7fa5a17fa640 1 -- 172.21.15.110:0/2175395815 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa594085830 con 0x7fa5a40feaa0 2024-09-21T04:22:19.139 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:22:19.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.138+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 msgr2=0x7fa58c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:19.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.138+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 0x7fa58c07d730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5a41027b0 tx=0x7fa598006c90 comp rx=0 tx=0).stop 2024-09-21T04:22:19.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.138+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 msgr2=0x7fa5a4102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.138+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa59402a9c0 tx=0x7fa594005610 comp rx=0 tx=0).stop 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 shutdown_connections 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa58c07b270 0x7fa58c07d730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a40ffca0 0x7fa5a4101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 --2- 172.21.15.110:0/2175395815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5a40feaa0 0x7fa5a4102a30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 >> 172.21.15.110:0/2175395815 conn(0x7fa5a40fa230 msgr2=0x7fa5a40fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 shutdown_connections 2024-09-21T04:22:19.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:19.139+0000 7fa5aa759640 1 -- 172.21.15.110:0/2175395815 wait complete. 2024-09-21T04:22:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:18 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4069695403' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:18 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/350985056' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:22:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:18 smithi110 ceph-mon[92455]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:18 smithi110 ceph-mon[92455]: pgmap v195: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T04:22:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:18 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4069695403' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:18 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/350985056' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:22:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:18 smithi148 ceph-mon[78701]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:18 smithi148 ceph-mon[78701]: pgmap v195: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T04:22:20.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:20 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2175395815' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:22:20.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:20 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:20 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2175395815' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:22:20.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:20.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:20 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:21 smithi110 ceph-mon[92455]: pgmap v196: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T04:22:21.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:21 smithi148 ceph-mon[78701]: pgmap v196: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-21T04:22:23.110 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:22 smithi110 ceph-mon[92455]: pgmap v197: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-21T04:22:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:22 smithi148 ceph-mon[78701]: pgmap v197: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-21T04:22:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.tpueww"]} : dispatch 2024-09-21T04:22:24.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:24 smithi110 ceph-mon[92455]: Upgrade: It appears safe to stop mds.cephfs.smithi148.tpueww 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:24.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.tpueww"]} : dispatch 2024-09-21T04:22:24.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:24 smithi148 ceph-mon[78701]: Upgrade: It appears safe to stop mds.cephfs.smithi148.tpueww 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: Upgrade: Updating mds.cephfs.smithi148.tpueww 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: Deploying daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:22:25.894 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:25 smithi148 ceph-mon[78701]: pgmap v198: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 364 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: Upgrade: Updating mds.cephfs.smithi148.tpueww 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.tpueww", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: Deploying daemon mds.cephfs.smithi148.tpueww on smithi148 2024-09-21T04:22:25.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:25 smithi110 ceph-mon[92455]: pgmap v198: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 364 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[92451]: 2024-09-21T04:22:26.430+0000 7f34a828c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: osdmap e94: 6 total, 6 up, 6 in 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: Standby daemon mds.cephfs.smithi148.fpatxi assigned to filesystem cephfs as rank 0 2024-09-21T04:22:26.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:26.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:26.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:26 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:replay} 2 up:standby 2024-09-21T04:22:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: osdmap e94: 6 total, 6 up, 6 in 2024-09-21T04:22:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: Standby daemon mds.cephfs.smithi148.fpatxi assigned to filesystem cephfs as rank 0 2024-09-21T04:22:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:26.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:26.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:26 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:replay} 2 up:standby 2024-09-21T04:22:27.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:27 smithi110 ceph-mon[92455]: pgmap v200: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T04:22:27.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:27 smithi148 ceph-mon[78701]: pgmap v200: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T04:22:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:28 smithi110 ceph-mon[92455]: pgmap v201: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T04:22:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:28 smithi148 ceph-mon[78701]: pgmap v201: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T04:22:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:30 smithi110 ceph-mon[92455]: pgmap v202: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 3 op/s 2024-09-21T04:22:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:30 smithi148 ceph-mon[78701]: pgmap v202: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 3 op/s 2024-09-21T04:22:33.366 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:32 smithi148 ceph-mon[78701]: pgmap v203: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-09-21T04:22:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:32 smithi110 ceph-mon[92455]: pgmap v203: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-09-21T04:22:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:34 smithi110 ceph-mon[92455]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0 2024-09-21T04:22:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:34 smithi110 ceph-mon[92455]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0.000999977 2024-09-21T04:22:35.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:34 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:reconnect 2024-09-21T04:22:35.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:34 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:reconnect} 2 up:standby 2024-09-21T04:22:35.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:34 smithi110 ceph-mon[92455]: pgmap v204: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-09-21T04:22:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:34 smithi148 ceph-mon[78701]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0 2024-09-21T04:22:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:34 smithi148 ceph-mon[78701]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0.000999977 2024-09-21T04:22:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:34 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:reconnect 2024-09-21T04:22:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:34 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:reconnect} 2 up:standby 2024-09-21T04:22:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:34 smithi148 ceph-mon[78701]: pgmap v204: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-09-21T04:22:36.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:35 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:rejoin 2024-09-21T04:22:36.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:35 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:rejoin} 2 up:standby 2024-09-21T04:22:36.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:35 smithi110 ceph-mon[92455]: daemon mds.cephfs.smithi148.fpatxi is now active in filesystem cephfs as rank 0 2024-09-21T04:22:36.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:35 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:rejoin 2024-09-21T04:22:36.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:35 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi148.fpatxi=up:rejoin} 2 up:standby 2024-09-21T04:22:36.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:35 smithi148 ceph-mon[78701]: daemon mds.cephfs.smithi148.fpatxi is now active in filesystem cephfs as rank 0 2024-09-21T04:22:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:37 smithi148 ceph-mon[78701]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:37 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:22:37.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:37 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:active 2024-09-21T04:22:37.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:37 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi148.fpatxi=up:active} 2 up:standby 2024-09-21T04:22:37.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:37 smithi148 ceph-mon[78701]: pgmap v205: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 97 B/s wr, 7 op/s 2024-09-21T04:22:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:37 smithi110 ceph-mon[92455]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:37 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:22:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:37 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6826/475239845,v1:172.21.15.148:6827/475239845] up:active 2024-09-21T04:22:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:37 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi148.fpatxi=up:active} 2 up:standby 2024-09-21T04:22:37.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:37 smithi110 ceph-mon[92455]: pgmap v205: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 97 B/s wr, 7 op/s 2024-09-21T04:22:39.077 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:39.077 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:39.077 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:39.077 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] up:boot 2024-09-21T04:22:39.078 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi148.fpatxi=up:active} 3 up:standby 2024-09-21T04:22:39.078 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:22:39.078 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] up:boot 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi148.fpatxi=up:active} 3 up:standby 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:22:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:40.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:39 smithi110 ceph-mon[92455]: pgmap v206: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T04:22:40.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:39 smithi148 ceph-mon[78701]: pgmap v206: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T04:22:41.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:41 smithi148 ceph-mon[78701]: pgmap v207: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T04:22:41.676 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.677 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:41 smithi110 ceph-mon[92455]: pgmap v207: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T04:22:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:41.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:43.269 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:42 smithi148 ceph-mon[78701]: pgmap v208: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T04:22:43.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:42 smithi110 ceph-mon[92455]: pgmap v208: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T04:22:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.fpatxi"]} : dispatch 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: Upgrade: It appears safe to stop mds.cephfs.smithi148.fpatxi 2024-09-21T04:22:45.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:45 smithi110 ceph-mon[92455]: pgmap v209: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:22:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi148.fpatxi"]} : dispatch 2024-09-21T04:22:45.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: Upgrade: It appears safe to stop mds.cephfs.smithi148.fpatxi 2024-09-21T04:22:45.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:45 smithi148 ceph-mon[78701]: pgmap v209: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:22:46.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:46 smithi148 ceph-mon[78701]: Upgrade: Updating mds.cephfs.smithi148.fpatxi 2024-09-21T04:22:46.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:46.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:46.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:46.858 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:46 smithi148 ceph-mon[78701]: Deploying daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:22:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:46 smithi110 ceph-mon[92455]: Upgrade: Updating mds.cephfs.smithi148.fpatxi 2024-09-21T04:22:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.fpatxi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:22:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:22:46.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:46 smithi110 ceph-mon[92455]: Deploying daemon mds.cephfs.smithi148.fpatxi on smithi148 2024-09-21T04:22:47.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[92451]: 2024-09-21T04:22:47.368+0000 7f34a828c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: pgmap v210: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: osdmap e95: 6 total, 6 up, 6 in 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: Standby daemon mds.cephfs.smithi110.trhyxr assigned to filesystem cephfs as rank 0 2024-09-21T04:22:47.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:47.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:47.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:47 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:replay} 2 up:standby 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: pgmap v210: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: osdmap e95: 6 total, 6 up, 6 in 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: Standby daemon mds.cephfs.smithi110.trhyxr assigned to filesystem cephfs as rank 0 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-21T04:22:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:47 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:replay} 2 up:standby 2024-09-21T04:22:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:48 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:22:49.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:48 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:22:49.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.225+0000 7f14688a5640 1 -- 172.21.15.110:0/335045597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 msgr2=0x7f14641000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.225+0000 7f14688a5640 1 --2- 172.21.15.110:0/335045597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f14641000b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1458009880 tx=0x7f145802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 -- 172.21.15.110:0/335045597 shutdown_connections 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 --2- 172.21.15.110:0/335045597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f14641000b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 --2- 172.21.15.110:0/335045597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 0x7f14640fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 -- 172.21.15.110:0/335045597 >> 172.21.15.110:0/335045597 conn(0x7f14640fa1c0 msgr2=0x7f14640fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 -- 172.21.15.110:0/335045597 shutdown_connections 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.226+0000 7f14688a5640 1 -- 172.21.15.110:0/335045597 wait complete. 2024-09-21T04:22:49.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 Processor -- start 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 -- start start 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 0x7f1464194540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1464195050 con 0x7f14640ffc30 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f14688a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641951c0 con 0x7f14640fea30 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f1461d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f1461d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56616/0 (socket says 172.21.15.110:56616) 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f1461d9b640 1 -- 172.21.15.110:0/2108775266 learned_addr learned my addr 172.21.15.110:0/2108775266 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.227+0000 7f146259c640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 0x7f1464194540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f1461d9b640 1 -- 172.21.15.110:0/2108775266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 msgr2=0x7f1464194540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f1461d9b640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 0x7f1464194540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f1461d9b640 1 -- 172.21.15.110:0/2108775266 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458009560 con 0x7f14640ffc30 2024-09-21T04:22:49.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f1461d9b640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1458005520 tx=0x7f1458005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1458004aa0 con 0x7f14640ffc30 2024-09-21T04:22:49.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1458004c00 con 0x7f14640ffc30 2024-09-21T04:22:49.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.228+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464199c00 con 0x7f14640ffc30 2024-09-21T04:22:49.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.229+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1458003b10 con 0x7f14640ffc30 2024-09-21T04:22:49.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.229+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146419a0f0 con 0x7f14640ffc30 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.230+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1458003d40 con 0x7f14640ffc30 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.230+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14640fee30 con 0x7f14640ffc30 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.231+0000 7f14537fe640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 0x7f144c07d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.231+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f14580bf330 con 0x7f14640ffc30 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.231+0000 7f146259c640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 0x7f144c07d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.232+0000 7f146259c640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 0x7f144c07d520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1454005d00 tx=0x7f1454005c50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.236+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1458082170 con 0x7f14640ffc30 2024-09-21T04:22:49.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.442+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f146402c450 con 0x7f144c07b060 2024-09-21T04:22:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.444+0000 7f14537fe640 1 -- 172.21.15.110:0/2108775266 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f146402c450 con 0x7f144c07b060 2024-09-21T04:22:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 msgr2=0x7f144c07d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 0x7f144c07d520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1454005d00 tx=0x7f1454005c50 comp rx=0 tx=0).stop 2024-09-21T04:22:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 msgr2=0x7f1464194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1458005520 tx=0x7f1458005550 comp rx=0 tx=0).stop 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 shutdown_connections 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f144c07b060 0x7f144c07d520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14640ffc30 0x7f1464194a80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 --2- 172.21.15.110:0/2108775266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640fea30 0x7f1464194540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 >> 172.21.15.110:0/2108775266 conn(0x7f14640fa1c0 msgr2=0x7f1464103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 shutdown_connections 2024-09-21T04:22:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.446+0000 7f14688a5640 1 -- 172.21.15.110:0/2108775266 wait complete. 2024-09-21T04:22:49.458 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:22:49.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.529+0000 7f4b21932640 1 -- 172.21.15.110:0/2826670141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c072eb0 msgr2=0x7f4b1c073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.529+0000 7f4b21932640 1 --2- 172.21.15.110:0/2826670141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c073310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009880 tx=0x7f4b1002a530 comp rx=0 tx=0).stop 2024-09-21T04:22:49.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.529+0000 7f4b21932640 1 -- 172.21.15.110:0/2826670141 shutdown_connections 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.529+0000 7f4b21932640 1 --2- 172.21.15.110:0/2826670141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c073310 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 --2- 172.21.15.110:0/2826670141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c074480 0x7f4b1c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 -- 172.21.15.110:0/2826670141 >> 172.21.15.110:0/2826670141 conn(0x7f4b1c0fb320 msgr2=0x7f4b1c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 -- 172.21.15.110:0/2826670141 shutdown_connections 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 -- 172.21.15.110:0/2826670141 wait complete. 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 Processor -- start 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.530+0000 7f4b21932640 1 -- start start 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b21932640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b21932640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b21932640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1982f0 con 0x7f4b1c074480 2024-09-21T04:22:49.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b21932640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c198460 con 0x7f4b1c072eb0 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56628/0 (socket says 172.21.15.110:56628) 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 -- 172.21.15.110:0/205628151 learned_addr learned my addr 172.21.15.110:0/205628151 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1affd640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 -- 172.21.15.110:0/205628151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c072eb0 msgr2=0x7f4b1c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 -- 172.21.15.110:0/205628151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009560 con 0x7f4b1c074480 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.531+0000 7f4b1a7fc640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b100055a0 tx=0x7f4b100055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.532+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b10004b20 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.532+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b10004c80 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.532+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b10003bc0 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.532+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c19cea0 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.532+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c19d320 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.533+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c191900 con 0x7f4b1c074480 2024-09-21T04:22:49.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.534+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4b1002ad50 con 0x7f4b1c074480 2024-09-21T04:22:49.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.535+0000 7f4b20930640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 0x7f4b0407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.535+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f4b100bf3b0 con 0x7f4b1c074480 2024-09-21T04:22:49.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.535+0000 7f4b1affd640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 0x7f4b0407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.536+0000 7f4b1affd640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 0x7f4b0407d730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00bbe0 tx=0x7f4b0c00b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.540+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4b10082320 con 0x7f4b1c074480 2024-09-21T04:22:49.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.774+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b1c02c5e0 con 0x7f4b0407b270 2024-09-21T04:22:49.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.775+0000 7f4b20930640 1 -- 172.21.15.110:0/205628151 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b1c02c5e0 con 0x7f4b0407b270 2024-09-21T04:22:49.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.777+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 msgr2=0x7f4b0407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.777+0000 7f4b21932640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 0x7f4b0407d730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00bbe0 tx=0x7f4b0c00b390 comp rx=0 tx=0).stop 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.777+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 msgr2=0x7f4b1c197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b100055a0 tx=0x7f4b100055d0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 shutdown_connections 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4b0407b270 0x7f4b0407d730 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b1c074480 0x7f4b1c197d20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 --2- 172.21.15.110:0/205628151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b1c072eb0 0x7f4b1c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 >> 172.21.15.110:0/205628151 conn(0x7f4b1c0fb320 msgr2=0x7f4b1c105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 shutdown_connections 2024-09-21T04:22:49.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.778+0000 7f4b21932640 1 -- 172.21.15.110:0/205628151 wait complete. 2024-09-21T04:22:49.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 -- 172.21.15.110:0/130494717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 msgr2=0x7f37200fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 --2- 172.21.15.110:0/130494717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f37200fff70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3710009880 tx=0x7f371002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 -- 172.21.15.110:0/130494717 shutdown_connections 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 --2- 172.21.15.110:0/130494717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37201011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 --2- 172.21.15.110:0/130494717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f37200fff70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 -- 172.21.15.110:0/130494717 >> 172.21.15.110:0/130494717 conn(0x7f37200fb2e0 msgr2=0x7f37200fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 -- 172.21.15.110:0/130494717 shutdown_connections 2024-09-21T04:22:49.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.859+0000 7f37249dd640 1 -- 172.21.15.110:0/130494717 wait complete. 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 Processor -- start 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 -- start start 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37200728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720072de0 con 0x7f37200ffb70 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f37249dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720072f50 con 0x7f3720100d70 2024-09-21T04:22:49.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f371e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f371e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56650/0 (socket says 172.21.15.110:56650) 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.860+0000 7f371e59c640 1 -- 172.21.15.110:0/2421247995 learned_addr learned my addr 172.21.15.110:0/2421247995 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371dd9b640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37200728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371e59c640 1 -- 172.21.15.110:0/2421247995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 msgr2=0x7f37200728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371e59c640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37200728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371e59c640 1 -- 172.21.15.110:0/2421247995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710009560 con 0x7f37200ffb70 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371dd9b640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37200728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f371e59c640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f371002a9c0 tx=0x7f3710005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710004b20 con 0x7f37200ffb70 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37200731d0 con 0x7f37200ffb70 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3710004c80 con 0x7f37200ffb70 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710003bc0 con 0x7f37200ffb70 2024-09-21T04:22:49.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.861+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372019d790 con 0x7f37200ffb70 2024-09-21T04:22:49.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.863+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37200ffff0 con 0x7f37200ffb70 2024-09-21T04:22:49.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.863+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f3710031640 con 0x7f37200ffb70 2024-09-21T04:22:49.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.864+0000 7f370f7fe640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 0x7f370807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:49.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.864+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f37100bf870 con 0x7f37200ffb70 2024-09-21T04:22:49.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.864+0000 7f371dd9b640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 0x7f370807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:49.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.865+0000 7f371dd9b640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 0x7f370807d740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3720073fd0 tx=0x7f3714008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:49.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:49.870+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f37100826b0 con 0x7f37200ffb70 2024-09-21T04:22:50.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.064+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37201056c0 con 0x7f370807b280 2024-09-21T04:22:50.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.073+0000 7f370f7fe640 1 -- 172.21.15.110:0/2421247995 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f37201056c0 con 0x7f370807b280 2024-09-21T04:22:50.076 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (14m) 30s ago 16m 25.9M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (7m) 30s ago 16m 8833k - 18.2.4-871-g6a1bdf85 0adab276c97e a4f277ac3df2 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (7m) 9s ago 15m 8824k - 18.2.4-871-g6a1bdf85 0adab276c97e 4976397775ab 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (4m) 30s ago 16m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (4m) 9s ago 15m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (14m) 30s ago 15m 83.9M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (36s) 30s ago 9m 11.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (58s) 30s ago 10m 13.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (9m) 9s ago 9m 84.0M - 18.2.4-871-g6a1bdf85 0adab276c97e c6474cfeab1a 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (12s) 9s ago 10m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (6m) 30s ago 17m 521M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (5m) 9s ago 15m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:22:50.077 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (5m) 30s ago 17m 51.1M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (5m) 9s ago 15m 41.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (16m) 30s ago 16m 16.9M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (15m) 9s ago 15m 15.2M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 30s ago 13m 117M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (3m) 30s ago 13m 152M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 30s ago 12m 174M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (2m) 9s ago 12m 140M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (106s) 9s ago 12m 103M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (72s) 9s ago 11m 102M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:22:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 30s ago 15m 64.5M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 msgr2=0x7f370807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 0x7f370807d740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3720073fd0 tx=0x7f3714008040 comp rx=0 tx=0).stop 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 msgr2=0x7f3720074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f371002a9c0 tx=0x7f3710005610 comp rx=0 tx=0).stop 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 shutdown_connections 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f370807b280 0x7f370807d740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3720100d70 0x7f37200728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 --2- 172.21.15.110:0/2421247995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37200ffb70 0x7f3720074250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 >> 172.21.15.110:0/2421247995 conn(0x7f37200fb2e0 msgr2=0x7f3720103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 shutdown_connections 2024-09-21T04:22:50.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.076+0000 7f37249dd640 1 -- 172.21.15.110:0/2421247995 wait complete. 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 -- 172.21.15.110:0/1767555772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f600ffb70 msgr2=0x7f8f600fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/1767555772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f600ffb70 0x7f8f600fff70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009880 tx=0x7f8f5402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 -- 172.21.15.110:0/1767555772 shutdown_connections 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/1767555772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f60100d70 0x7f8f601011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/1767555772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f600ffb70 0x7f8f600fff70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 -- 172.21.15.110:0/1767555772 >> 172.21.15.110:0/1767555772 conn(0x7f8f600fb2c0 msgr2=0x7f8f600fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.163 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:49 smithi110 ceph-mon[92455]: pgmap v212: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-21T04:22:50.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 -- 172.21.15.110:0/1767555772 shutdown_connections 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.160+0000 7f8f66c5c640 1 -- 172.21.15.110:0/1767555772 wait complete. 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 Processor -- start 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 -- start start 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f600ffb70 0x7f8f60197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60198360 con 0x7f8f60100d70 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f66c5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601984d0 con 0x7f8f600ffb70 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f5ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f5ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56670/0 (socket says 172.21.15.110:56670) 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.161+0000 7f8f5ffff640 1 -- 172.21.15.110:0/93839613 learned_addr learned my addr 172.21.15.110:0/93839613 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5ffff640 1 -- 172.21.15.110:0/93839613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f600ffb70 msgr2=0x7f8f60197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5ffff640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f600ffb70 0x7f8f60197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5ffff640 1 -- 172.21.15.110:0/93839613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54009560 con 0x7f8f60100d70 2024-09-21T04:22:50.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5ffff640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c920 tx=0x7f8f4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c007b80 con 0x7f8f60100d70 2024-09-21T04:22:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f4c0054e0 con 0x7f8f60100d70 2024-09-21T04:22:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6019cf70 con 0x7f8f60100d70 2024-09-21T04:22:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.162+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c00f450 con 0x7f8f60100d70 2024-09-21T04:22:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.163+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6019d510 con 0x7f8f60100d70 2024-09-21T04:22:50.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.164+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f600fff70 con 0x7f8f60100d70 2024-09-21T04:22:50.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.164+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8f4c004f60 con 0x7f8f60100d70 2024-09-21T04:22:50.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.165+0000 7f8f5dffb640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 0x7f8f5007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.165+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f8f4c09e230 con 0x7f8f60100d70 2024-09-21T04:22:50.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.165+0000 7f8f649f8640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 0x7f8f5007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.166+0000 7f8f649f8640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 0x7f8f5007d740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f5402a9c0 tx=0x7f8f54000bc0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.169+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8f4c0610f0 con 0x7f8f60100d70 2024-09-21T04:22:50.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:49 smithi148 ceph-mon[78701]: pgmap v212: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-21T04:22:50.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.465+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f6002c570 con 0x7f8f60100d70 2024-09-21T04:22:50.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.465+0000 7f8f5dffb640 1 -- 172.21.15.110:0/93839613 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8f4c064210 con 0x7f8f60100d70 2024-09-21T04:22:50.468 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:50.469 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:22:50.470 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 msgr2=0x7f8f5007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 0x7f8f5007d740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f5402a9c0 tx=0x7f8f54000bc0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 msgr2=0x7f8f60197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c920 tx=0x7f8f4c00cde0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 shutdown_connections 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8f5007b280 0x7f8f5007d740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f60100d70 0x7f8f60197d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 --2- 172.21.15.110:0/93839613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f600ffb70 0x7f8f60197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 >> 172.21.15.110:0/93839613 conn(0x7f8f600fb2c0 msgr2=0x7f8f60101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 shutdown_connections 2024-09-21T04:22:50.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.468+0000 7f8f66c5c640 1 -- 172.21.15.110:0/93839613 wait complete. 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.549+0000 7fe426afd640 1 -- 172.21.15.110:0/1563148116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420102fb0 msgr2=0x7fe4201053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.549+0000 7fe426afd640 1 --2- 172.21.15.110:0/1563148116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420102fb0 0x7fe4201053a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009880 tx=0x7fe40c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- 172.21.15.110:0/1563148116 shutdown_connections 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 --2- 172.21.15.110:0/1563148116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420102fb0 0x7fe4201053a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 --2- 172.21.15.110:0/1563148116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420100680 0x7fe420102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- 172.21.15.110:0/1563148116 >> 172.21.15.110:0/1563148116 conn(0x7fe4200fa1c0 msgr2=0x7fe4200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- 172.21.15.110:0/1563148116 shutdown_connections 2024-09-21T04:22:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- 172.21.15.110:0/1563148116 wait complete. 2024-09-21T04:22:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 Processor -- start 2024-09-21T04:22:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- start start 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 0x7fe4200fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200feb70 con 0x7fe420100680 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.550+0000 7fe426afd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200fece0 con 0x7fe420102fb0 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56696/0 (socket says 172.21.15.110:56696) 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 -- 172.21.15.110:0/108904408 learned_addr learned my addr 172.21.15.110:0/108904408 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe41ffff640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 0x7fe4200fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 -- 172.21.15.110:0/108904408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 msgr2=0x7fe4200fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 0x7fe4200fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 -- 172.21.15.110:0/108904408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c009560 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe41ffff640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 0x7fe4200fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe424899640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe41400b420 tx=0x7fe41400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe414011800 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe414011e20 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4200fefc0 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.551+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41400f2e0 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.552+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42019d780 con 0x7fe420100680 2024-09-21T04:22:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.552+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4201087a0 con 0x7fe420100680 2024-09-21T04:22:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.553+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fe414010380 con 0x7fe420100680 2024-09-21T04:22:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.554+0000 7fe41dffb640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 0x7fe41007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.554+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe41409dcb0 con 0x7fe420100680 2024-09-21T04:22:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.554+0000 7fe41ffff640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 0x7fe41007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.555+0000 7fe41ffff640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 0x7fe41007d740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000c00 tx=0x7fe40c0314a0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.558+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe414060b70 con 0x7fe420100680 2024-09-21T04:22:50.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.783+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe42019d910 con 0x7fe420100680 2024-09-21T04:22:50.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.784+0000 7fe41dffb640 1 -- 172.21.15.110:0/108904408 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1793 (secure 0 0 0) 0x7fe42019d910 con 0x7fe420100680 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:e31 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:47:375015+0000 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 31 2024-09-21T04:22:50.787 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:47.374997+0000 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 95 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:22:50.788 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:34404} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:50.789 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:22:50.790 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:22:50.790 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:22:50.790 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 31 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 msgr2=0x7fe41007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 0x7fe41007d740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000c00 tx=0x7fe40c0314a0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 msgr2=0x7fe4200fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe41400b420 tx=0x7fe41400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 shutdown_connections 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe41007b280 0x7fe41007d740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe420102fb0 0x7fe4200fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 --2- 172.21.15.110:0/108904408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe420100680 0x7fe4200fffe0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 >> 172.21.15.110:0/108904408 conn(0x7fe4200fa1c0 msgr2=0x7fe4200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 shutdown_connections 2024-09-21T04:22:50.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.787+0000 7fe426afd640 1 -- 172.21.15.110:0/108904408 wait complete. 2024-09-21T04:22:50.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- 172.21.15.110:0/1378837425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 msgr2=0x7fd0e00fe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/1378837425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e00fe7b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009880 tx=0x7fd0dc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- 172.21.15.110:0/1378837425 shutdown_connections 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/1378837425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0e00fecf0 0x7fd0e00ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/1378837425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e00fe7b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- 172.21.15.110:0/1378837425 >> 172.21.15.110:0/1378837425 conn(0x7fd0e00fa1c0 msgr2=0x7fd0e00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- 172.21.15.110:0/1378837425 shutdown_connections 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- 172.21.15.110:0/1378837425 wait complete. 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 Processor -- start 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.871+0000 7fd0e87c6640 1 -- start start 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e87c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e87c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0e00fecf0 0x7fd0e0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e87c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0198310 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e87c6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0198480 con 0x7fd0e00fecf0 2024-09-21T04:22:50.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56716/0 (socket says 172.21.15.110:56716) 2024-09-21T04:22:50.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 -- 172.21.15.110:0/511236477 learned_addr learned my addr 172.21.15.110:0/511236477 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 -- 172.21.15.110:0/511236477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0e00fecf0 msgr2=0x7fd0e0197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0e00fecf0 0x7fd0e0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 -- 172.21.15.110:0/511236477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009560 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.872+0000 7fd0e6562640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005520 tx=0x7fd0dc005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.873+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0dc004aa0 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.873+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0dc004c00 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.873+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e019ce70 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.873+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0dc003b10 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.873+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e019d3b0 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.874+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e0191890 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.875+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd0dc003d60 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.876+0000 7fd0d77fe640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 0x7fd0cc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:50.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.876+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fd0dc0bf740 con 0x7fd0e00fe3b0 2024-09-21T04:22:50.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.876+0000 7fd0e5d61640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 0x7fd0cc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:50.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.876+0000 7fd0e5d61640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 0x7fd0cc07d740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0005fd0 tx=0x7fd0d0005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:50.884+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd0dc082580 con 0x7fd0e00fe3b0 2024-09-21T04:22:51.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.089+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0e002c5a0 con 0x7fd0cc07b280 2024-09-21T04:22:51.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:50 smithi110 ceph-mon[92455]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:51.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:50 smithi110 ceph-mon[92455]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:51.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:50 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/93839613' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:51.093 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:50 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/108904408' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:22:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.091+0000 7fd0d77fe640 1 -- 172.21.15.110:0/511236477 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd0e002c5a0 con 0x7fd0cc07b280 2024-09-21T04:22:51.094 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:22:51.094 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-09-21T04:22:51.095 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:22:51.096 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:22:51.096 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:22:51.096 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:22:51.096 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:22:51.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 msgr2=0x7fd0cc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 0x7fd0cc07d740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0005fd0 tx=0x7fd0d0005850 comp rx=0 tx=0).stop 2024-09-21T04:22:51.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 msgr2=0x7fd0e0197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005520 tx=0x7fd0dc005550 comp rx=0 tx=0).stop 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 shutdown_connections 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd0cc07b280 0x7fd0cc07d740 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0e00fecf0 0x7fd0e0197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 --2- 172.21.15.110:0/511236477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e00fe3b0 0x7fd0e0197800 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 >> 172.21.15.110:0/511236477 conn(0x7fd0e00fa1c0 msgr2=0x7fd0e0106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 shutdown_connections 2024-09-21T04:22:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.093+0000 7fd0e87c6640 1 -- 172.21.15.110:0/511236477 wait complete. 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 -- 172.21.15.110:0/914677996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c500ffb70 msgr2=0x7f7c500fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/914677996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c500ffb70 0x7f7c500fff70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009880 tx=0x7f7c4002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 -- 172.21.15.110:0/914677996 shutdown_connections 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/914677996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c50100d70 0x7f7c501011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/914677996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c500ffb70 0x7f7c500fff70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.174+0000 7f7c56a0a640 1 -- 172.21.15.110:0/914677996 >> 172.21.15.110:0/914677996 conn(0x7f7c500fb2e0 msgr2=0x7f7c500fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.175+0000 7f7c56a0a640 1 -- 172.21.15.110:0/914677996 shutdown_connections 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.175+0000 7f7c56a0a640 1 -- 172.21.15.110:0/914677996 wait complete. 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.175+0000 7f7c56a0a640 1 Processor -- start 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.175+0000 7f7c56a0a640 1 -- start start 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c56a0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffb70 0x7f7c50195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c56a0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c56a0a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50196240 con 0x7f7c50100d70 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c56a0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501963b0 con 0x7f7c500ffb70 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56734/0 (socket says 172.21.15.110:56734) 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 -- 172.21.15.110:0/679107541 learned_addr learned my addr 172.21.15.110:0/679107541 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4ffff640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffb70 0x7f7c50195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 -- 172.21.15.110:0/679107541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffb70 msgr2=0x7f7c50195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffb70 0x7f7c50195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 -- 172.21.15.110:0/679107541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40009560 con 0x7f7c50100d70 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4f7fe640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b450 tx=0x7f7c4400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.176+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c44011860 con 0x7f7c50100d70 2024-09-21T04:22:51.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.177+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c44011e80 con 0x7f7c50100d70 2024-09-21T04:22:51.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.177+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5019ae50 con 0x7f7c50100d70 2024-09-21T04:22:51.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.177+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4400f570 con 0x7f7c50100d70 2024-09-21T04:22:51.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.177+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5019b3f0 con 0x7f7c50100d70 2024-09-21T04:22:51.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.178+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c005280 con 0x7f7c50100d70 2024-09-21T04:22:51.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.179+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f7c440119c0 con 0x7f7c50100d70 2024-09-21T04:22:51.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.180+0000 7f7c4d7fa640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 0x7f7c3807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:22:51.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.180+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f7c4409e1b0 con 0x7f7c50100d70 2024-09-21T04:22:51.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.180+0000 7f7c4ffff640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 0x7f7c3807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:22:51.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.180+0000 7f7c4ffff640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 0x7f7c3807d740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40031500 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:22:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:50 smithi148 ceph-mon[78701]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:50 smithi148 ceph-mon[78701]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:50 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/93839613' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:22:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:50 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/108904408' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:22:51.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.188+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f7c44064770 con 0x7f7c50100d70 2024-09-21T04:22:51.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.445+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c3c0050f0 con 0x7f7c50100d70 2024-09-21T04:22:51.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.445+0000 7f7c4d7fa640 1 -- 172.21.15.110:0/679107541 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f7c44064110 con 0x7f7c50100d70 2024-09-21T04:22:51.448 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem is degraded 2024-09-21T04:22:51.449 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:22:51.449 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs is degraded 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 msgr2=0x7f7c3807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 0x7f7c3807d740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006010 tx=0x7f7c40031500 comp rx=0 tx=0).stop 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 msgr2=0x7f7c50195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400b450 tx=0x7f7c4400b910 comp rx=0 tx=0).stop 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 shutdown_connections 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7c3807b280 0x7f7c3807d740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c50100d70 0x7f7c50195c70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 --2- 172.21.15.110:0/679107541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffb70 0x7f7c50195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 >> 172.21.15.110:0/679107541 conn(0x7f7c500fb2e0 msgr2=0x7f7c50103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:22:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 shutdown_connections 2024-09-21T04:22:51.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:22:51.448+0000 7f7c56a0a640 1 -- 172.21.15.110:0/679107541 wait complete. 2024-09-21T04:22:52.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:51 smithi148 ceph-mon[78701]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:52.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:51 smithi148 ceph-mon[78701]: pgmap v213: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-09-21T04:22:52.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:51 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/679107541' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:22:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:51 smithi110 ceph-mon[92455]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:51 smithi110 ceph-mon[92455]: pgmap v213: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-09-21T04:22:52.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:51 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/679107541' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:22:53.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:52 smithi148 ceph-mon[78701]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:53.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:52 smithi110 ceph-mon[92455]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:22:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:53 smithi148 ceph-mon[78701]: pgmap v214: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-09-21T04:22:54.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:54.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:53 smithi110 ceph-mon[92455]: pgmap v214: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-09-21T04:22:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:22:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:55 smithi148 ceph-mon[78701]: pgmap v215: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-09-21T04:22:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:55 smithi148 ceph-mon[78701]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0.000999983 2024-09-21T04:22:56.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:55 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:reconnect 2024-09-21T04:22:56.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:55 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:reconnect} 2 up:standby 2024-09-21T04:22:56.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:55 smithi148 ceph-mon[78701]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0.00399993 2024-09-21T04:22:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:55 smithi110 ceph-mon[92455]: pgmap v215: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-09-21T04:22:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:55 smithi110 ceph-mon[92455]: reconnect by client.24325 v1:172.21.15.110:0/290757620 after 0.000999983 2024-09-21T04:22:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:55 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:reconnect 2024-09-21T04:22:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:55 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:reconnect} 2 up:standby 2024-09-21T04:22:56.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:55 smithi110 ceph-mon[92455]: reconnect by client.14600 v1:172.21.15.148:0/3441795895 after 0.00399993 2024-09-21T04:22:57.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:56 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:rejoin 2024-09-21T04:22:57.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:56 smithi148 ceph-mon[78701]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:rejoin} 2 up:standby 2024-09-21T04:22:57.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:56 smithi148 ceph-mon[78701]: daemon mds.cephfs.smithi110.trhyxr is now active in filesystem cephfs as rank 0 2024-09-21T04:22:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:56 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:rejoin 2024-09-21T04:22:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:56 smithi110 ceph-mon[92455]: fsmap cephfs:1/1 {0=cephfs.smithi110.trhyxr=up:rejoin} 2 up:standby 2024-09-21T04:22:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:56 smithi110 ceph-mon[92455]: daemon mds.cephfs.smithi110.trhyxr is now active in filesystem cephfs as rank 0 2024-09-21T04:22:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: pgmap v216: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:22:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: Cluster is now healthy 2024-09-21T04:22:58.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:active 2024-09-21T04:22:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2 up:standby 2024-09-21T04:22:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:58.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: pgmap v216: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: Cluster is now healthy 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] up:active 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 2 up:standby 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:22:58.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:22:59.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:58 smithi148 ceph-mon[78701]: mds.? [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] up:boot 2024-09-21T04:22:59.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:58 smithi148 ceph-mon[78701]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:22:59.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:58 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:22:59.264 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:22:58 smithi148 ceph-mon[78701]: pgmap v217: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 88 B/s wr, 6 op/s 2024-09-21T04:22:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:58 smithi110 ceph-mon[92455]: mds.? [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] up:boot 2024-09-21T04:22:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:58 smithi110 ceph-mon[92455]: fsmap cephfs:1 {0=cephfs.smithi110.trhyxr=up:active} 3 up:standby 2024-09-21T04:22:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:58 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "mds metadata", "who": "cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:22:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:22:58 smithi110 ceph-mon[92455]: pgmap v217: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 88 B/s wr, 6 op/s 2024-09-21T04:23:01.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:01 smithi148 ceph-mon[78701]: pgmap v218: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:23:01.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:01 smithi110 ceph-mon[92455]: pgmap v218: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:23:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:01.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:03.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:02 smithi110 ceph-mon[92455]: pgmap v219: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T04:23:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:02 smithi148 ceph-mon[78701]: pgmap v219: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-21T04:23:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all mds 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.faxwju"}]': finished 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.trhyxr"}]': finished 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.fpatxi"}]': finished 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.tpueww"}]': finished 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all rgw 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.419 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.420 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T04:23:05.420 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.420 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.420 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:05 smithi110 ceph-mon[92455]: pgmap v220: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-21T04:23:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all mds 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.faxwju"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.faxwju"}]': finished 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.trhyxr"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.trhyxr"}]': finished 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.fpatxi"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.fpatxi"}]': finished 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.tpueww"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.tpueww"}]': finished 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all rgw 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:05.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:05.443 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:05 smithi148 ceph-mon[78701]: pgmap v220: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-21T04:23:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:06 smithi110 ceph-mon[92455]: Upgrade: Updating ceph-exporter.smithi110 (1/2) 2024-09-21T04:23:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:23:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:06 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:07.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:06 smithi110 ceph-mon[92455]: Deploying daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:23:07.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:06 smithi148 ceph-mon[78701]: Upgrade: Updating ceph-exporter.smithi110 (1/2) 2024-09-21T04:23:07.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:07.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi110", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:23:07.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:06 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:07.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:06 smithi148 ceph-mon[78701]: Deploying daemon ceph-exporter.smithi110 on smithi110 2024-09-21T04:23:08.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:07 smithi110 ceph-mon[92455]: pgmap v221: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:23:08.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:07 smithi148 ceph-mon[78701]: pgmap v221: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-09-21T04:23:09.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:09.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:09.356 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:09 smithi110 ceph-mon[92455]: pgmap v222: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-09-21T04:23:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:09 smithi148 ceph-mon[78701]: pgmap v222: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-09-21T04:23:10.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:10 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:10.857 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:10 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:10.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:10 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:10.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:10 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:11.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:11 smithi110 ceph-mon[92455]: pgmap v223: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 21 KiB/s rd, 4.3 KiB/s wr, 4 op/s 2024-09-21T04:23:11.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:11.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:23:11.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:11 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:11 smithi148 ceph-mon[78701]: pgmap v223: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 21 KiB/s rd, 4.3 KiB/s wr, 4 op/s 2024-09-21T04:23:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi148", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T04:23:11.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:11 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:12.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:12 smithi110 ceph-mon[92455]: Upgrade: Updating ceph-exporter.smithi148 (2/2) 2024-09-21T04:23:12.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:12 smithi110 ceph-mon[92455]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:23:12.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:12 smithi148 ceph-mon[78701]: Upgrade: Updating ceph-exporter.smithi148 (2/2) 2024-09-21T04:23:12.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:12 smithi148 ceph-mon[78701]: Deploying daemon ceph-exporter.smithi148 on smithi148 2024-09-21T04:23:13.679 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:13 smithi148 ceph-mon[78701]: pgmap v224: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 20 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-21T04:23:13.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:13 smithi110 ceph-mon[92455]: pgmap v224: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 20 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-21T04:23:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:15.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:15 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:15.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:15 smithi110 ceph-mon[92455]: pgmap v225: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:15.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:15 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:15.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:15 smithi148 ceph-mon[78701]: pgmap v225: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:17.921 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:17 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:17.921 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:17 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:17.921 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:17 smithi148 ceph-mon[78701]: pgmap v226: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:17.922 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:17 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:17.922 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:17 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:18.038 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:17 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:18.039 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:17 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:18.039 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:17 smithi110 ceph-mon[92455]: pgmap v226: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:18.039 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:17 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:18.039 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:17 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:19.994 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:19.994 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:19.995 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:19 smithi110 ceph-mon[92455]: pgmap v227: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:19.995 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:19.995 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:19 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:20.092 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:20.092 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:20.092 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:19 smithi148 ceph-mon[78701]: pgmap v227: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:20.092 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:20.092 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:19 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:21.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.533+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/2513942761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad01005c0 msgr2=0x7f8ad01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.533+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/2513942761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad01005c0 0x7f8ad01029b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098b0 tx=0x7f8acc02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.533+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/2513942761 shutdown_connections 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/2513942761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad0102ef0 0x7f8ad01052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/2513942761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad01005c0 0x7f8ad01029b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/2513942761 >> 172.21.15.110:0/2513942761 conn(0x7f8ad00fa1c0 msgr2=0x7f8ad00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/2513942761 shutdown_connections 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/2513942761 wait complete. 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 Processor -- start 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- start start 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad01005c0 0x7f8ad0195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad0196190 con 0x7f8ad0102ef0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.534+0000 7f8ad7dfb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad0196300 con 0x7f8ad01005c0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35388/0 (socket says 172.21.15.110:35388) 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 -- 172.21.15.110:0/3688960286 learned_addr learned my addr 172.21.15.110:0/3688960286 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5b97640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad01005c0 0x7f8ad0195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 -- 172.21.15.110:0/3688960286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad01005c0 msgr2=0x7f8ad0195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad01005c0 0x7f8ad0195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 -- 172.21.15.110:0/3688960286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009560 con 0x7f8ad0102ef0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad5396640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c920 tx=0x7f8ac000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac0007b80 con 0x7f8ad0102ef0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac0007ce0 con 0x7f8ad0102ef0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad019ada0 con 0x7f8ad0102ef0 2024-09-21T04:23:21.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac00187d0 con 0x7f8ad0102ef0 2024-09-21T04:23:21.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.535+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad019b340 con 0x7f8ad0102ef0 2024-09-21T04:23:21.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.537+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad01009a0 con 0x7f8ad0102ef0 2024-09-21T04:23:21.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.537+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8ac000f450 con 0x7f8ad0102ef0 2024-09-21T04:23:21.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.538+0000 7f8ac6ffd640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 0x7f8abc07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.538+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f8ac009e590 con 0x7f8ad0102ef0 2024-09-21T04:23:21.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.538+0000 7f8ad5b97640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 0x7f8abc07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.539+0000 7f8ad5b97640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 0x7f8abc07d740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8acc006010 tx=0x7f8acc0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:21.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.543+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8ac00613d0 con 0x7f8ad0102ef0 2024-09-21T04:23:21.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:21 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:21.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:21 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:21.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:21 smithi148 ceph-mon[78701]: pgmap v228: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:21.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.767+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ad0103990 con 0x7f8abc07b280 2024-09-21T04:23:21.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:21 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:21.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:21 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:21.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:21 smithi110 ceph-mon[92455]: pgmap v228: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:21.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.768+0000 7f8ac6ffd640 1 -- 172.21.15.110:0/3688960286 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f8ad0103990 con 0x7f8abc07b280 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.770+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 msgr2=0x7f8abc07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.770+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 0x7f8abc07d740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8acc006010 tx=0x7f8acc0314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 msgr2=0x7f8ad0195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c920 tx=0x7f8ac000cde0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 shutdown_connections 2024-09-21T04:23:21.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8abc07b280 0x7f8abc07d740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ad0102ef0 0x7f8ad0195bc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 --2- 172.21.15.110:0/3688960286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad01005c0 0x7f8ad0195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 >> 172.21.15.110:0/3688960286 conn(0x7f8ad00fa1c0 msgr2=0x7f8ad00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:21.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 shutdown_connections 2024-09-21T04:23:21.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.771+0000 7f8ad7dfb640 1 -- 172.21.15.110:0/3688960286 wait complete. 2024-09-21T04:23:21.782 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:23:21.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.853+0000 7fdc868e1640 1 -- 172.21.15.110:0/56392119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fed50 msgr2=0x7fdc800ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.853+0000 7fdc868e1640 1 --2- 172.21.15.110:0/56392119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fed50 0x7fdc800ff1b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009880 tx=0x7fdc7402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.853+0000 7fdc868e1640 1 -- 172.21.15.110:0/56392119 shutdown_connections 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.853+0000 7fdc868e1640 1 --2- 172.21.15.110:0/56392119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fed50 0x7fdc800ff1b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 --2- 172.21.15.110:0/56392119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fe410 0x7fdc800fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- 172.21.15.110:0/56392119 >> 172.21.15.110:0/56392119 conn(0x7fdc800fa1c0 msgr2=0x7fdc800fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- 172.21.15.110:0/56392119 shutdown_connections 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- 172.21.15.110:0/56392119 wait complete. 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 Processor -- start 2024-09-21T04:23:21.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- start start 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fed50 0x7fdc80197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801982f0 con 0x7fdc800fe410 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.854+0000 7fdc868e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80198460 con 0x7fdc800fed50 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35400/0 (socket says 172.21.15.110:35400) 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 -- 172.21.15.110:0/1434049093 learned_addr learned my addr 172.21.15.110:0/1434049093 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7f7fe640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fed50 0x7fdc80197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 -- 172.21.15.110:0/1434049093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fed50 msgr2=0x7fdc80197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fed50 0x7fdc80197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 -- 172.21.15.110:0/1434049093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74009560 con 0x7fdc800fe410 2024-09-21T04:23:21.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7ffff640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d720 tx=0x7fdc7000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc700051f0 con 0x7fdc800fe410 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8019cf00 con 0x7fdc800fe410 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc70009ce0 con 0x7fdc800fe410 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc70010460 con 0x7fdc800fe410 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.855+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8019d440 con 0x7fdc800fe410 2024-09-21T04:23:21.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.856+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8010a920 con 0x7fdc800fe410 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.857+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fdc70005350 con 0x7fdc800fe410 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.857+0000 7fdc7d7fa640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 0x7fdc6807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.858+0000 7fdc7f7fe640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 0x7fdc6807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.858+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fdc7009e080 con 0x7fdc800fe410 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.858+0000 7fdc7f7fe640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 0x7fdc6807d740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc74000c00 tx=0x7fdc740314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:21.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:21.862+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fdc70060ec0 con 0x7fdc800fe410 2024-09-21T04:23:22.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.062+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc80108160 con 0x7fdc6807b280 2024-09-21T04:23:22.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.064+0000 7fdc7d7fa640 1 -- 172.21.15.110:0/1434049093 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fdc80108160 con 0x7fdc6807b280 2024-09-21T04:23:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 msgr2=0x7fdc6807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 0x7fdc6807d740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc74000c00 tx=0x7fdc740314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 msgr2=0x7fdc801977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d720 tx=0x7fdc7000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 shutdown_connections 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdc6807b280 0x7fdc6807d740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc800fed50 0x7fdc80197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 --2- 172.21.15.110:0/1434049093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc800fe410 0x7fdc801977e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 >> 172.21.15.110:0/1434049093 conn(0x7fdc800fa1c0 msgr2=0x7fdc80106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 shutdown_connections 2024-09-21T04:23:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.066+0000 7fdc868e1640 1 -- 172.21.15.110:0/1434049093 wait complete. 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1225048027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d68100d70 msgr2=0x7f6d681031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1225048027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d68100d70 0x7f6d681031e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009880 tx=0x7f6d5c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1225048027 shutdown_connections 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1225048027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d68100d70 0x7f6d681031e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1225048027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d680fe420 0x7f6d68100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1225048027 >> 172.21.15.110:0/1225048027 conn(0x7f6d680fa230 msgr2=0x7f6d680fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1225048027 shutdown_connections 2024-09-21T04:23:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.149+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1225048027 wait complete. 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 Processor -- start 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 -- start start 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d68100d70 0x7f6d6819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6819c740 con 0x7f6d680fe420 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6ffb5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6819c8b0 con 0x7f6d68100d70 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6dd51640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6dd51640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58870/0 (socket says 172.21.15.110:58870) 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6dd51640 1 -- 172.21.15.110:0/1314459347 learned_addr learned my addr 172.21.15.110:0/1314459347 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.150+0000 7f6d6d550640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d68100d70 0x7f6d6819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6dd51640 1 -- 172.21.15.110:0/1314459347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d68100d70 msgr2=0x7f6d6819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6dd51640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d68100d70 0x7f6d6819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6dd51640 1 -- 172.21.15.110:0/1314459347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009560 con 0x7f6d680fe420 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6dd51640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800cc60 tx=0x7f6d5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d58004b20 con 0x7f6d680fe420 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d681a1340 con 0x7f6d680fe420 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d58004c80 con 0x7f6d680fe420 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5800f640 con 0x7f6d680fe420 2024-09-21T04:23:22.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.151+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d681a1910 con 0x7f6d680fe420 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.152+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6810ed90 con 0x7f6d680fe420 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.153+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f6d580055e0 con 0x7f6d680fe420 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.153+0000 7f6d66ffd640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 0x7f6d5007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.154+0000 7f6d6d550640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 0x7f6d5007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.154+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f6d5809e690 con 0x7f6d680fe420 2024-09-21T04:23:22.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.154+0000 7f6d6d550640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 0x7f6d5007d730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000c00 tx=0x7f6d5c004400 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.158+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f6d580614d0 con 0x7f6d680fe420 2024-09-21T04:23:22.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.347+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d68101710 con 0x7f6d5007b270 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.357+0000 7f6d66ffd640 1 -- 172.21.15.110:0/1314459347 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6d68101710 con 0x7f6d5007b270 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (14m) 3s ago 16m 26.0M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (13s) 3s ago 16m 8166k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 9c05a31416c2 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (8s) 5s ago 15m 7281k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3ed27bd24a31 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 3s ago 16m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (5m) 5s ago 15m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:23:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (14m) 3s ago 16m 84.8M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (68s) 3s ago 10m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (90s) 3s ago 10m 83.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (24s) 5s ago 10m 13.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 55fa666e3a5c 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (45s) 5s ago 10m 13.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (7m) 3s ago 17m 523M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (6m) 5s ago 15m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (6m) 3s ago 17m 56.6M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (5m) 5s ago 15m 44.6M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (16m) 3s ago 16m 17.2M - 1.5.0 0da6a335fe13 09e658c90eb1 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (15m) 5s ago 15m 14.9M - 1.5.0 0da6a335fe13 bab9d045d4b2 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 3s ago 14m 121M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 3s ago 13m 156M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:23:22.361 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (3m) 3s ago 13m 175M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:23:22.362 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (2m) 5s ago 12m 142M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:23:22.362 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (2m) 5s ago 12m 105M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:23:22.362 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (104s) 5s ago 12m 104M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:23:22.362 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 3s ago 16m 66.7M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:23:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 msgr2=0x7f6d5007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 0x7f6d5007d730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000c00 tx=0x7f6d5c004400 comp rx=0 tx=0).stop 2024-09-21T04:23:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 msgr2=0x7f6d6819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800cc60 tx=0x7f6d5800a620 comp rx=0 tx=0).stop 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 shutdown_connections 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6d5007b270 0x7f6d5007d730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d68100d70 0x7f6d6819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 --2- 172.21.15.110:0/1314459347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d680fe420 0x7f6d6819bc30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.359+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 >> 172.21.15.110:0/1314459347 conn(0x7f6d680fa230 msgr2=0x7f6d680fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.360+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 shutdown_connections 2024-09-21T04:23:22.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.360+0000 7f6d6ffb5640 1 -- 172.21.15.110:0/1314459347 wait complete. 2024-09-21T04:23:22.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 -- 172.21.15.110:0/933147356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fed40 msgr2=0x7fefe80ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 --2- 172.21.15.110:0/933147356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fed40 0x7fefe80ff1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009880 tx=0x7fefd802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 -- 172.21.15.110:0/933147356 shutdown_connections 2024-09-21T04:23:22.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 --2- 172.21.15.110:0/933147356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fed40 0x7fefe80ff1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 --2- 172.21.15.110:0/933147356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fe400 0x7fefe80fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 -- 172.21.15.110:0/933147356 >> 172.21.15.110:0/933147356 conn(0x7fefe80fa210 msgr2=0x7fefe80fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 -- 172.21.15.110:0/933147356 shutdown_connections 2024-09-21T04:23:22.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.446+0000 7feff027d640 1 -- 172.21.15.110:0/933147356 wait complete. 2024-09-21T04:23:22.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 Processor -- start 2024-09-21T04:23:22.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 -- start start 2024-09-21T04:23:22.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 0x7fefe8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8198320 con 0x7fefe80fe400 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7feff027d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8198490 con 0x7fefe80fed40 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58890/0 (socket says 172.21.15.110:58890) 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 -- 172.21.15.110:0/3212310553 learned_addr learned my addr 172.21.15.110:0/3212310553 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefed818640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 0x7fefe8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 -- 172.21.15.110:0/3212310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 msgr2=0x7fefe8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 0x7fefe8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefee019640 1 -- 172.21.15.110:0/3212310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009560 con 0x7fefe80fe400 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.447+0000 7fefed818640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 0x7fefe8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7fefee019640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d900 tx=0x7fefe400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe4009c00 con 0x7fefe80fe400 2024-09-21T04:23:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefe400b660 con 0x7fefe80fe400 2024-09-21T04:23:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe819cf30 con 0x7fefe80fe400 2024-09-21T04:23:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe400f680 con 0x7fefe80fe400 2024-09-21T04:23:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.448+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe819d470 con 0x7fefe80fe400 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.449+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80fe800 con 0x7fefe80fe400 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.450+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fefe4010ab0 con 0x7fefe80fe400 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.450+0000 7fefdf7fe640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 0x7fefd407d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.450+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fefe409e370 con 0x7fefe80fe400 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.450+0000 7fefed818640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 0x7fefd407d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.451+0000 7fefed818640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 0x7fefd407d790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd80314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.455+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fefe4061110 con 0x7fefe80fe400 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:22.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:22.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:22.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:22.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:22.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.725+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fefe802c4e0 con 0x7fefe80fe400 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.726+0000 7fefdf7fe640 1 -- 172.21.15.110:0/3212310553 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fefe4064230 con 0x7fefe80fe400 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:23:22.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:23:22.731 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 msgr2=0x7fefd407d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 0x7fefd407d790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd80314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 msgr2=0x7fefe8197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d900 tx=0x7fefe400ddc0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 shutdown_connections 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fefd407b2d0 0x7fefd407d790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe80fed40 0x7fefe8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 --2- 172.21.15.110:0/3212310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefe80fe400 0x7fefe8197810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 >> 172.21.15.110:0/3212310553 conn(0x7fefe80fa210 msgr2=0x7fefe8106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 shutdown_connections 2024-09-21T04:23:22.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.728+0000 7feff027d640 1 -- 172.21.15.110:0/3212310553 wait complete. 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 -- 172.21.15.110:0/3912160409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 msgr2=0x7f46a80fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 --2- 172.21.15.110:0/3912160409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a80fffb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4698009880 tx=0x7f469802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 -- 172.21.15.110:0/3912160409 shutdown_connections 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 --2- 172.21.15.110:0/3912160409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 0x7f46a8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 --2- 172.21.15.110:0/3912160409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a80fffb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 -- 172.21.15.110:0/3912160409 >> 172.21.15.110:0/3912160409 conn(0x7f46a80fb320 msgr2=0x7f46a80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 -- 172.21.15.110:0/3912160409 shutdown_connections 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.821+0000 7f46ae65e640 1 -- 172.21.15.110:0/3912160409 wait complete. 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 Processor -- start 2024-09-21T04:23:22.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 -- start start 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 0x7f46a8195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8196140 con 0x7f46a80ffbb0 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46ae65e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81962b0 con 0x7f46a8100db0 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.822+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58918/0 (socket says 172.21.15.110:58918) 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a7fff640 1 -- 172.21.15.110:0/2215904008 learned_addr learned my addr 172.21.15.110:0/2215904008 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a77fe640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 0x7f46a8195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a7fff640 1 -- 172.21.15.110:0/2215904008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 msgr2=0x7f46a8195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:22.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a7fff640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 0x7f46a8195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a7fff640 1 -- 172.21.15.110:0/2215904008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009560 con 0x7f46a80ffbb0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a7fff640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46980054f0 tx=0x7f4698005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4698004aa0 con 0x7f46a80ffbb0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a819acf0 con 0x7f46a80ffbb0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4698004c00 con 0x7f46a80ffbb0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4698003ac0 con 0x7f46a80ffbb0 2024-09-21T04:23:22.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.823+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a819b210 con 0x7f46a80ffbb0 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.824+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a818f750 con 0x7f46a80ffbb0 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.825+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4698003d30 con 0x7f46a80ffbb0 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.826+0000 7f46a57fa640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 0x7f469007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.826+0000 7f46a77fe640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 0x7f469007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.826+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f46980bf630 con 0x7f46a80ffbb0 2024-09-21T04:23:22.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.826+0000 7f46a77fe640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 0x7f469007d730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46a8196b50 tx=0x7f469c00b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:22.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:22.830+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f46980825a0 con 0x7f46a80ffbb0 2024-09-21T04:23:23.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.058+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46a8108a50 con 0x7f46a80ffbb0 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.059+0000 7f46a57fa640 1 -- 172.21.15.110:0/2215904008 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1982 (secure 0 0 0) 0x7f46980856c0 con 0x7f46a80ffbb0 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:57:935402+0000 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:23:23.062 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 34 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:56.930115+0000 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:23:23.063 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 95 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:23:23.064 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:34404} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:23.065 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 msgr2=0x7f469007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 0x7f469007d730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46a8196b50 tx=0x7f469c00b390 comp rx=0 tx=0).stop 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 msgr2=0x7f46a8195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46980054f0 tx=0x7f4698005520 comp rx=0 tx=0).stop 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 shutdown_connections 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f469007b270 0x7f469007d730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a8100db0 0x7f46a8195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 --2- 172.21.15.110:0/2215904008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46a80ffbb0 0x7f46a8195630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 >> 172.21.15.110:0/2215904008 conn(0x7f46a80fb320 msgr2=0x7f46a8104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 shutdown_connections 2024-09-21T04:23:23.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.061+0000 7f46ae65e640 1 -- 172.21.15.110:0/2215904008 wait complete. 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- 172.21.15.110:0/2887243707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 msgr2=0x7f1780105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 --2- 172.21.15.110:0/2887243707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780105350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f177c0098b0 tx=0x7f177c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- 172.21.15.110:0/2887243707 shutdown_connections 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 --2- 172.21.15.110:0/2887243707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780105350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 --2- 172.21.15.110:0/2887243707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 0x7f1780102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- 172.21.15.110:0/2887243707 >> 172.21.15.110:0/2887243707 conn(0x7f17800fa230 msgr2=0x7f17800fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- 172.21.15.110:0/2887243707 shutdown_connections 2024-09-21T04:23:23.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- 172.21.15.110:0/2887243707 wait complete. 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 Processor -- start 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.144+0000 7f1787583640 1 -- start start 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1787583640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 0x7f1780195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1787583640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1787583640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1780196170 con 0x7f1780102f60 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1787583640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17801962e0 con 0x7f1780100630 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58942/0 (socket says 172.21.15.110:58942) 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 -- 172.21.15.110:0/416783324 learned_addr learned my addr 172.21.15.110:0/416783324 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f178531f640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 0x7f1780195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 -- 172.21.15.110:0/416783324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 msgr2=0x7f1780195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 0x7f1780195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 -- 172.21.15.110:0/416783324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c009560 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.145+0000 7f1784b1e640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f177c005610 tx=0x7f177c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.146+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f177c004ba0 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.146+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177c004d00 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.146+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178019ad20 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.146+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f177c003bf0 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.146+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178019b290 con 0x7f1780102f60 2024-09-21T04:23:23.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.147+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17801087b0 con 0x7f1780102f60 2024-09-21T04:23:23.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.148+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f177c031640 con 0x7f1780102f60 2024-09-21T04:23:23.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.148+0000 7f17767fc640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 0x7f176c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.149+0000 7f178531f640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 0x7f176c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.149+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f177c0bf7a0 con 0x7f1780102f60 2024-09-21T04:23:23.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.149+0000 7f178531f640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 0x7f176c07d730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17700058c0 tx=0x7f177000a250 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:23.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.153+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f177c082690 con 0x7f1780102f60 2024-09-21T04:23:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.359+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1780103960 con 0x7f176c07b270 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.360+0000 7f17767fc640 1 -- 172.21.15.110:0/416783324 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f1780103960 con 0x7f176c07b270 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph-exporter", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "mds" 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "18/23 daemons upgraded", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading node-exporter daemons", 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:23:23.364 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:23:23.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 msgr2=0x7f176c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 0x7f176c07d730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17700058c0 tx=0x7f177000a250 comp rx=0 tx=0).stop 2024-09-21T04:23:23.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 msgr2=0x7f1780195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f177c005610 tx=0x7f177c005640 comp rx=0 tx=0).stop 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 shutdown_connections 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f176c07b270 0x7f176c07d730 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1780102f60 0x7f1780195ba0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 --2- 172.21.15.110:0/416783324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1780100630 0x7f1780195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 >> 172.21.15.110:0/416783324 conn(0x7f17800fa230 msgr2=0x7f17800fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 shutdown_connections 2024-09-21T04:23:23.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.362+0000 7f1787583640 1 -- 172.21.15.110:0/416783324 wait complete. 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 -- 172.21.15.110:0/2059816961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0ffc30 msgr2=0x7fb45c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2059816961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0ffc30 0x7fb45c1000b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb444009880 tx=0x7fb44402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 -- 172.21.15.110:0/2059816961 shutdown_connections 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2059816961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0ffc30 0x7fb45c1000b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2059816961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0fea30 0x7fb45c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 -- 172.21.15.110:0/2059816961 >> 172.21.15.110:0/2059816961 conn(0x7fb45c0fa1c0 msgr2=0x7fb45c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 -- 172.21.15.110:0/2059816961 shutdown_connections 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.444+0000 7fb4624f2640 1 -- 172.21.15.110:0/2059816961 wait complete. 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 Processor -- start 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 -- start start 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0ffc30 0x7fb45c195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c196160 con 0x7fb45c0fea30 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb4624f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1962d0 con 0x7fb45c0ffc30 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58956/0 (socket says 172.21.15.110:58956) 2024-09-21T04:23:23.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.445+0000 7fb45bfff640 1 -- 172.21.15.110:0/2078739581 learned_addr learned my addr 172.21.15.110:0/2078739581 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb453fff640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0ffc30 0x7fb45c195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb45bfff640 1 -- 172.21.15.110:0/2078739581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0ffc30 msgr2=0x7fb45c195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb45bfff640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0ffc30 0x7fb45c195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb45bfff640 1 -- 172.21.15.110:0/2078739581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb444009560 con 0x7fb45c0fea30 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb45bfff640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d900 tx=0x7fb44c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44c009c00 con 0x7fb45c0fea30 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c19ad70 con 0x7fb45c0fea30 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb44c009d60 con 0x7fb45c0fea30 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44c00f7b0 con 0x7fb45c0fea30 2024-09-21T04:23:23.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.446+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c19b230 con 0x7fb45c0fea30 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.447+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c107620 con 0x7fb45c0fea30 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.448+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb44c010460 con 0x7fb45c0fea30 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.448+0000 7fb459ffb640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 0x7fb44807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.448+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fb44c09e850 con 0x7fb45c0fea30 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.448+0000 7fb453fff640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 0x7fb44807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:23.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.449+0000 7fb453fff640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 0x7fb44807d740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb444000c00 tx=0x7fb444005d90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:23.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.453+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb44c061690 con 0x7fb45c0fea30 2024-09-21T04:23:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all ceph-exporter 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi110"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi110"}]': finished 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi148"} : dispatch 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi148"}]': finished 2024-09-21T04:23:23.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all iscsi 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all nfs 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: Upgrade: Setting container_image for all nvmeof 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3212310553' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: pgmap v229: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2215904008' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:23:23.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:23.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.709+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb45c107870 con 0x7fb45c0fea30 2024-09-21T04:23:23.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.710+0000 7fb459ffb640 1 -- 172.21.15.110:0/2078739581 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb44c0647b0 con 0x7fb45c0fea30 2024-09-21T04:23:23.713 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 msgr2=0x7fb44807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 0x7fb44807d740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb444000c00 tx=0x7fb444005d90 comp rx=0 tx=0).stop 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 msgr2=0x7fb45c195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d900 tx=0x7fb44c00ddc0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 shutdown_connections 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb44807b280 0x7fb44807d740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb45c0ffc30 0x7fb45c195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 --2- 172.21.15.110:0/2078739581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c0fea30 0x7fb45c195650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 >> 172.21.15.110:0/2078739581 conn(0x7fb45c0fa1c0 msgr2=0x7fb45c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 shutdown_connections 2024-09-21T04:23:23.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:23.712+0000 7fb4624f2640 1 -- 172.21.15.110:0/2078739581 wait complete. 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all ceph-exporter 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi110"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi110"}]': finished 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi148"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi148"}]': finished 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all iscsi 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all nfs 2024-09-21T04:23:23.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: Upgrade: Setting container_image for all nvmeof 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3212310553' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: pgmap v229: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2215904008' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:23:23.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:24 smithi110 ceph-mon[92455]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:24.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:24 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2078739581' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:23:24.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:24 smithi148 ceph-mon[78701]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:24.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:24 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2078739581' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:23:25.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:25 smithi148 ceph-mon[78701]: Upgrade: Updating node-exporter.smithi110 (1/2) 2024-09-21T04:23:25.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:25 smithi148 ceph-mon[78701]: Deploying daemon node-exporter.smithi110 on smithi110 2024-09-21T04:23:25.855 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:25 smithi148 ceph-mon[78701]: pgmap v230: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:25.903 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:25 smithi110 ceph-mon[92455]: Upgrade: Updating node-exporter.smithi110 (1/2) 2024-09-21T04:23:25.903 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:25 smithi110 ceph-mon[92455]: Deploying daemon node-exporter.smithi110 on smithi110 2024-09-21T04:23:25.903 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:25 smithi110 ceph-mon[92455]: pgmap v230: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:27.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:26 smithi110 ceph-mon[92455]: pgmap v231: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:26 smithi148 ceph-mon[78701]: pgmap v231: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:30.083 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:29 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:30.083 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:29 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:30.083 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:29 smithi148 ceph-mon[78701]: pgmap v232: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:30.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:29 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:30.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:29 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:30.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:29 smithi110 ceph-mon[92455]: pgmap v232: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:31.163 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:30 smithi148 ceph-mon[78701]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-09-21T04:23:31.163 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:30 smithi148 ceph-mon[78701]: Deploying daemon node-exporter.smithi148 on smithi148 2024-09-21T04:23:31.163 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:30 smithi148 ceph-mon[78701]: pgmap v233: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:30 smithi110 ceph-mon[92455]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-09-21T04:23:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:30 smithi110 ceph-mon[92455]: Deploying daemon node-exporter.smithi148 on smithi148 2024-09-21T04:23:31.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:30 smithi110 ceph-mon[92455]: pgmap v233: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:32 smithi110 ceph-mon[92455]: pgmap v234: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:32 smithi148 ceph-mon[78701]: pgmap v234: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:36.054 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:35 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:36.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:35 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:36.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:35 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:36.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:35 smithi148 ceph-mon[78701]: pgmap v235: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:36.097 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:35 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:36.097 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:35 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:36.097 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:35 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:36.097 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:35 smithi110 ceph-mon[92455]: pgmap v235: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:37 smithi110 ceph-mon[92455]: pgmap v236: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:37 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.962 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.962 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.962 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:37 smithi148 ceph-mon[78701]: pgmap v236: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:37.962 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:37.962 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:37 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:38.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:39 smithi148 ceph-mon[78701]: pgmap v237: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:39.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:40.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:40.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:40.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:39 smithi110 ceph-mon[92455]: pgmap v237: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:40.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:40.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:41.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:41.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:41 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:41.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:41 smithi110 ceph-mon[92455]: pgmap v238: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:41 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:41.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:41 smithi148 ceph-mon[78701]: pgmap v238: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:43.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.918 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.919 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:43 smithi110 ceph-mon[92455]: pgmap v239: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:43.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:43.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:43 smithi148 ceph-mon[78701]: pgmap v239: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:45.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:44 smithi148 ceph-mon[78701]: Upgrade: Updating prometheus.smithi110 2024-09-21T04:23:45.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:44 smithi148 ceph-mon[78701]: Deploying daemon prometheus.smithi110 on smithi110 2024-09-21T04:23:45.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:44 smithi148 ceph-mon[78701]: pgmap v240: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:44 smithi110 ceph-mon[92455]: Upgrade: Updating prometheus.smithi110 2024-09-21T04:23:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:44 smithi110 ceph-mon[92455]: Deploying daemon prometheus.smithi110 on smithi110 2024-09-21T04:23:45.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:44 smithi110 ceph-mon[92455]: pgmap v240: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:46 smithi110 ceph-mon[92455]: pgmap v241: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:46 smithi148 ceph-mon[78701]: pgmap v241: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:49.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:48 smithi110 ceph-mon[92455]: pgmap v242: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:49.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:48 smithi148 ceph-mon[78701]: pgmap v242: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:51.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:50 smithi110 ceph-mon[92455]: pgmap v243: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:51.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:50 smithi148 ceph-mon[78701]: pgmap v243: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:53.260 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:52 smithi110 ceph-mon[92455]: pgmap v244: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:53.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:52 smithi148 ceph-mon[78701]: pgmap v244: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:53.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 -- 172.21.15.110:0/2519673381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000feaa0 msgr2=0x7fda000feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 --2- 172.21.15.110:0/2519673381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000feaa0 0x7fda000feea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009c20 tx=0x7fd9f402a970 comp rx=0 tx=0).stop 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 -- 172.21.15.110:0/2519673381 shutdown_connections 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 --2- 172.21.15.110:0/2519673381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000ffca0 0x7fda00100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 --2- 172.21.15.110:0/2519673381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000feaa0 0x7fda000feea0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 -- 172.21.15.110:0/2519673381 >> 172.21.15.110:0/2519673381 conn(0x7fda000fa230 msgr2=0x7fda000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 -- 172.21.15.110:0/2519673381 shutdown_connections 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.808+0000 7fda074c7640 1 -- 172.21.15.110:0/2519673381 wait complete. 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 Processor -- start 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 -- start start 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 0x7fda00194540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda00195050 con 0x7fda000ffca0 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda074c7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001951c0 con 0x7fda000feaa0 2024-09-21T04:23:53.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda04a62640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda04a62640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58334/0 (socket says 172.21.15.110:58334) 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda04a62640 1 -- 172.21.15.110:0/2932481424 learned_addr learned my addr 172.21.15.110:0/2932481424 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.809+0000 7fda05263640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 0x7fda00194540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda04a62640 1 -- 172.21.15.110:0/2932481424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 msgr2=0x7fda00194540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda04a62640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 0x7fda00194540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda04a62640 1 -- 172.21.15.110:0/2932481424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4009900 con 0x7fda000ffca0 2024-09-21T04:23:53.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda05263640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 0x7fda00194540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:23:53.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda04a62640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007920 tx=0x7fd9fc007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:53.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc011070 con 0x7fda000ffca0 2024-09-21T04:23:53.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9fc00c2e0 con 0x7fda000ffca0 2024-09-21T04:23:53.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda00068460 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc00e9b0 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.810+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda00068a30 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.811+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda000feea0 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.812+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd9fc0046c0 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.812+0000 7fd9f27fc640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 0x7fd9ec07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.812+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fd9fc09fcd0 con 0x7fda000ffca0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.813+0000 7fda05263640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 0x7fd9ec07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:53.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.813+0000 7fda05263640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 0x7fd9ec07d730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402fd20 tx=0x7fd9f40093a0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:53.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:53.817+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd9fc062b10 con 0x7fda000ffca0 2024-09-21T04:23:54.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.018+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda0018e2a0 con 0x7fd9ec07b270 2024-09-21T04:23:54.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.019+0000 7fd9f27fc640 1 -- 172.21.15.110:0/2932481424 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7fda0018e2a0 con 0x7fd9ec07b270 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 msgr2=0x7fd9ec07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 0x7fd9ec07d730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402fd20 tx=0x7fd9f40093a0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 msgr2=0x7fda00194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007920 tx=0x7fd9fc007de0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 shutdown_connections 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd9ec07b270 0x7fd9ec07d730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda000ffca0 0x7fda00194a80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 --2- 172.21.15.110:0/2932481424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda000feaa0 0x7fda00194540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 >> 172.21.15.110:0/2932481424 conn(0x7fda000fa230 msgr2=0x7fda00103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 shutdown_connections 2024-09-21T04:23:54.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.022+0000 7fda074c7640 1 -- 172.21.15.110:0/2932481424 wait complete. 2024-09-21T04:23:54.033 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 -- 172.21.15.110:0/3812476313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000ffca0 msgr2=0x7fb000100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 --2- 172.21.15.110:0/3812476313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000ffca0 0x7fb000100120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faff4009880 tx=0x7faff402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 -- 172.21.15.110:0/3812476313 shutdown_connections 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 --2- 172.21.15.110:0/3812476313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000ffca0 0x7fb000100120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 --2- 172.21.15.110:0/3812476313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000feaa0 0x7fb0000feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 -- 172.21.15.110:0/3812476313 >> 172.21.15.110:0/3812476313 conn(0x7fb0000fa230 msgr2=0x7fb0000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 -- 172.21.15.110:0/3812476313 shutdown_connections 2024-09-21T04:23:54.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.105+0000 7fb005b6e640 1 -- 172.21.15.110:0/3812476313 wait complete. 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 Processor -- start 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 -- start start 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000ffca0 0x7fb000195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb000196140 con 0x7fb0000feaa0 2024-09-21T04:23:54.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fb005b6e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0001962b0 con 0x7fb0000ffca0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fafff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fafff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58338/0 (socket says 172.21.15.110:58338) 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.106+0000 7fafff7fe640 1 -- 172.21.15.110:0/572997688 learned_addr learned my addr 172.21.15.110:0/572997688 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7faffeffd640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000ffca0 0x7fb000195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fafff7fe640 1 -- 172.21.15.110:0/572997688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000ffca0 msgr2=0x7fb000195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fafff7fe640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000ffca0 0x7fb000195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fafff7fe640 1 -- 172.21.15.110:0/572997688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009560 con 0x7fb0000feaa0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fafff7fe640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faff000d720 tx=0x7faff000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff00041f0 con 0x7fb0000feaa0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faff0004350 con 0x7fb0000feaa0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00019ad50 con 0x7fb0000feaa0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff0003df0 con 0x7fb0000feaa0 2024-09-21T04:23:54.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.107+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00019b290 con 0x7fb0000feaa0 2024-09-21T04:23:54.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.108+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb000107690 con 0x7fb0000feaa0 2024-09-21T04:23:54.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.109+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7faff00035f0 con 0x7fb0000feaa0 2024-09-21T04:23:54.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.109+0000 7faffcff9640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 0x7fafe807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.110+0000 7faffeffd640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 0x7fafe807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.110+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7faff009deb0 con 0x7fb0000feaa0 2024-09-21T04:23:54.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.110+0000 7faffeffd640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 0x7fafe807d730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faff4000c00 tx=0x7faff40314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.114+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7faff0060d70 con 0x7fb0000feaa0 2024-09-21T04:23:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.316+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb000068bc0 con 0x7fafe807b270 2024-09-21T04:23:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.318+0000 7faffcff9640 1 -- 172.21.15.110:0/572997688 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7fb000068bc0 con 0x7fafe807b270 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 msgr2=0x7fafe807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 0x7fafe807d730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faff4000c00 tx=0x7faff40314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 msgr2=0x7fb000195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faff000d720 tx=0x7faff000dbe0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 shutdown_connections 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fafe807b270 0x7fafe807d730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0000ffca0 0x7fb000195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 --2- 172.21.15.110:0/572997688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000feaa0 0x7fb000195630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 >> 172.21.15.110:0/572997688 conn(0x7fb0000fa230 msgr2=0x7fb0000684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 shutdown_connections 2024-09-21T04:23:54.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.320+0000 7fb005b6e640 1 -- 172.21.15.110:0/572997688 wait complete. 2024-09-21T04:23:54.341 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.403+0000 7f1924d5b640 1 -- 172.21.15.110:0/632528901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 msgr2=0x7f19201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.403+0000 7f1924d5b640 1 --2- 172.21.15.110:0/632528901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f19201029b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19100098b0 tx=0x7f191002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 -- 172.21.15.110:0/632528901 shutdown_connections 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 --2- 172.21.15.110:0/632528901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 0x7f19201052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 --2- 172.21.15.110:0/632528901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f19201029b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 -- 172.21.15.110:0/632528901 >> 172.21.15.110:0/632528901 conn(0x7f19200fa1c0 msgr2=0x7f19200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 -- 172.21.15.110:0/632528901 shutdown_connections 2024-09-21T04:23:54.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.404+0000 7f1924d5b640 1 -- 172.21.15.110:0/632528901 wait complete. 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 Processor -- start 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 -- start start 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 0x7f1920195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920196190 con 0x7f19201005c0 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f1924d5b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920196300 con 0x7f1920102ef0 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58354/0 (socket says 172.21.15.110:58354) 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 -- 172.21.15.110:0/605955368 learned_addr learned my addr 172.21.15.110:0/605955368 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191dd9b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 0x7f1920195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 -- 172.21.15.110:0/605955368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 msgr2=0x7f1920195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 0x7f1920195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.405+0000 7f191e59c640 1 -- 172.21.15.110:0/605955368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009560 con 0x7f19201005c0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f191e59c640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19100042a0 tx=0x7f19100042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19100051d0 con 0x7f19201005c0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1910004b80 con 0x7f19201005c0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1910003b10 con 0x7f19201005c0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192019ad40 con 0x7f19201005c0 2024-09-21T04:23:54.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.406+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192019b210 con 0x7f19201005c0 2024-09-21T04:23:54.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.407+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1920108770 con 0x7f19201005c0 2024-09-21T04:23:54.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.408+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1910003d80 con 0x7f19201005c0 2024-09-21T04:23:54.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.408+0000 7f190f7fe640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 0x7f190807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.409+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f19100bf610 con 0x7f19201005c0 2024-09-21T04:23:54.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.409+0000 7f191dd9b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 0x7f190807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.409+0000 7f191dd9b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 0x7f190807d740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1914005fd0 tx=0x7f1914005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.413+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1910082450 con 0x7f19201005c0 2024-09-21T04:23:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.615+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1920103990 con 0x7f190807b280 2024-09-21T04:23:54.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.626+0000 7f190f7fe640 1 -- 172.21.15.110:0/605955368 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1920103990 con 0x7f190807b280 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (15m) 15s ago 17m 26.1M - 0.25.0 c8568f914cd2 1356b1142989 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (45s) 15s ago 17m 8242k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 9c05a31416c2 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (40s) 17s ago 16m 8102k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3ed27bd24a31 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 15s ago 17m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (5m) 17s ago 16m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (15m) 15s ago 16m 84.9M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (100s) 15s ago 11m 14.4M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (2m) 15s ago 11m 83.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (57s) 17s ago 11m 13.3M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 55fa666e3a5c 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (77s) 17s ago 11m 13.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (8m) 15s ago 18m 524M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:23:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (6m) 17s ago 16m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (6m) 15s ago 18m 57.1M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (6m) 17s ago 16m 45.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (26s) 15s ago 17m 3468k - 1.7.0 72c9c2088986 52919d9343b8 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (20s) 17s ago 16m 3338k - 1.7.0 72c9c2088986 091b9b7135aa 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 15s ago 14m 121M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 15s ago 14m 156M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 15s ago 13m 174M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (3m) 17s ago 13m 142M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (2m) 17s ago 13m 105M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (2m) 17s ago 12m 104M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:23:54.630 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 15s ago 16m 66.7M - 2.43.0 a07b618ecd1d 3b474300bf26 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 msgr2=0x7f190807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 0x7f190807d740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1914005fd0 tx=0x7f1914005850 comp rx=0 tx=0).stop 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 msgr2=0x7f1920195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19100042a0 tx=0x7f19100042d0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 shutdown_connections 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f190807b280 0x7f190807d740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1920102ef0 0x7f1920195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 --2- 172.21.15.110:0/605955368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19201005c0 0x7f1920195680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 >> 172.21.15.110:0/605955368 conn(0x7f19200fa1c0 msgr2=0x7f19200fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 shutdown_connections 2024-09-21T04:23:54.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.628+0000 7f1924d5b640 1 -- 172.21.15.110:0/605955368 wait complete. 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 -- 172.21.15.110:0/1346741654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40fea30 msgr2=0x7f5ab40fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/1346741654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40fea30 0x7f5ab40fee30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009880 tx=0x7f5aa402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 -- 172.21.15.110:0/1346741654 shutdown_connections 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/1346741654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40ffc30 0x7f5ab41000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/1346741654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40fea30 0x7f5ab40fee30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 -- 172.21.15.110:0/1346741654 >> 172.21.15.110:0/1346741654 conn(0x7f5ab40fa1c0 msgr2=0x7f5ab40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 -- 172.21.15.110:0/1346741654 shutdown_connections 2024-09-21T04:23:54.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.728+0000 7f5ab8e36640 1 -- 172.21.15.110:0/1346741654 wait complete. 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 Processor -- start 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 -- start start 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 0x7f5ab4194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4195040 con 0x7f5ab40ffc30 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab8e36640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab41951b0 con 0x7f5ab40fea30 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58382/0 (socket says 172.21.15.110:58382) 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 -- 172.21.15.110:0/4022089378 learned_addr learned my addr 172.21.15.110:0/4022089378 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab259c640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 0x7f5ab4194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 -- 172.21.15.110:0/4022089378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 msgr2=0x7f5ab4194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 0x7f5ab4194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab1d9b640 1 -- 172.21.15.110:0/4022089378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4009560 con 0x7f5ab40ffc30 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.729+0000 7f5ab259c640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 0x7f5ab4194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5ab1d9b640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800e9a0 tx=0x7f5aa800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa800cd50 con 0x7f5ab40ffc30 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab4199c50 con 0x7f5ab40ffc30 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5aa800ceb0 con 0x7f5ab40ffc30 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa8010660 con 0x7f5ab40ffc30 2024-09-21T04:23:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.730+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab419a220 con 0x7f5ab40ffc30 2024-09-21T04:23:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.731+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab4107640 con 0x7f5ab40ffc30 2024-09-21T04:23:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.732+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f5aa80108d0 con 0x7f5ab40ffc30 2024-09-21T04:23:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.732+0000 7f5aa37fe640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 0x7f5a9c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.732+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f5aa8014070 con 0x7f5ab40ffc30 2024-09-21T04:23:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.732+0000 7f5ab259c640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 0x7f5a9c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.733+0000 7f5ab259c640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 0x7f5a9c07d740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009230 tx=0x7f5aa40314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.737+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f5aa80618d0 con 0x7f5ab40ffc30 2024-09-21T04:23:55.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.997+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ab419a470 con 0x7f5ab40ffc30 2024-09-21T04:23:55.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:54.997+0000 7f5aa37fe640 1 -- 172.21.15.110:0/4022089378 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5ab419a470 con 0x7f5ab40ffc30 2024-09-21T04:23:55.000 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:23:55.000 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:23:55.000 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:23:55.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:23:55.002 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 msgr2=0x7f5a9c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 0x7f5a9c07d740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009230 tx=0x7f5aa40314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 msgr2=0x7f5ab4194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800e9a0 tx=0x7f5aa800eda0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 shutdown_connections 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5a9c07b280 0x7f5a9c07d740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4194a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 --2- 172.21.15.110:0/4022089378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab40fea30 0x7f5ab4194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 >> 172.21.15.110:0/4022089378 conn(0x7f5ab40fa1c0 msgr2=0x7f5ab4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 shutdown_connections 2024-09-21T04:23:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.000+0000 7f5ab8e36640 1 -- 172.21.15.110:0/4022089378 wait complete. 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 -- 172.21.15.110:0/2835986019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 msgr2=0x7f8bec102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 --2- 172.21.15.110:0/2835986019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec102a20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8be80098b0 tx=0x7f8be802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 -- 172.21.15.110:0/2835986019 shutdown_connections 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 --2- 172.21.15.110:0/2835986019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 0x7f8bec105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 --2- 172.21.15.110:0/2835986019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec102a20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 -- 172.21.15.110:0/2835986019 >> 172.21.15.110:0/2835986019 conn(0x7f8bec0fa230 msgr2=0x7f8bec0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 -- 172.21.15.110:0/2835986019 shutdown_connections 2024-09-21T04:23:55.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.082+0000 7f8bf372b640 1 -- 172.21.15.110:0/2835986019 wait complete. 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 Processor -- start 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 -- start start 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 0x7f8bec0fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0feb20 con 0x7f8bec100630 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf372b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0fec90 con 0x7f8bec102f60 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58396/0 (socket says 172.21.15.110:58396) 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 -- 172.21.15.110:0/677415308 learned_addr learned my addr 172.21.15.110:0/677415308 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf0cc6640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 0x7f8bec0fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 -- 172.21.15.110:0/677415308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 msgr2=0x7f8bec0fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 0x7f8bec0fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 -- 172.21.15.110:0/677415308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009560 con 0x7f8bec100630 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8bf14c7640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006010 tx=0x7f8be80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.083+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be8004aa0 con 0x7f8bec100630 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.084+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8be8005d50 con 0x7f8bec100630 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.084+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec0fef10 con 0x7f8bec100630 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.084+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be8003a20 con 0x7f8bec100630 2024-09-21T04:23:55.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.084+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec19d6c0 con 0x7f8bec100630 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.085+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec1087e0 con 0x7f8bec100630 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.085+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8be8031c00 con 0x7f8bec100630 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.086+0000 7f8be27fc640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 0x7f8bd807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.086+0000 7f8bf0cc6640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 0x7f8bd807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.086+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f8be80bf580 con 0x7f8bec100630 2024-09-21T04:23:55.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.086+0000 7f8bf0cc6640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 0x7f8bd807d730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ffd10 tx=0x7f8bdc005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.090+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8be8082440 con 0x7f8bec100630 2024-09-21T04:23:55.274 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:54 smithi110 ceph-mon[92455]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.274 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:54 smithi110 ceph-mon[92455]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.274 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:54 smithi110 ceph-mon[92455]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.274 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:54 smithi110 ceph-mon[92455]: pgmap v245: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.320+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8bec19d850 con 0x7f8bec100630 2024-09-21T04:23:55.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.321+0000 7f8be27fc640 1 -- 172.21.15.110:0/677415308 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1982 (secure 0 0 0) 0x7f8be8004c00 con 0x7f8bec100630 2024-09-21T04:23:55.324 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-09-21T04:23:55.324 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:57:935402+0000 2024-09-21T04:23:55.324 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 34 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:56.930115+0000 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:23:55.325 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 95 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:23:55.326 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:34404} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:23:55.327 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:55.328 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:55.328 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.323+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 msgr2=0x7f8bd807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.323+0000 7f8bf372b640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 0x7f8bd807d730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0ffd10 tx=0x7f8bdc005f90 comp rx=0 tx=0).stop 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.323+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 msgr2=0x7f8bec0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.323+0000 7f8bf372b640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006010 tx=0x7f8be80055a0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 shutdown_connections 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8bd807b270 0x7f8bd807d730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec102f60 0x7f8bec0fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 --2- 172.21.15.110:0/677415308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bec100630 0x7f8bec0fff90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 >> 172.21.15.110:0/677415308 conn(0x7f8bec0fa230 msgr2=0x7f8bec0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 shutdown_connections 2024-09-21T04:23:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.324+0000 7f8bf372b640 1 -- 172.21.15.110:0/677415308 wait complete. 2024-09-21T04:23:55.367 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:55 smithi148 ceph-mon[78701]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.367 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:55 smithi148 ceph-mon[78701]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.367 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:55 smithi148 ceph-mon[78701]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:55.367 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:55 smithi148 ceph-mon[78701]: pgmap v245: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:55.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 -- 172.21.15.110:0/1085599770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0ffc50 msgr2=0x7f08ec1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 --2- 172.21.15.110:0/1085599770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0ffc50 0x7f08ec1000d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009880 tx=0x7f08d402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 -- 172.21.15.110:0/1085599770 shutdown_connections 2024-09-21T04:23:55.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 --2- 172.21.15.110:0/1085599770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0ffc50 0x7f08ec1000d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 --2- 172.21.15.110:0/1085599770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0fea50 0x7f08ec0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.411+0000 7f08f20b3640 1 -- 172.21.15.110:0/1085599770 >> 172.21.15.110:0/1085599770 conn(0x7f08ec0fa1c0 msgr2=0x7f08ec0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.412+0000 7f08f20b3640 1 -- 172.21.15.110:0/1085599770 shutdown_connections 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.412+0000 7f08f20b3640 1 -- 172.21.15.110:0/1085599770 wait complete. 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.412+0000 7f08f20b3640 1 Processor -- start 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.412+0000 7f08f20b3640 1 -- start start 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.412+0000 7f08f20b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08f20b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0ffc50 0x7f08ec19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08f20b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec19a900 con 0x7f08ec0fea50 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08f20b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec19aa70 con 0x7f08ec0ffc50 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58424/0 (socket says 172.21.15.110:58424) 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 -- 172.21.15.110:0/397834383 learned_addr learned my addr 172.21.15.110:0/397834383 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eaffd640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0ffc50 0x7f08ec19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 -- 172.21.15.110:0/397834383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0ffc50 msgr2=0x7f08ec19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0ffc50 0x7f08ec19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 -- 172.21.15.110:0/397834383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4009560 con 0x7f08ec0fea50 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08eb7fe640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ea20 tx=0x7f08dc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08dc00cdb0 con 0x7f08ec0fea50 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec19ad50 con 0x7f08ec0fea50 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08dc003e30 con 0x7f08ec0fea50 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08dc0116a0 con 0x7f08ec0fea50 2024-09-21T04:23:55.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.413+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec19b290 con 0x7f08ec0fea50 2024-09-21T04:23:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.415+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec107660 con 0x7f08ec0fea50 2024-09-21T04:23:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.415+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f08dc003970 con 0x7f08ec0fea50 2024-09-21T04:23:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.416+0000 7f08e8ff9640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 0x7f08d807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.416+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f08dc09e870 con 0x7f08ec0fea50 2024-09-21T04:23:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.416+0000 7f08eaffd640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 0x7f08d807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.416+0000 7f08eaffd640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 0x7f08d807d740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.421+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f08dc0617e0 con 0x7f08ec0fea50 2024-09-21T04:23:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.635+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08ec19b530 con 0x7f08d807b280 2024-09-21T04:23:55.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.637+0000 7f08e8ff9640 1 -- 172.21.15.110:0/397834383 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f08ec19b530 con 0x7f08d807b280 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph-exporter", 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: "mds" 2024-09-21T04:23:55.640 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:23:55.641 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "18/23 daemons upgraded", 2024-09-21T04:23:55.641 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading prometheus daemons", 2024-09-21T04:23:55.641 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:23:55.641 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:23:55.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 msgr2=0x7f08d807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 0x7f08d807d740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40314e0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 msgr2=0x7f08ec068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ea20 tx=0x7f08dc00eee0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 shutdown_connections 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08d807b280 0x7f08d807d740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08ec0ffc50 0x7f08ec19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 --2- 172.21.15.110:0/397834383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08ec0fea50 0x7f08ec068fa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 >> 172.21.15.110:0/397834383 conn(0x7f08ec0fa1c0 msgr2=0x7f08ec102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 shutdown_connections 2024-09-21T04:23:55.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.639+0000 7f08f20b3640 1 -- 172.21.15.110:0/397834383 wait complete. 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.725+0000 7f7c0b014640 1 -- 172.21.15.110:0/71037818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c04100d60 msgr2=0x7f7c041011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.725+0000 7f7c0b014640 1 --2- 172.21.15.110:0/71037818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c04100d60 0x7f7c041011e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009880 tx=0x7f7bf002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 -- 172.21.15.110:0/71037818 shutdown_connections 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 --2- 172.21.15.110:0/71037818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c04100d60 0x7f7c041011e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 --2- 172.21.15.110:0/71037818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c040ffb60 0x7f7c040fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 -- 172.21.15.110:0/71037818 >> 172.21.15.110:0/71037818 conn(0x7f7c040fb330 msgr2=0x7f7c040fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:55.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 -- 172.21.15.110:0/71037818 shutdown_connections 2024-09-21T04:23:55.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 -- 172.21.15.110:0/71037818 wait complete. 2024-09-21T04:23:55.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 Processor -- start 2024-09-21T04:23:55.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 -- start start 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.726+0000 7f7c0b014640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 0x7f7c04195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c0b014640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 0x7f7c04195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c0b014640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04196120 con 0x7f7c040ffb60 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c0b014640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04196290 con 0x7f7c04100d60 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c08db0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 0x7f7c04195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c08db0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 0x7f7c04195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58434/0 (socket says 172.21.15.110:58434) 2024-09-21T04:23:55.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c08db0640 1 -- 172.21.15.110:0/1673851028 learned_addr learned my addr 172.21.15.110:0/1673851028 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c03fff640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 0x7f7c04195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c03fff640 1 -- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 msgr2=0x7f7c04195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c03fff640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 0x7f7c04195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c03fff640 1 -- 172.21.15.110:0/1673851028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0009560 con 0x7f7c04100d60 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.727+0000 7f7c03fff640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 0x7f7c04195b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7bf002a9c0 tx=0x7f7bf00054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.728+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0004aa0 con 0x7f7c04100d60 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.728+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0419acd0 con 0x7f7c04100d60 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.728+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf0004c00 con 0x7f7c04100d60 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.728+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0003a60 con 0x7f7c04100d60 2024-09-21T04:23:55.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.728+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0419b1c0 con 0x7f7c04100d60 2024-09-21T04:23:55.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.729+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0418f6e0 con 0x7f7c04100d60 2024-09-21T04:23:55.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.730+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f7bf0003cd0 con 0x7f7c04100d60 2024-09-21T04:23:55.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.731+0000 7f7c01ffb640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 0x7f7bf407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:23:55.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.731+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f7bf00bf590 con 0x7f7c04100d60 2024-09-21T04:23:55.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.731+0000 7f7c08db0640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 0x7f7bf407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:23:55.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.732+0000 7f7c08db0640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 0x7f7bf407d740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005fd0 tx=0x7f7bf800b4a0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:23:55.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:55.739+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f7bf0082500 con 0x7f7c04100d60 2024-09-21T04:23:56.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.008+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c04108a00 con 0x7f7c04100d60 2024-09-21T04:23:56.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.008+0000 7f7c01ffb640 1 -- 172.21.15.110:0/1673851028 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7bf0082500 con 0x7f7c04100d60 2024-09-21T04:23:56.011 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:23:56.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 msgr2=0x7f7bf407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 0x7f7bf407d740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005fd0 tx=0x7f7bf800b4a0 comp rx=0 tx=0).stop 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 msgr2=0x7f7c04195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 0x7f7c04195b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7bf002a9c0 tx=0x7f7bf00054f0 comp rx=0 tx=0).stop 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 shutdown_connections 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7bf407b280 0x7f7bf407d740 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c04100d60 0x7f7c04195b50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 --2- 172.21.15.110:0/1673851028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c040ffb60 0x7f7c04195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.010+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 >> 172.21.15.110:0/1673851028 conn(0x7f7c040fb330 msgr2=0x7f7c04104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.011+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 shutdown_connections 2024-09-21T04:23:56.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:23:56.011+0000 7f7c0b014640 1 -- 172.21.15.110:0/1673851028 wait complete. 2024-09-21T04:23:56.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:56 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4022089378' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:56.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:56 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/677415308' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:23:56.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:56 smithi110 ceph-mon[92455]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:56 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4022089378' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:23:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:56 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/677415308' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:23:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:56 smithi148 ceph-mon[78701]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:23:57.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:57 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1673851028' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:23:57.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:57.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:57.255 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:57 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:57.256 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:57 smithi110 ceph-mon[92455]: pgmap v246: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:57.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:57 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1673851028' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:23:57.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:57.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:23:57.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:57 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:23:57.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:57 smithi148 ceph-mon[78701]: pgmap v246: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:59.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:23:58 smithi110 ceph-mon[92455]: pgmap v247: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:23:59.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:23:58 smithi148 ceph-mon[78701]: pgmap v247: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:01.904 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:01.904 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:01.904 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:01 smithi110 ceph-mon[92455]: pgmap v248: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:01.904 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:01.904 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:01 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:01 smithi148 ceph-mon[78701]: pgmap v248: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:02.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:01 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:03.279 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:02 smithi110 ceph-mon[92455]: pgmap v249: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:02 smithi148 ceph-mon[78701]: pgmap v249: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:05.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:05.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:05.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.910 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.911 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:05 smithi110 ceph-mon[92455]: pgmap v250: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:05.943 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:05 smithi148 ceph-mon[78701]: pgmap v250: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:06 smithi110 ceph-mon[92455]: Upgrade: Updating alertmanager.smithi110 2024-09-21T04:24:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:06 smithi110 ceph-mon[92455]: Deploying daemon alertmanager.smithi110 on smithi110 2024-09-21T04:24:06.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:06 smithi148 ceph-mon[78701]: Upgrade: Updating alertmanager.smithi110 2024-09-21T04:24:06.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:06 smithi148 ceph-mon[78701]: Deploying daemon alertmanager.smithi110 on smithi110 2024-09-21T04:24:07.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:07 smithi110 ceph-mon[92455]: pgmap v251: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:07.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:07 smithi148 ceph-mon[78701]: pgmap v251: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:08.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:08.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:09.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:09 smithi110 ceph-mon[92455]: pgmap v252: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:09.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:09 smithi148 ceph-mon[78701]: pgmap v252: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:11.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:10 smithi110 ceph-mon[92455]: pgmap v253: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:11.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:10 smithi148 ceph-mon[78701]: pgmap v253: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:13.154 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:12 smithi110 ceph-mon[92455]: pgmap v254: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:12 smithi148 ceph-mon[78701]: pgmap v254: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:15.266 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:14 smithi148 ceph-mon[78701]: pgmap v255: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:15.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:14 smithi110 ceph-mon[92455]: pgmap v255: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:16.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:16 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:16.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:16 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:16.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:16 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:16.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:16 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:16.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:16 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:16.691 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:16 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:17.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:17 smithi110 ceph-mon[92455]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:17.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:17 smithi148 ceph-mon[78701]: pgmap v256: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:19.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:18 smithi110 ceph-mon[92455]: pgmap v257: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:18 smithi148 ceph-mon[78701]: pgmap v257: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:20 smithi110 ceph-mon[92455]: pgmap v258: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:21.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:20 smithi148 ceph-mon[78701]: pgmap v258: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:22.297 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.297 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.297 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.298 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:22 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:22.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:22 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:23.280 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:23 smithi110 ceph-mon[92455]: pgmap v259: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:23 smithi148 ceph-mon[78701]: pgmap v259: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:24.309 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:24.341 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:25.989 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:25.989 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:25.989 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:25.989 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.990 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:25.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:25 smithi148 ceph-mon[78701]: pgmap v260: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/2209945619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 msgr2=0x7fe9c0105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/2209945619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0105330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098b0 tx=0x7fe9b002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/2209945619 shutdown_connections 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/2209945619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0105330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/2209945619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 0x7fe9c0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/2209945619 >> 172.21.15.110:0/2209945619 conn(0x7fe9c00fa230 msgr2=0x7fe9c00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/2209945619 shutdown_connections 2024-09-21T04:24:26.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.097+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/2209945619 wait complete. 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:24:26.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:26.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:25 smithi110 ceph-mon[92455]: pgmap v260: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:26.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 Processor -- start 2024-09-21T04:24:26.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 -- start start 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 0x7fe9c0195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0196180 con 0x7fe9c0102f40 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c7f5e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c01962f0 con 0x7fe9c0100610 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39998/0 (socket says 172.21.15.110:39998) 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 -- 172.21.15.110:0/4223641861 learned_addr learned my addr 172.21.15.110:0/4223641861 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c5cfa640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 0x7fe9c0195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 -- 172.21.15.110:0/4223641861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 msgr2=0x7fe9c0195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 0x7fe9c0195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.098+0000 7fe9c54f9640 1 -- 172.21.15.110:0/4223641861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0009560 con 0x7fe9c0102f40 2024-09-21T04:24:26.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9c54f9640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00055a0 tx=0x7fe9b00055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9b0004aa0 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c019ad30 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9b0004c00 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9b0003b50 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.099+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c019b2a0 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.100+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0108790 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.101+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fe9b0003cb0 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.101+0000 7fe9b6ffd640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 0x7fe9ac07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.101+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe9b00bf6e0 con 0x7fe9c0102f40 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.101+0000 7fe9c5cfa640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 0x7fe9ac07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.102+0000 7fe9c5cfa640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 0x7fe9ac07d730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005fd0 tx=0x7fe9bc00b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.106+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe9b00825d0 con 0x7fe9c0102f40 2024-09-21T04:24:26.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.335+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9c0103960 con 0x7fe9ac07b270 2024-09-21T04:24:26.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.338+0000 7fe9b6ffd640 1 -- 172.21.15.110:0/4223641861 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fe9c0103960 con 0x7fe9ac07b270 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 msgr2=0x7fe9ac07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 0x7fe9ac07d730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005fd0 tx=0x7fe9bc00b390 comp rx=0 tx=0).stop 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 msgr2=0x7fe9c0195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00055a0 tx=0x7fe9b00055d0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 shutdown_connections 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe9ac07b270 0x7fe9ac07d730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9c0102f40 0x7fe9c0195bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 --2- 172.21.15.110:0/4223641861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c0100610 0x7fe9c0195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 >> 172.21.15.110:0/4223641861 conn(0x7fe9c00fa230 msgr2=0x7fe9c00fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 shutdown_connections 2024-09-21T04:24:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.341+0000 7fe9c7f5e640 1 -- 172.21.15.110:0/4223641861 wait complete. 2024-09-21T04:24:26.353 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 -- 172.21.15.110:0/3369571616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb000ffb60 msgr2=0x7fdb000fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 --2- 172.21.15.110:0/3369571616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb000ffb60 0x7fdb000fff60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009880 tx=0x7fdafc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 -- 172.21.15.110:0/3369571616 shutdown_connections 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 --2- 172.21.15.110:0/3369571616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb00100d60 0x7fdb001011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 --2- 172.21.15.110:0/3369571616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb000ffb60 0x7fdb000fff60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 -- 172.21.15.110:0/3369571616 >> 172.21.15.110:0/3369571616 conn(0x7fdb000fb330 msgr2=0x7fdb000fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 -- 172.21.15.110:0/3369571616 shutdown_connections 2024-09-21T04:24:26.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 -- 172.21.15.110:0/3369571616 wait complete. 2024-09-21T04:24:26.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.428+0000 7fdb07d24640 1 Processor -- start 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb07d24640 1 -- start start 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb07d24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb000ffb60 0x7fdb001955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb07d24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb07d24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb001960e0 con 0x7fdb00100d60 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb07d24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00196250 con 0x7fdb000ffb60 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40020/0 (socket says 172.21.15.110:40020) 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 -- 172.21.15.110:0/273310553 learned_addr learned my addr 172.21.15.110:0/273310553 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb05ac0640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb000ffb60 0x7fdb001955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 -- 172.21.15.110:0/273310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb000ffb60 msgr2=0x7fdb001955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb000ffb60 0x7fdb001955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 -- 172.21.15.110:0/273310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009490 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.429+0000 7fdb052bf640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000ea10 tx=0x7fdaf000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.430+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf000cd50 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.430+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc009560 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.430+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaf000ceb0 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.430+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf0004a60 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.430+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0019b130 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.431+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0018f6e0 con 0x7fdb00100d60 2024-09-21T04:24:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.432+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fdaf0004d20 con 0x7fdb00100d60 2024-09-21T04:24:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.432+0000 7fdaf6ffd640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 0x7fdaec07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.432+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fdaf0014070 con 0x7fdb00100d60 2024-09-21T04:24:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.432+0000 7fdb05ac0640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 0x7fdaec07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.433+0000 7fdb05ac0640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 0x7fdaec07d740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009230 tx=0x7fdafc0314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.436+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fdaf0061570 con 0x7fdb00100d60 2024-09-21T04:24:26.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.641+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb0002c4f0 con 0x7fdaec07b280 2024-09-21T04:24:26.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.655+0000 7fdaf6ffd640 1 -- 172.21.15.110:0/273310553 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fdb0002c4f0 con 0x7fdaec07b280 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 msgr2=0x7fdaec07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 0x7fdaec07d740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009230 tx=0x7fdafc0314e0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 msgr2=0x7fdb00195b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000ea10 tx=0x7fdaf000eed0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 shutdown_connections 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdaec07b280 0x7fdaec07d740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb00100d60 0x7fdb00195b10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 --2- 172.21.15.110:0/273310553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb000ffb60 0x7fdb001955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 >> 172.21.15.110:0/273310553 conn(0x7fdb000fb330 msgr2=0x7fdb001044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 shutdown_connections 2024-09-21T04:24:26.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.657+0000 7fdb07d24640 1 -- 172.21.15.110:0/273310553 wait complete. 2024-09-21T04:24:26.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 -- 172.21.15.110:0/3303653905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8100620 msgr2=0x7f76e8102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 --2- 172.21.15.110:0/3303653905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8100620 0x7f76e8102a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76e40097e0 tx=0x7f76e402a3f0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 -- 172.21.15.110:0/3303653905 shutdown_connections 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 --2- 172.21.15.110:0/3303653905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8102f50 0x7f76e8105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 --2- 172.21.15.110:0/3303653905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8100620 0x7f76e8102a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 -- 172.21.15.110:0/3303653905 >> 172.21.15.110:0/3303653905 conn(0x7f76e80fa1c0 msgr2=0x7f76e80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 -- 172.21.15.110:0/3303653905 shutdown_connections 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 -- 172.21.15.110:0/3303653905 wait complete. 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.755+0000 7f76efc66640 1 Processor -- start 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76efc66640 1 -- start start 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76efc66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76efc66640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 0x7f76e80fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76efc66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80feb10 con 0x7f76e8100620 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76efc66640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80fec80 con 0x7f76e8102f50 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40036/0 (socket says 172.21.15.110:40036) 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 -- 172.21.15.110:0/3724752908 learned_addr learned my addr 172.21.15.110:0/3724752908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76ed201640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 0x7f76e80fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 -- 172.21.15.110:0/3724752908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 msgr2=0x7f76e80fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 0x7f76e80fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 -- 172.21.15.110:0/3724752908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc009560 con 0x7f76e8100620 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76ed201640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 0x7f76e80fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.756+0000 7f76eda02640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e4030c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.757+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e40037c0 con 0x7f76e8100620 2024-09-21T04:24:26.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.757+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76e40044f0 con 0x7f76e8100620 2024-09-21T04:24:26.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.757+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e4031d80 con 0x7f76e8100620 2024-09-21T04:24:26.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.757+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e4009490 con 0x7f76e8100620 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.757+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e80ff320 con 0x7f76e8100620 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.758+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e81086e0 con 0x7f76e8100620 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.759+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f76e4031400 con 0x7f76e8100620 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.759+0000 7f76daffd640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 0x7f76d407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.759+0000 7f76ed201640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 0x7f76d407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:26.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.759+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f76e40bfa50 con 0x7f76e8100620 2024-09-21T04:24:26.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.760+0000 7f76ed201640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 0x7f76d407d740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ffd00 tx=0x7f76dc009260 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:26.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.764+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f76e40828b0 con 0x7f76e8100620 2024-09-21T04:24:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.958+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e8103930 con 0x7f76d407b280 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.968+0000 7f76daffd640 1 -- 172.21.15.110:0/3724752908 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f76e8103930 con 0x7f76d407b280 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (12s) 5s ago 17m 17.7M - 0.27.0 11f11916f8cd 33ba9a4a9ab1 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (77s) 5s ago 17m 8418k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 9c05a31416c2 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (72s) 50s ago 16m 8102k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3ed27bd24a31 2024-09-21T04:24:26.971 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 5s ago 17m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (6m) 50s ago 16m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (15m) 5s ago 17m 85.5M - 9.4.7 954c08fa6188 284ac8ab462c 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (2m) 5s ago 11m 14.7M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (2m) 5s ago 11m 83.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (89s) 50s ago 11m 13.3M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 55fa666e3a5c 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (109s) 50s ago 11m 13.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (8m) 5s ago 18m 526M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (7m) 50s ago 16m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 5s ago 19m 58.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (6m) 50s ago 16m 45.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (58s) 5s ago 17m 9.98M - 1.7.0 72c9c2088986 52919d9343b8 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (52s) 50s ago 16m 3338k - 1.7.0 72c9c2088986 091b9b7135aa 2024-09-21T04:24:26.972 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 5s ago 15m 121M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 5s ago 14m 156M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 5s ago 14m 175M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (3m) 50s ago 14m 142M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (3m) 50s ago 13m 105M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (2m) 50s ago 13m 104M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (30s) 5s ago 17m 43.4M - 2.51.0 1d3b7f56885b 1b120422a054 2024-09-21T04:24:26.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 msgr2=0x7f76d407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 0x7f76d407d740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ffd00 tx=0x7f76dc009260 comp rx=0 tx=0).stop 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 msgr2=0x7f76e80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e4030c00 comp rx=0 tx=0).stop 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 shutdown_connections 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f76d407b280 0x7f76d407d740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8102f50 0x7f76e80fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 --2- 172.21.15.110:0/3724752908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8100620 0x7f76e80fff80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 >> 172.21.15.110:0/3724752908 conn(0x7f76e80fa1c0 msgr2=0x7f76e80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 shutdown_connections 2024-09-21T04:24:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:26.970+0000 7f76efc66640 1 -- 172.21.15.110:0/3724752908 wait complete. 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3327055785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 msgr2=0x7f9ac4105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3327055785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac4105380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009880 tx=0x7f9ab802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3327055785 shutdown_connections 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3327055785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac4105380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3327055785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3327055785 >> 172.21.15.110:0/3327055785 conn(0x7f9ac40fa1c0 msgr2=0x7f9ac40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3327055785 shutdown_connections 2024-09-21T04:24:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.052+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3327055785 wait complete. 2024-09-21T04:24:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 Processor -- start 2024-09-21T04:24:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 -- start start 2024-09-21T04:24:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac40fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40feb50 con 0x7f9ac4102f90 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9aca7a0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40fecc0 con 0x7f9ac4100660 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40050/0 (socket says 172.21.15.110:40050) 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 -- 172.21.15.110:0/3071301968 learned_addr learned my addr 172.21.15.110:0/3071301968 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac3fff640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac40fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 -- 172.21.15.110:0/3071301968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 msgr2=0x7f9ac40fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac40fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac37fe640 1 -- 172.21.15.110:0/3071301968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009560 con 0x7f9ac4102f90 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.053+0000 7f9ac3fff640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9ac37fe640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ab802a9c0 tx=0x7f9ab8005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab8004b20 con 0x7f9ac4102f90 2024-09-21T04:24:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab8004c80 con 0x7f9ac4102f90 2024-09-21T04:24:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac40fef40 con 0x7f9ac4102f90 2024-09-21T04:24:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab8003b80 con 0x7f9ac4102f90 2024-09-21T04:24:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.054+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac419d670 con 0x7f9ac4102f90 2024-09-21T04:24:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.055+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac4100a40 con 0x7f9ac4102f90 2024-09-21T04:24:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.055+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f9ab8031640 con 0x7f9ac4102f90 2024-09-21T04:24:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.056+0000 7f9ac17fa640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 0x7f9aac07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.056+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f9ab80bf8d0 con 0x7f9ac4102f90 2024-09-21T04:24:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.056+0000 7f9ac3fff640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 0x7f9aac07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.056+0000 7f9ac3fff640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 0x7f9aac07d740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40058c0 tx=0x7f9ab4005d30 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.060+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f9ab8082710 con 0x7f9ac4102f90 2024-09-21T04:24:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.309+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ac419d900 con 0x7f9ac4102f90 2024-09-21T04:24:27.312 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:26 smithi110 ceph-mon[92455]: Upgrade: Updating grafana.smithi110 2024-09-21T04:24:27.312 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:26 smithi110 ceph-mon[92455]: Deploying daemon grafana.smithi110 on smithi110 2024-09-21T04:24:27.313 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:26 smithi110 ceph-mon[92455]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:27.313 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:26 smithi110 ceph-mon[92455]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:27.313 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:26 smithi110 ceph-mon[92455]: pgmap v261: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.310+0000 7f9ac17fa640 1 -- 172.21.15.110:0/3071301968 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9ab8085830 con 0x7f9ac4102f90 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:24:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:24:27.314 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 msgr2=0x7f9aac07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 0x7f9aac07d740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40058c0 tx=0x7f9ab4005d30 comp rx=0 tx=0).stop 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 msgr2=0x7f9ac40fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ab802a9c0 tx=0x7f9ab8005610 comp rx=0 tx=0).stop 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 shutdown_connections 2024-09-21T04:24:27.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f9aac07b280 0x7f9aac07d740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac4102f90 0x7f9ac40fe610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 --2- 172.21.15.110:0/3071301968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac4100660 0x7f9ac40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 >> 172.21.15.110:0/3071301968 conn(0x7f9ac40fa1c0 msgr2=0x7f9ac40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 shutdown_connections 2024-09-21T04:24:27.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.312+0000 7f9aca7a0640 1 -- 172.21.15.110:0/3071301968 wait complete. 2024-09-21T04:24:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 -- 172.21.15.110:0/3863644683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928100630 msgr2=0x7f5928102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 --2- 172.21.15.110:0/3863644683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928100630 0x7f5928102a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f59180098b0 tx=0x7f591802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 -- 172.21.15.110:0/3863644683 shutdown_connections 2024-09-21T04:24:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 --2- 172.21.15.110:0/3863644683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928102f60 0x7f5928105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 --2- 172.21.15.110:0/3863644683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928100630 0x7f5928102a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 -- 172.21.15.110:0/3863644683 >> 172.21.15.110:0/3863644683 conn(0x7f59280fa230 msgr2=0x7f59280fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 -- 172.21.15.110:0/3863644683 shutdown_connections 2024-09-21T04:24:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.396+0000 7f592d5b6640 1 -- 172.21.15.110:0/3863644683 wait complete. 2024-09-21T04:24:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 Processor -- start 2024-09-21T04:24:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 -- start start 2024-09-21T04:24:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928100630 0x7f5928195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928196290 con 0x7f5928102f60 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f592d5b6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928196400 con 0x7f5928100630 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40068/0 (socket says 172.21.15.110:40068) 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 -- 172.21.15.110:0/2649315099 learned_addr learned my addr 172.21.15.110:0/2649315099 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f5926ffd640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928100630 0x7f5928195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 -- 172.21.15.110:0/2649315099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928100630 msgr2=0x7f5928195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928100630 0x7f5928195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.397+0000 7f59267fc640 1 -- 172.21.15.110:0/2649315099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5918009560 con 0x7f5928102f60 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f59267fc640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b400 tx=0x7f591c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591c011800 con 0x7f5928102f60 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f591c011e20 con 0x7f5928102f60 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592819ae50 con 0x7f5928102f60 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591c00f2e0 con 0x7f5928102f60 2024-09-21T04:24:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.398+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592819b3f0 con 0x7f5928102f60 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.399+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928108830 con 0x7f5928102f60 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.400+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f591c010380 con 0x7f5928102f60 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.400+0000 7f590ffff640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 0x7f591007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.400+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f591c09dfd0 con 0x7f5928102f60 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.400+0000 7f5926ffd640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 0x7f591007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.401+0000 7f5926ffd640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 0x7f591007d730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5918006010 tx=0x7f59180314e0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.405+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f591c060e10 con 0x7f5928102f60 2024-09-21T04:24:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:26 smithi148 ceph-mon[78701]: Upgrade: Updating grafana.smithi110 2024-09-21T04:24:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:26 smithi148 ceph-mon[78701]: Deploying daemon grafana.smithi110 on smithi110 2024-09-21T04:24:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:26 smithi148 ceph-mon[78701]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:27.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:26 smithi148 ceph-mon[78701]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:27.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:26 smithi148 ceph-mon[78701]: pgmap v261: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:27.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.630+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5928108a10 con 0x7f5928102f60 2024-09-21T04:24:27.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.631+0000 7f590ffff640 1 -- 172.21.15.110:0/2649315099 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1982 (secure 0 0 0) 0x7f591c063f30 con 0x7f5928102f60 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:57:935402+0000 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:24:27.634 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 34 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:56.930115+0000 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 95 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:24:27.635 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:34404} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:24:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:24:27.637 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 msgr2=0x7f591007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 0x7f591007d730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5918006010 tx=0x7f59180314e0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 msgr2=0x7f5928195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b400 tx=0x7f591c00b8c0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 shutdown_connections 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f591007b270 0x7f591007d730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5928102f60 0x7f5928195cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 --2- 172.21.15.110:0/2649315099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5928100630 0x7f5928195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 >> 172.21.15.110:0/2649315099 conn(0x7f59280fa230 msgr2=0x7f59280fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 shutdown_connections 2024-09-21T04:24:27.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.634+0000 7f592d5b6640 1 -- 172.21.15.110:0/2649315099 wait complete. 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 -- 172.21.15.110:0/2097269044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 msgr2=0x7f241c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 --2- 172.21.15.110:0/2097269044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c0fee30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2418009880 tx=0x7f241802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 -- 172.21.15.110:0/2097269044 shutdown_connections 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 --2- 172.21.15.110:0/2097269044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 0x7f241c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 --2- 172.21.15.110:0/2097269044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c0fee30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 -- 172.21.15.110:0/2097269044 >> 172.21.15.110:0/2097269044 conn(0x7f241c0fa1c0 msgr2=0x7f241c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 -- 172.21.15.110:0/2097269044 shutdown_connections 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 -- 172.21.15.110:0/2097269044 wait complete. 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.721+0000 7f24233bf640 1 Processor -- start 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f24233bf640 1 -- start start 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f24233bf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f24233bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 0x7f241c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f24233bf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c196150 con 0x7f241c0fea30 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f24233bf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1962c0 con 0x7f241c0ffc30 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40096/0 (socket says 172.21.15.110:40096) 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 -- 172.21.15.110:0/927824746 learned_addr learned my addr 172.21.15.110:0/927824746 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242095a640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 0x7f241c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 -- 172.21.15.110:0/927824746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 msgr2=0x7f241c195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 0x7f241c195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 -- 172.21.15.110:0/927824746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009560 con 0x7f241c0fea30 2024-09-21T04:24:27.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.722+0000 7f242115b640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2418004650 tx=0x7f24180055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.723+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2418004aa0 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.723+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2418005d50 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.723+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c19ad00 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.723+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24180039e0 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.723+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c19b220 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.724+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c107640 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.725+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f2418031c00 con 0x7f241c0fea30 2024-09-21T04:24:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.725+0000 7f24127fc640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 0x7f240807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.725+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f24180bf580 con 0x7f241c0fea30 2024-09-21T04:24:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.725+0000 7f242095a640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 0x7f240807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.726+0000 7f242095a640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 0x7f240807d740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f241c196b60 tx=0x7f240c005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:27.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.730+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f2418082440 con 0x7f241c0fea30 2024-09-21T04:24:27.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.936+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f241c19b460 con 0x7f240807b280 2024-09-21T04:24:27.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.938+0000 7f24127fc640 1 -- 172.21.15.110:0/927824746 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f241c19b460 con 0x7f240807b280 2024-09-21T04:24:27.940 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph-exporter", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "mds" 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "18/23 daemons upgraded", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading grafana daemons", 2024-09-21T04:24:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:24:27.942 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 msgr2=0x7f240807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 0x7f240807d740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f241c196b60 tx=0x7f240c005f90 comp rx=0 tx=0).stop 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 msgr2=0x7f241c195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2418004650 tx=0x7f24180055a0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 shutdown_connections 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f240807b280 0x7f240807d740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241c0ffc30 0x7f241c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 --2- 172.21.15.110:0/927824746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f241c0fea30 0x7f241c195640 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 >> 172.21.15.110:0/927824746 conn(0x7f241c0fa1c0 msgr2=0x7f241c067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:27.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 shutdown_connections 2024-09-21T04:24:27.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:27.940+0000 7f24233bf640 1 -- 172.21.15.110:0/927824746 wait complete. 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 -- 172.21.15.110:0/464615527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 msgr2=0x7fdd781011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/464615527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd781011e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009880 tx=0x7fdd6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 -- 172.21.15.110:0/464615527 shutdown_connections 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/464615527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd781011e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/464615527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 0x7fdd780fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 -- 172.21.15.110:0/464615527 >> 172.21.15.110:0/464615527 conn(0x7fdd780fb2f0 msgr2=0x7fdd780fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:28.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 -- 172.21.15.110:0/464615527 shutdown_connections 2024-09-21T04:24:28.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.023+0000 7fdd7dc46640 1 -- 172.21.15.110:0/464615527 wait complete. 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 Processor -- start 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 -- start start 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 0x7fdd78195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78196130 con 0x7fdd78100d60 2024-09-21T04:24:28.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd7dc46640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781962a0 con 0x7fdd780ffb60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40106/0 (socket says 172.21.15.110:40106) 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.024+0000 7fdd76ffd640 1 -- 172.21.15.110:0/1249804767 learned_addr learned my addr 172.21.15.110:0/1249804767 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd777fe640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 0x7fdd78195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd76ffd640 1 -- 172.21.15.110:0/1249804767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 msgr2=0x7fdd78195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd76ffd640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 0x7fdd78195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd76ffd640 1 -- 172.21.15.110:0/1249804767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009560 con 0x7fdd78100d60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd76ffd640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005520 tx=0x7fdd6c005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c004aa0 con 0x7fdd78100d60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd6c004c00 con 0x7fdd78100d60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7819ac90 con 0x7fdd78100d60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c003b10 con 0x7fdd78100d60 2024-09-21T04:24:28.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.025+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7819b200 con 0x7fdd78100d60 2024-09-21T04:24:28.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.026+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7818f6e0 con 0x7fdd78100d60 2024-09-21T04:24:28.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.027+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fdd6c031d20 con 0x7fdd78100d60 2024-09-21T04:24:28.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.027+0000 7fdd74ff9640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 0x7fdd6007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:28.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.027+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fdd6c0bf740 con 0x7fdd78100d60 2024-09-21T04:24:28.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.027+0000 7fdd777fe640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 0x7fdd6007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:28.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.028+0000 7fdd777fe640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 0x7fdd6007d740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005fd0 tx=0x7fdd68005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:28.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.032+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fdd6c082580 con 0x7fdd78100d60 2024-09-21T04:24:28.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.296+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdd7802c480 con 0x7fdd78100d60 2024-09-21T04:24:28.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:27 smithi110 ceph-mon[92455]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:28.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:27 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3071301968' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:28.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:27 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2649315099' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:24:28.300 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:27 smithi110 ceph-mon[92455]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:28.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.297+0000 7fdd74ff9640 1 -- 172.21.15.110:0/1249804767 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdd6c0856a0 con 0x7fdd78100d60 2024-09-21T04:24:28.300 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:24:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 msgr2=0x7fdd6007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 0x7fdd6007d740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005fd0 tx=0x7fdd68005850 comp rx=0 tx=0).stop 2024-09-21T04:24:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 msgr2=0x7fdd78195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005520 tx=0x7fdd6c005550 comp rx=0 tx=0).stop 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 shutdown_connections 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fdd6007b280 0x7fdd6007d740 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78100d60 0x7fdd78195b60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 --2- 172.21.15.110:0/1249804767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd780ffb60 0x7fdd78195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 >> 172.21.15.110:0/1249804767 conn(0x7fdd780fb2f0 msgr2=0x7fdd781044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 shutdown_connections 2024-09-21T04:24:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:28.299+0000 7fdd7dc46640 1 -- 172.21.15.110:0/1249804767 wait complete. 2024-09-21T04:24:28.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:27 smithi148 ceph-mon[78701]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:28.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:27 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3071301968' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:28.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:27 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2649315099' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:24:28.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:27 smithi148 ceph-mon[78701]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:29 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1249804767' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:24:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:29 smithi110 ceph-mon[92455]: pgmap v262: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:29 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1249804767' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:24:29.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:29 smithi148 ceph-mon[78701]: pgmap v262: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:31.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:30 smithi110 ceph-mon[92455]: pgmap v263: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:31.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:30 smithi148 ceph-mon[78701]: pgmap v263: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:32 smithi110 ceph-mon[92455]: pgmap v264: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:33 smithi148 ceph-mon[78701]: pgmap v264: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:35.151 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:34 smithi110 ceph-mon[92455]: pgmap v265: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:35.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:34 smithi148 ceph-mon[78701]: pgmap v265: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:37.295 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:36 smithi110 ceph-mon[92455]: pgmap v266: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:37.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:36 smithi148 ceph-mon[78701]: pgmap v266: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:39.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:39.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:39.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:40.150 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:39 smithi110 ceph-mon[92455]: pgmap v267: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:40.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:39 smithi148 ceph-mon[78701]: pgmap v267: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:41.391 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:40 smithi110 ceph-mon[92455]: pgmap v268: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:41.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:41 smithi148 ceph-mon[78701]: pgmap v268: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:43.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:43 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.272 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:43 smithi110 ceph-mon[92455]: pgmap v269: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:43.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:43 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:43.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:43 smithi148 ceph-mon[78701]: pgmap v269: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:45.410 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:44 smithi110 ceph-mon[92455]: pgmap v270: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:45.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:45 smithi148 ceph-mon[78701]: pgmap v270: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.170 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.171 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: Upgrade: Finalizing container_image settings 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T04:24:47.172 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-21T04:24:47.173 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: Upgrade: Complete! 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T04:24:47.174 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.175 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.192 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: Upgrade: Finalizing container_image settings 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-21T04:24:47.193 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.194 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: Upgrade: Complete! 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:24:47.195 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:24:47.196 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:48.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:47 smithi110 ceph-mon[92455]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:48.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:47 smithi148 ceph-mon[78701]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:49 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:49.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:49 smithi110 ceph-mon[92455]: pgmap v272: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:49 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:24:49.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:49 smithi148 ceph-mon[78701]: pgmap v272: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:50 smithi148 ceph-mon[78701]: pgmap v273: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:51.303 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:50 smithi110 ceph-mon[92455]: pgmap v273: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:53.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:53 smithi110 ceph-mon[92455]: pgmap v274: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:53.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:53 smithi148 ceph-mon[78701]: pgmap v274: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:54.341 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:54 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:54.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:54 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:24:55.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:55 smithi110 ceph-mon[92455]: pgmap v275: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:55.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:55 smithi148 ceph-mon[78701]: pgmap v275: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:57.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:57 smithi110 ceph-mon[92455]: pgmap v276: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:57.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:57 smithi148 ceph-mon[78701]: pgmap v276: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:58.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.395+0000 7f69bce29640 1 -- 172.21.15.110:0/3796091482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8074460 msgr2=0x7f69b80728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:58.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.395+0000 7f69bce29640 1 --2- 172.21.15.110:0/3796091482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8074460 0x7f69b80728c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009880 tx=0x7f69a802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 -- 172.21.15.110:0/3796091482 shutdown_connections 2024-09-21T04:24:58.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 --2- 172.21.15.110:0/3796091482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8072e00 0x7f69b8073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 --2- 172.21.15.110:0/3796091482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8074460 0x7f69b80728c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 -- 172.21.15.110:0/3796091482 >> 172.21.15.110:0/3796091482 conn(0x7f69b80fb320 msgr2=0x7f69b80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 -- 172.21.15.110:0/3796091482 shutdown_connections 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 -- 172.21.15.110:0/3796091482 wait complete. 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.396+0000 7f69bce29640 1 Processor -- start 2024-09-21T04:24:58.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69bce29640 1 -- start start 2024-09-21T04:24:58.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69bce29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:58.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69bce29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8074460 0x7f69b8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:58.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69bce29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b8198330 con 0x7f69b8072e00 2024-09-21T04:24:58.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69bce29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81984a0 con 0x7f69b8074460 2024-09-21T04:24:58.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35096/0 (socket says 172.21.15.110:35096) 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 -- 172.21.15.110:0/582327374 learned_addr learned my addr 172.21.15.110:0/582327374 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b5d9b640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8074460 0x7f69b8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 -- 172.21.15.110:0/582327374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8074460 msgr2=0x7f69b8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8074460 0x7f69b8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.397+0000 7f69b659c640 1 -- 172.21.15.110:0/582327374 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8009560 con 0x7f69b8072e00 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69b659c640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f69a80055a0 tx=0x7f69a80055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a8004aa0 con 0x7f69b8072e00 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69a8004c00 con 0x7f69b8072e00 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b819ce90 con 0x7f69b8072e00 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a8003b10 con 0x7f69b8072e00 2024-09-21T04:24:58.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.398+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b819d380 con 0x7f69b8072e00 2024-09-21T04:24:58.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.399+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b8191910 con 0x7f69b8072e00 2024-09-21T04:24:58.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.399+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f69a8031d20 con 0x7f69b8072e00 2024-09-21T04:24:58.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.400+0000 7f69a77fe640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 0x7f69a007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:24:58.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.400+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f69a80bf620 con 0x7f69b8072e00 2024-09-21T04:24:58.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.400+0000 7f69b5d9b640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 0x7f69a007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:24:58.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.401+0000 7f69b5d9b640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 0x7f69a007d730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69b8198cf0 tx=0x7f69ac005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:24:58.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.405+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f69a80825d0 con 0x7f69b8072e00 2024-09-21T04:24:58.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.598+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69b802c550 con 0x7f69a007b270 2024-09-21T04:24:58.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.599+0000 7f69a77fe640 1 -- 172.21.15.110:0/582327374 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f69b802c550 con 0x7f69a007b270 2024-09-21T04:24:58.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.601+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 msgr2=0x7f69a007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.601+0000 7f69bce29640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 0x7f69a007d730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69b8198cf0 tx=0x7f69ac005f90 comp rx=0 tx=0).stop 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.601+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 msgr2=0x7f69b8197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.601+0000 7f69bce29640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f69a80055a0 tx=0x7f69a80055d0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 shutdown_connections 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f69a007b270 0x7f69a007d730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b8074460 0x7f69b8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 --2- 172.21.15.110:0/582327374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b8072e00 0x7f69b8197820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 >> 172.21.15.110:0/582327374 conn(0x7f69b80fb320 msgr2=0x7f69b8105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 shutdown_connections 2024-09-21T04:24:58.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:24:58.602+0000 7f69bce29640 1 -- 172.21.15.110:0/582327374 wait complete. 2024-09-21T04:24:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:59 smithi110 ceph-mon[92455]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:24:59 smithi110 ceph-mon[92455]: pgmap v277: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:59.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch ps' 2024-09-21T04:24:59.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:59 smithi148 ceph-mon[78701]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:24:59.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:24:59 smithi148 ceph-mon[78701]: pgmap v277: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:24:59.551 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:01.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 -- 172.21.15.110:0/1390283501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 msgr2=0x7f78b40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:01.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/1390283501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b40fffa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c20 tx=0x7f78a802a970 comp rx=0 tx=0).stop 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 -- 172.21.15.110:0/1390283501 shutdown_connections 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/1390283501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b40fffa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/1390283501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 0x7f78b4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 -- 172.21.15.110:0/1390283501 >> 172.21.15.110:0/1390283501 conn(0x7f78b40fb840 msgr2=0x7f78b40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 -- 172.21.15.110:0/1390283501 shutdown_connections 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.163+0000 7f78bc3c7640 1 -- 172.21.15.110:0/1390283501 wait complete. 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 Processor -- start 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 -- start start 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 0x7f78b419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b419c370 con 0x7f78b40ffb40 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4195950 con 0x7f78b4105bf0 2024-09-21T04:25:01.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35120/0 (socket says 172.21.15.110:35120) 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 -- 172.21.15.110:0/342503417 learned_addr learned my addr 172.21.15.110:0/342503417 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78b9962640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 0x7f78b419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 -- 172.21.15.110:0/342503417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 msgr2=0x7f78b419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 0x7f78b419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 -- 172.21.15.110:0/342503417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009900 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78ba163640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e970 tx=0x7f78ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78ac00cc90 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78ac00cdf0 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b4195c30 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78ac010430 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.164+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b4196200 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.166+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b41011f0 con 0x7f78b40ffb40 2024-09-21T04:25:01.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.167+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f78ac0046c0 con 0x7f78b40ffb40 2024-09-21T04:25:01.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.167+0000 7f78b37fe640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 0x7f789c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:01.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.167+0000 7f78b9962640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 0x7f789c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:01.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.167+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f78ac014070 con 0x7f78b40ffb40 2024-09-21T04:25:01.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.167+0000 7f78b9962640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 0x7f789c07d730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f78b4196d70 tx=0x7f78a803d040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:01.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.171+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f78ac061900 con 0x7f78b40ffb40 2024-09-21T04:25:01.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.361+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b4108080 con 0x7f789c07b270 2024-09-21T04:25:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:01 smithi110 ceph-mon[92455]: pgmap v278: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:01.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.370+0000 7f78b37fe640 1 -- 172.21.15.110:0/342503417 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f78b4108080 con 0x7f789c07b270 2024-09-21T04:25:01.372 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (46s) 19s ago 18m 17.7M - 0.27.0 11f11916f8cd 33ba9a4a9ab1 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (112s) 19s ago 18m 8451k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 9c05a31416c2 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (107s) 84s ago 17m 8102k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3ed27bd24a31 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 19s ago 18m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (6m) 84s ago 17m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (23s) 19s ago 17m 45.8M - 10.4.8 5aad1d7cf004 828a5abe7bd9 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (2m) 19s ago 12m 14.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (3m) 19s ago 12m 83.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (2m) 84s ago 12m 13.3M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 55fa666e3a5c 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (2m) 84s ago 12m 13.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (9m) 19s ago 19m 533M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (7m) 84s ago 17m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 19s ago 19m 58.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:25:01.373 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (7m) 84s ago 17m 45.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (93s) 19s ago 18m 9.98M - 1.7.0 72c9c2088986 52919d9343b8 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (87s) 84s ago 17m 3338k - 1.7.0 72c9c2088986 091b9b7135aa 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 19s ago 15m 121M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 19s ago 15m 156M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 19s ago 14m 175M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (4m) 84s ago 14m 142M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (3m) 84s ago 14m 105M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (3m) 84s ago 13m 104M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:25:01.374 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (65s) 19s ago 17m 43.8M - 2.51.0 1d3b7f56885b 1b120422a054 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 msgr2=0x7f789c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 0x7f789c07d730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f78b4196d70 tx=0x7f78a803d040 comp rx=0 tx=0).stop 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 msgr2=0x7f78b419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e970 tx=0x7f78ac00ed70 comp rx=0 tx=0).stop 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 shutdown_connections 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f789c07b270 0x7f789c07d730 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b4105bf0 0x7f78b419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 --2- 172.21.15.110:0/342503417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b40ffb40 0x7f78b419b810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 >> 172.21.15.110:0/342503417 conn(0x7f78b40fb840 msgr2=0x7f78b40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:01.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 shutdown_connections 2024-09-21T04:25:01.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:01.373+0000 7f78bc3c7640 1 -- 172.21.15.110:0/342503417 wait complete. 2024-09-21T04:25:01.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:01 smithi148 ceph-mon[78701]: pgmap v278: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:02.132 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:02 smithi110 ceph-mon[92455]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:25:02.155 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade status' 2024-09-21T04:25:02.358 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:02.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:02 smithi148 ceph-mon[78701]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:25:03.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:03 smithi110 ceph-mon[92455]: pgmap v279: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:03.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:03 smithi148 ceph-mon[78701]: pgmap v279: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:05.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:05 smithi110 ceph-mon[92455]: pgmap v280: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:05.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:05 smithi148 ceph-mon[78701]: pgmap v280: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:07.326 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:07 smithi110 ceph-mon[92455]: pgmap v281: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:07.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:07 smithi148 ceph-mon[78701]: pgmap v281: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:07.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3135989788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b0105b80 msgr2=0x7fb6b0105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:07.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3135989788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b0105b80 0x7fb6b0105f60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009880 tx=0x7fb6a002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3135989788 shutdown_connections 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3135989788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3135989788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b0105b80 0x7fb6b0105f60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3135989788 >> 172.21.15.110:0/3135989788 conn(0x7fb6b00fb810 msgr2=0x7fb6b00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3135989788 shutdown_connections 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.544+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3135989788 wait complete. 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 Processor -- start 2024-09-21T04:25:07.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 -- start start 2024-09-21T04:25:07.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:07.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b0105b80 0x7fb6b019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:07.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0195a00 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6b5d97640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0195b40 con 0x7fb6b0105b80 2024-09-21T04:25:07.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60528/0 (socket says 172.21.15.110:60528) 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 -- 172.21.15.110:0/3973736825 learned_addr learned my addr 172.21.15.110:0/3973736825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6a6ffd640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b0105b80 0x7fb6b019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 -- 172.21.15.110:0/3973736825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b0105b80 msgr2=0x7fb6b019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b0105b80 0x7fb6b019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.545+0000 7fb6af7fe640 1 -- 172.21.15.110:0/3973736825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009560 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6af7fe640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a00055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a0004aa0 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6a0031470 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b0195dc0 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a0003940 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.546+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b01962b0 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.547+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b01011f0 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.548+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb6a0031b90 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.548+0000 7fb6ad7fa640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 0x7fb69c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.548+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fb6a00bf690 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.548+0000 7fb6a6ffd640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 0x7fb69c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:07.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.549+0000 7fb6a6ffd640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 0x7fb69c07d740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb698005fd0 tx=0x7fb698005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:07.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.553+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb6a00824d0 con 0x7fb6b00ffaf0 2024-09-21T04:25:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.749+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6b010cd60 con 0x7fb69c07b280 2024-09-21T04:25:07.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.751+0000 7fb6ad7fa640 1 -- 172.21.15.110:0/3973736825 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fb6b010cd60 con 0x7fb69c07b280 2024-09-21T04:25:07.753 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": null, 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": false, 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "", 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": null, 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-09-21T04:25:07.754 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 msgr2=0x7fb69c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 0x7fb69c07d740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb698005fd0 tx=0x7fb698005850 comp rx=0 tx=0).stop 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 msgr2=0x7fb6b019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a00055a0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 shutdown_connections 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb69c07b280 0x7fb69c07d740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b0105b80 0x7fb6b019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 --2- 172.21.15.110:0/3973736825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b00ffaf0 0x7fb6b019b9a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 >> 172.21.15.110:0/3973736825 conn(0x7fb6b00fb810 msgr2=0x7fb6b00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 shutdown_connections 2024-09-21T04:25:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:07.753+0000 7fb6b5d97640 1 -- 172.21.15.110:0/3973736825 wait complete. 2024-09-21T04:25:08.394 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:08 smithi110 ceph-mon[92455]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:25:08.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph health detail' 2024-09-21T04:25:08.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:08 smithi148 ceph-mon[78701]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:25:08.618 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:09 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:09.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:09 smithi110 ceph-mon[92455]: pgmap v282: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:09 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:09.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:09 smithi148 ceph-mon[78701]: pgmap v282: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 -- 172.21.15.110:0/2790160465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c0ffb40 msgr2=0x7fa43c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 --2- 172.21.15.110:0/2790160465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c0ffb40 0x7fa43c0fffa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa430009880 tx=0x7fa43002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 -- 172.21.15.110:0/2790160465 shutdown_connections 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 --2- 172.21.15.110:0/2790160465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c0ffb40 0x7fa43c0fffa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 --2- 172.21.15.110:0/2790160465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c105bf0 0x7fa43c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 -- 172.21.15.110:0/2790160465 >> 172.21.15.110:0/2790160465 conn(0x7fa43c0fb840 msgr2=0x7fa43c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 -- 172.21.15.110:0/2790160465 shutdown_connections 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 -- 172.21.15.110:0/2790160465 wait complete. 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.234+0000 7fa440925640 1 Processor -- start 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 -- start start 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c0ffb40 0x7fa43c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c195a10 con 0x7fa43c105bf0 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c195b50 con 0x7fa43c0ffb40 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60536/0 (socket says 172.21.15.110:60536) 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 -- 172.21.15.110:0/30631410 learned_addr learned my addr 172.21.15.110:0/30631410 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 -- 172.21.15.110:0/30631410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c0ffb40 msgr2=0x7fa43c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa43a59c640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c0ffb40 0x7fa43c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c0ffb40 0x7fa43c19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 -- 172.21.15.110:0/30631410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa430009560 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa439d9b640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa43002a9c0 tx=0x7fa4300055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa430004aa0 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.235+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c195dd0 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.236+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa430004c00 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.236+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa430003ab0 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.236+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1962c0 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.236+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c1011f0 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.237+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fa430031d20 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.237+0000 7fa42b7fe640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 0x7fa42407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.238+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fa4300bf610 con 0x7fa43c105bf0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.238+0000 7fa43a59c640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 0x7fa42407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.238+0000 7fa43a59c640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 0x7fa42407d730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00a810 tx=0x7fa42c008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:10.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.242+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fa430082450 con 0x7fa43c105bf0 2024-09-21T04:25:10.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.488+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa43c108010 con 0x7fa43c105bf0 2024-09-21T04:25:10.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.489+0000 7fa42b7fe640 1 -- 172.21.15.110:0/30631410 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa430085570 con 0x7fa43c105bf0 2024-09-21T04:25:10.492 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_OK 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 msgr2=0x7fa42407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 0x7fa42407d730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00a810 tx=0x7fa42c008040 comp rx=0 tx=0).stop 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 msgr2=0x7fa43c19bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa43002a9c0 tx=0x7fa4300055a0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 shutdown_connections 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fa42407b270 0x7fa42407d730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa43c105bf0 0x7fa43c19bef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 --2- 172.21.15.110:0/30631410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa43c0ffb40 0x7fa43c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 >> 172.21.15.110:0/30631410 conn(0x7fa43c0fb840 msgr2=0x7fa43c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:10.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 shutdown_connections 2024-09-21T04:25:10.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:10.491+0000 7fa440925640 1 -- 172.21.15.110:0/30631410 wait complete. 2024-09-21T04:25:10.784 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:10 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/30631410' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:25:10.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:10 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/30631410' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:25:11.107 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions' 2024-09-21T04:25:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:11.738 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:11 smithi110 ceph-mon[92455]: pgmap v283: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:11.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:11 smithi148 ceph-mon[78701]: pgmap v283: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.863+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1890551751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 msgr2=0x7f4e80101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.863+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1890551751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e80101040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009880 tx=0x7f4e7002a530 comp rx=0 tx=0).stop 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1890551751 shutdown_connections 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1890551751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 0x7f4e80109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1890551751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e80101040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1890551751 >> 172.21.15.110:0/1890551751 conn(0x7f4e800758c0 msgr2=0x7f4e800fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:12.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1890551751 shutdown_connections 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1890551751 wait complete. 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 Processor -- start 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.864+0000 7f4e85c8a640 1 -- start start 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 0x7f4e8019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8019c460 con 0x7f4e80100c60 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e801959f0 con 0x7f4e80101580 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52498/0 (socket says 172.21.15.110:52498) 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 -- 172.21.15.110:0/1302791140 learned_addr learned my addr 172.21.15.110:0/1302791140 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7effd640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 0x7f4e8019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 -- 172.21.15.110:0/1302791140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 msgr2=0x7f4e8019be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 0x7f4e8019be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 -- 172.21.15.110:0/1302791140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009560 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7f7fe640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002aa50 tx=0x7f4e700055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e70004aa0 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e70004c00 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e80195c70 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e70003ab0 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.865+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e80196160 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.866+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e800684c0 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.867+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4e7002ad50 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.868+0000 7f4e7cff9640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 0x7f4e6c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.868+0000 7f4e7effd640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 0x7f4e6c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.868+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f4e700bf2e0 con 0x7f4e80100c60 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.868+0000 7f4e7effd640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 0x7f4e6c07d740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4e80196e60 tx=0x7f4e68008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:25:12.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:12.872+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4e700821a0 con 0x7f4e80100c60 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.120+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4e80104360 con 0x7f4e80100c60 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.121+0000 7f4e7cff9640 1 -- 172.21.15.110:0/1302791140 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4e700852c0 con 0x7f4e80100c60 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:25:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:25:13.125 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 msgr2=0x7f4e6c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 0x7f4e6c07d740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4e80196e60 tx=0x7f4e68008040 comp rx=0 tx=0).stop 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 msgr2=0x7f4e8019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002aa50 tx=0x7f4e700055a0 comp rx=0 tx=0).stop 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 shutdown_connections 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4e6c07b280 0x7f4e6c07d740 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e80101580 0x7f4e8019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:13.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 --2- 172.21.15.110:0/1302791140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e80100c60 0x7f4e8019b900 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:25:13.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 >> 172.21.15.110:0/1302791140 conn(0x7f4e800758c0 msgr2=0x7f4e80101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:25:13.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 shutdown_connections 2024-09-21T04:25:13.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:25:13.123+0000 7f4e85c8a640 1 -- 172.21.15.110:0/1302791140 wait complete. 2024-09-21T04:25:13.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:13 smithi110 ceph-mon[92455]: pgmap v284: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:13.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:13 smithi148 ceph-mon[78701]: pgmap v284: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:13.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-21T04:25:14.021 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:14.143 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:14 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1302791140' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:25:14.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:14 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1302791140' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:25:15.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:15 smithi110 ceph-mon[92455]: pgmap v285: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:15.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:15 smithi148 ceph-mon[78701]: pgmap v285: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:15.570 INFO:teuthology.orchestra.run.smithi110.stdout:wait for servicemap items w/ changing names to refresh 2024-09-21T04:25:16.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'sleep 60' 2024-09-21T04:25:16.445 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:25:17.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:17 smithi110 ceph-mon[92455]: pgmap v286: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:17.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:17 smithi148 ceph-mon[78701]: pgmap v286: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:19.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:19 smithi110 ceph-mon[92455]: pgmap v287: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:19.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:19 smithi148 ceph-mon[78701]: pgmap v287: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:21.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:21 smithi148 ceph-mon[78701]: pgmap v288: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:21.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:21 smithi110 ceph-mon[92455]: pgmap v288: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:23.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:23 smithi110 ceph-mon[92455]: pgmap v289: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:23.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:23 smithi148 ceph-mon[78701]: pgmap v289: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:24.341 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:24 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:24.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:24 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:25.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:25 smithi110 ceph-mon[92455]: pgmap v290: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:25.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:25 smithi148 ceph-mon[78701]: pgmap v290: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:27.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:27 smithi110 ceph-mon[92455]: pgmap v291: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:27.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:27 smithi148 ceph-mon[78701]: pgmap v291: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:29.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:29 smithi110 ceph-mon[92455]: pgmap v292: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:29.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:29 smithi148 ceph-mon[78701]: pgmap v292: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:31.417 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:31 smithi148 ceph-mon[78701]: pgmap v293: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:31.418 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:31 smithi110 ceph-mon[92455]: pgmap v293: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:33.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:33 smithi110 ceph-mon[92455]: pgmap v294: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:33.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:33 smithi148 ceph-mon[78701]: pgmap v294: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:36.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:36 smithi110 ceph-mon[92455]: pgmap v295: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:36.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:36 smithi148 ceph-mon[78701]: pgmap v295: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:38.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:38 smithi110 ceph-mon[92455]: pgmap v296: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:38.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:38 smithi148 ceph-mon[78701]: pgmap v296: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:39.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:39 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:39.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:39 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:40.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:40 smithi110 ceph-mon[92455]: pgmap v297: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:40.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:40 smithi148 ceph-mon[78701]: pgmap v297: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:42.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:42 smithi110 ceph-mon[92455]: pgmap v298: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:42.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:42 smithi148 ceph-mon[78701]: pgmap v298: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:44.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:44 smithi110 ceph-mon[92455]: pgmap v299: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:44.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:44 smithi148 ceph-mon[78701]: pgmap v299: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:46.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:46 smithi110 ceph-mon[92455]: pgmap v300: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:46.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:46 smithi148 ceph-mon[78701]: pgmap v300: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:25:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:25:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:25:47.417 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:25:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:25:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:25:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:25:47.440 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:25:48.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:48 smithi110 ceph-mon[92455]: pgmap v301: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:48.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:48 smithi148 ceph-mon[78701]: pgmap v301: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:50.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:50 smithi110 ceph-mon[92455]: pgmap v302: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:50.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:50 smithi148 ceph-mon[78701]: pgmap v302: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:52.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:52 smithi110 ceph-mon[92455]: pgmap v303: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:52.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:52 smithi148 ceph-mon[78701]: pgmap v303: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:53.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:25:54.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:54 smithi148 ceph-mon[78701]: pgmap v304: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:54.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:54 smithi110 ceph-mon[92455]: pgmap v304: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:56.797 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:56 smithi148 ceph-mon[78701]: pgmap v305: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:56 smithi110 ceph-mon[92455]: pgmap v305: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:58.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:25:58 smithi110 ceph-mon[92455]: pgmap v306: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:25:58.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:25:58 smithi148 ceph-mon[78701]: pgmap v306: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:00 smithi110 ceph-mon[92455]: pgmap v307: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:00.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:00 smithi148 ceph-mon[78701]: pgmap v307: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:02.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:02 smithi110 ceph-mon[92455]: pgmap v308: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:02.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:02 smithi148 ceph-mon[78701]: pgmap v308: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:04.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:04 smithi110 ceph-mon[92455]: pgmap v309: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:04.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:04 smithi148 ceph-mon[78701]: pgmap v309: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:06 smithi110 ceph-mon[92455]: pgmap v310: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:06.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:06 smithi148 ceph-mon[78701]: pgmap v310: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:08.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:08 smithi110 ceph-mon[92455]: pgmap v311: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:08.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:08.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:08 smithi148 ceph-mon[78701]: pgmap v311: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:08.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:10.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:10 smithi110 ceph-mon[92455]: pgmap v312: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:10.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:10 smithi148 ceph-mon[78701]: pgmap v312: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:12.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:12 smithi110 ceph-mon[92455]: pgmap v313: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:12.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:12 smithi148 ceph-mon[78701]: pgmap v313: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:14.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:14 smithi110 ceph-mon[92455]: pgmap v314: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:14.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:14 smithi148 ceph-mon[78701]: pgmap v314: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:16.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:16 smithi110 ceph-mon[92455]: pgmap v315: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:16.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:16 smithi148 ceph-mon[78701]: pgmap v315: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:18.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:18 smithi110 ceph-mon[92455]: pgmap v316: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:18.641 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch ps' 2024-09-21T04:26:18.847 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:18.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:18 smithi148 ceph-mon[78701]: pgmap v316: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 -- 172.21.15.110:0/349243871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c105bf0 msgr2=0x7f8e9c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 --2- 172.21.15.110:0/349243871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c105fd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009880 tx=0x7f8e9802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 -- 172.21.15.110:0/349243871 shutdown_connections 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 --2- 172.21.15.110:0/349243871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 --2- 172.21.15.110:0/349243871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c105fd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 -- 172.21.15.110:0/349243871 >> 172.21.15.110:0/349243871 conn(0x7f8e9c0fb840 msgr2=0x7f8e9c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:20.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 -- 172.21.15.110:0/349243871 shutdown_connections 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.471+0000 7f8ea3739640 1 -- 172.21.15.110:0/349243871 wait complete. 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 Processor -- start 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 -- start start 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c19c350 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea3739640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c195930 con 0x7f8e9c105bf0 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea14d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea14d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36020/0 (socket says 172.21.15.110:36020) 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea14d5640 1 -- 172.21.15.110:0/1822626140 learned_addr learned my addr 172.21.15.110:0/1822626140 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.472+0000 7f8ea0cd4640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea14d5640 1 -- 172.21.15.110:0/1822626140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c105bf0 msgr2=0x7f8e9c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea14d5640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea14d5640 1 -- 172.21.15.110:0/1822626140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009560 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea14d5640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e9802a9c0 tx=0x7f8e980055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e98004aa0 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e98004c00 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c195bb0 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e98003980 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.473+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1960a0 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.474+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c1011f0 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.475+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f8e98003b20 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.475+0000 7f8e927fc640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 0x7f8e8807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.475+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f8e980bf430 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.475+0000 7f8ea0cd4640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 0x7f8e8807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:20.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.476+0000 7f8ea0cd4640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 0x7f8e8807d730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c196d50 tx=0x7f8e8c00b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:20.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.480+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f8e980823a0 con 0x7f8e9c0ffb40 2024-09-21T04:26:20.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.668+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 --> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e9c10cdb0 con 0x7f8e8807b270 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.678+0000 7f8e927fc640 1 -- 172.21.15.110:0/1822626140 <== mgr.24563 v2:172.21.15.110:6800/1486080047 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8e9c10cdb0 con 0x7f8e8807b270 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (2m) 98s ago 19m 17.7M - 0.27.0 11f11916f8cd 33ba9a4a9ab1 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi110 smithi110 running (3m) 98s ago 19m 8451k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 9c05a31416c2 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-exporter.smithi148 smithi148 running (3m) 2m ago 18m 8102k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3ed27bd24a31 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 98s ago 19m 6891k - 19.3.0-5103-gf36283e7 48ed0a2cf89d d7eb61281ecb 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi148 smithi148 running (8m) 2m ago 18m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 0058977ec9aa 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (103s) 98s ago 19m 45.8M - 10.4.8 5aad1d7cf004 828a5abe7bd9 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.faxwju smithi110 running (4m) 98s ago 13m 14.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d eb0d7ea9bcbf 2024-09-21T04:26:20.681 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.trhyxr smithi110 running (4m) 98s ago 13m 83.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 767147bbcbda 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.fpatxi smithi148 running (3m) 2m ago 13m 13.3M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 55fa666e3a5c 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi148.tpueww smithi148 running (3m) 2m ago 13m 13.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 3843357dd7de 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.axmqzb smithi110 *:8443,9283,8765 running (10m) 98s ago 20m 533M - 19.3.0-5103-gf36283e7 48ed0a2cf89d cf7c52bda114 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi148.zjrahi smithi148 *:8443,9283,8765 running (9m) 2m ago 18m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 563d89986d4b 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 98s ago 20m 58.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 76f20187adf7 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi148 smithi148 running (8m) 2m ago 18m 45.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 62be042bab1c 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (2m) 98s ago 19m 9.98M - 1.7.0 72c9c2088986 52919d9343b8 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 2m ago 18m 3338k - 1.7.0 72c9c2088986 091b9b7135aa 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 98s ago 17m 121M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d a67c7fde8c6b 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (7m) 98s ago 16m 156M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d f054d20509ea 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 98s ago 16m 175M 1216M 19.3.0-5103-gf36283e7 48ed0a2cf89d 72f3c7b658aa 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi148 running (5m) 2m ago 15m 142M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 36931325021e 2024-09-21T04:26:20.682 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi148 running (5m) 2m ago 15m 105M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d 0dc491d8eac7 2024-09-21T04:26:20.683 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi148 running (4m) 2m ago 15m 104M 2240M 19.3.0-5103-gf36283e7 48ed0a2cf89d c0f3036b7f8d 2024-09-21T04:26:20.683 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 98s ago 19m 43.8M - 2.51.0 1d3b7f56885b 1b120422a054 2024-09-21T04:26:20.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 msgr2=0x7f8e8807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:20.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 0x7f8e8807d730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c196d50 tx=0x7f8e8c00b390 comp rx=0 tx=0).stop 2024-09-21T04:26:20.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 msgr2=0x7f8e9c19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e9802a9c0 tx=0x7f8e980055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 shutdown_connections 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f8e8807b270 0x7f8e8807d730 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9c105bf0 0x7f8e9c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 --2- 172.21.15.110:0/1822626140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e9c0ffb40 0x7f8e9c19b7f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 >> 172.21.15.110:0/1822626140 conn(0x7f8e9c0fb840 msgr2=0x7f8e9c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 shutdown_connections 2024-09-21T04:26:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:20.680+0000 7f8ea3739640 1 -- 172.21.15.110:0/1822626140 wait complete. 2024-09-21T04:26:20.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:20 smithi110 ceph-mon[92455]: pgmap v317: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:20.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:20 smithi148 ceph-mon[78701]: pgmap v317: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:21.326 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions' 2024-09-21T04:26:21.545 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:21.810 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:21 smithi110 ceph-mon[92455]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:21.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:21 smithi148 ceph-mon[78701]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:22.705 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:22 smithi110 ceph-mon[92455]: pgmap v318: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:22.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:22 smithi148 ceph-mon[78701]: pgmap v318: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 -- 172.21.15.110:0/2745592368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9840fea20 msgr2=0x7fb9840fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 --2- 172.21.15.110:0/2745592368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9840fea20 0x7fb9840fee80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb978009880 tx=0x7fb97802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 -- 172.21.15.110:0/2745592368 shutdown_connections 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 --2- 172.21.15.110:0/2745592368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9840fea20 0x7fb9840fee80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 --2- 172.21.15.110:0/2745592368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb984104a20 0x7fb984104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 -- 172.21.15.110:0/2745592368 >> 172.21.15.110:0/2745592368 conn(0x7fb9840fa720 msgr2=0x7fb9840fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 -- 172.21.15.110:0/2745592368 shutdown_connections 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.203+0000 7fb989442640 1 -- 172.21.15.110:0/2745592368 wait complete. 2024-09-21T04:26:23.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 Processor -- start 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 -- start start 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9840fea20 0x7fb984196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841975d0 con 0x7fb984104a20 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb989442640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984197710 con 0x7fb9840fea20 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb9827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb9827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37030/0 (socket says 172.21.15.110:37030) 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb9827fc640 1 -- 172.21.15.110:0/559182687 learned_addr learned my addr 172.21.15.110:0/559182687 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.204+0000 7fb982ffd640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9840fea20 0x7fb984196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb9827fc640 1 -- 172.21.15.110:0/559182687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9840fea20 msgr2=0x7fb984196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb9827fc640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9840fea20 0x7fb984196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb9827fc640 1 -- 172.21.15.110:0/559182687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb978009560 con 0x7fb984104a20 2024-09-21T04:26:23.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb9827fc640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb978006010 tx=0x7fb9780055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb978004aa0 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb978004c00 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98419fae0 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9780038f0 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.205+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98419fef0 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.206+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984100160 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.207+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb978003bb0 con 0x7fb984104a20 2024-09-21T04:26:23.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.207+0000 7fb96bfff640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 0x7fb96c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:23.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.207+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fb9780bf760 con 0x7fb984104a20 2024-09-21T04:26:23.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.207+0000 7fb982ffd640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 0x7fb96c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:23.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.208+0000 7fb982ffd640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 0x7fb96c07d730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb974009990 tx=0x7fb974009110 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:23.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.212+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb9780825a0 con 0x7fb984104a20 2024-09-21T04:26:23.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.467+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb984107000 con 0x7fb984104a20 2024-09-21T04:26:23.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.467+0000 7fb96bfff640 1 -- 172.21.15.110:0/559182687 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb978003e60 con 0x7fb984104a20 2024-09-21T04:26:23.470 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T04:26:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-09-21T04:26:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:26:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T04:26:23.471 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T04:26:23.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 msgr2=0x7fb96c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 0x7fb96c07d730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb974009990 tx=0x7fb974009110 comp rx=0 tx=0).stop 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 msgr2=0x7fb984196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb978006010 tx=0x7fb9780055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 shutdown_connections 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb96c07b270 0x7fb96c07d730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb984104a20 0x7fb984196fb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 --2- 172.21.15.110:0/559182687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9840fea20 0x7fb984196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.469+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 >> 172.21.15.110:0/559182687 conn(0x7fb9840fa720 msgr2=0x7fb9840fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.470+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 shutdown_connections 2024-09-21T04:26:23.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:23.470+0000 7fb989442640 1 -- 172.21.15.110:0/559182687 wait complete. 2024-09-21T04:26:23.788 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:23.788 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:23 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/559182687' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:23.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:23 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/559182687' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:24.111 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-21T04:26:24.316 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:24.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:24 smithi110 ceph-mon[92455]: pgmap v319: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:24.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:24 smithi148 ceph-mon[78701]: pgmap v319: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 -- 172.21.15.110:0/365702456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd8105a80 msgr2=0x7f5fd8105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/365702456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd8105a80 0x7f5fd8105e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009880 tx=0x7f5fcc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 -- 172.21.15.110:0/365702456 shutdown_connections 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/365702456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd80ffa80 0x7f5fd80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/365702456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd8105a80 0x7f5fd8105e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 -- 172.21.15.110:0/365702456 >> 172.21.15.110:0/365702456 conn(0x7f5fd80fb7a0 msgr2=0x7f5fd80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 -- 172.21.15.110:0/365702456 shutdown_connections 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.919+0000 7f5fdf20f640 1 -- 172.21.15.110:0/365702456 wait complete. 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 Processor -- start 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 -- start start 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8105a80 0x7f5fd819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd819c360 con 0x7f5fd80ffa80 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdf20f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81958f0 con 0x7f5fd8105a80 2024-09-21T04:26:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37058/0 (socket says 172.21.15.110:37058) 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 -- 172.21.15.110:0/3349565604 learned_addr learned my addr 172.21.15.110:0/3349565604 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fd7fff640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8105a80 0x7f5fd819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 -- 172.21.15.110:0/3349565604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8105a80 msgr2=0x7f5fd819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8105a80 0x7f5fd819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 -- 172.21.15.110:0/3349565604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009560 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.920+0000 7f5fdcfab640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02a9c0 tx=0x7f5fcc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.921+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc004aa0 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.921+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd8195b70 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.921+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fcc004c00 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.921+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fcc003ab0 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.921+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8196060 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.922+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd81011c0 con 0x7f5fd80ffa80 2024-09-21T04:26:25.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.923+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f5fcc031d20 con 0x7f5fd80ffa80 2024-09-21T04:26:25.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.923+0000 7f5fd5ffb640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 0x7f5fc807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:25.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.923+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f5fcc0bf560 con 0x7f5fd80ffa80 2024-09-21T04:26:25.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.923+0000 7f5fd7fff640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 0x7f5fc807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:25.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.924+0000 7f5fd7fff640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 0x7f5fc807d730 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8196d60 tx=0x7f5fc400a2d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:25.927+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f5fcc082450 con 0x7f5fd80ffa80 2024-09-21T04:26:26.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.182+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5fd8102c90 con 0x7f5fd80ffa80 2024-09-21T04:26:26.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.182+0000 7f5fd5ffb640 1 -- 172.21.15.110:0/3349565604 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5fcc085570 con 0x7f5fd80ffa80 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 msgr2=0x7f5fc807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 0x7f5fc807d730 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8196d60 tx=0x7f5fc400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 msgr2=0x7f5fd819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02a9c0 tx=0x7f5fcc0055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 shutdown_connections 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f5fc807b270 0x7f5fc807d730 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8105a80 0x7f5fd819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:26.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 --2- 172.21.15.110:0/3349565604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fd80ffa80 0x7f5fd819b800 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:26.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 >> 172.21.15.110:0/3349565604 conn(0x7f5fd80fb7a0 msgr2=0x7f5fd80fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:26.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 shutdown_connections 2024-09-21T04:26:26.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:26.185+0000 7f5fdf20f640 1 -- 172.21.15.110:0/3349565604 wait complete. 2024-09-21T04:26:26.197 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-09-21T04:26:26.829 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:26 smithi110 ceph-mon[92455]: pgmap v320: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:26.830 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:26 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3349565604' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:26.852 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-21T04:26:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:26 smithi148 ceph-mon[78701]: pgmap v320: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:26.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:26 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3349565604' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:28.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 -- 172.21.15.110:0/1337981517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a8104990 msgr2=0x7fc6a8104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:28.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 --2- 172.21.15.110:0/1337981517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a8104990 0x7fc6a8104d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc698009880 tx=0x7fc69802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 -- 172.21.15.110:0/1337981517 shutdown_connections 2024-09-21T04:26:28.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 --2- 172.21.15.110:0/1337981517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a80fec40 0x7fc6a80ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 --2- 172.21.15.110:0/1337981517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a8104990 0x7fc6a8104d70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.624+0000 7fc6ad825640 1 -- 172.21.15.110:0/1337981517 >> 172.21.15.110:0/1337981517 conn(0x7fc6a8074700 msgr2=0x7fc6a8074b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.625+0000 7fc6ad825640 1 -- 172.21.15.110:0/1337981517 shutdown_connections 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.625+0000 7fc6ad825640 1 -- 172.21.15.110:0/1337981517 wait complete. 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.625+0000 7fc6ad825640 1 Processor -- start 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.625+0000 7fc6ad825640 1 -- start start 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 0x7fc6a819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a81947e0 con 0x7fc6a80fec40 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8194950 con 0x7fc6a8104990 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37090/0 (socket says 172.21.15.110:37090) 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 -- 172.21.15.110:0/2814741864 learned_addr learned my addr 172.21.15.110:0/2814741864 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a67fc640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 0x7fc6a819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 -- 172.21.15.110:0/2814741864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 msgr2=0x7fc6a819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 0x7fc6a819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 -- 172.21.15.110:0/2814741864 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009560 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a67fc640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 0x7fc6a819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6a6ffd640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc69802a9c0 tx=0x7fc6980055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69803e070 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc698031640 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a8194bd0 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc698003820 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.626+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a81950c0 con 0x7fc6a80fec40 2024-09-21T04:26:28.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.627+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a81000d0 con 0x7fc6a80fec40 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.628+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fc6980317b0 con 0x7fc6a80fec40 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.628+0000 7fc6ac823640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 0x7fc69007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.629+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fc6980c0930 con 0x7fc6a80fec40 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.629+0000 7fc6a67fc640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 0x7fc69007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.629+0000 7fc6a67fc640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 0x7fc69007d740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0058c0 tx=0x7fc69c005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:28.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.633+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fc698083770 con 0x7fc6a80fec40 2024-09-21T04:26:28.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:28 smithi110 ceph-mon[92455]: pgmap v321: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:28.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.878+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6a810bc00 con 0x7fc6a80fec40 2024-09-21T04:26:28.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.879+0000 7fc6ac823640 1 -- 172.21.15.110:0/2814741864 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc698086890 con 0x7fc6a80fec40 2024-09-21T04:26:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 msgr2=0x7fc69007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 0x7fc69007d740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0058c0 tx=0x7fc69c005850 comp rx=0 tx=0).stop 2024-09-21T04:26:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 msgr2=0x7fc6a819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc69802a9c0 tx=0x7fc6980055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 shutdown_connections 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc69007b280 0x7fc69007d740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6a8104990 0x7fc6a819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 --2- 172.21.15.110:0/2814741864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6a80fec40 0x7fc6a819a6a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 >> 172.21.15.110:0/2814741864 conn(0x7fc6a8074700 msgr2=0x7fc6a80fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 shutdown_connections 2024-09-21T04:26:28.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:28.881+0000 7fc6ad825640 1 -- 172.21.15.110:0/2814741864 wait complete. 2024-09-21T04:26:28.893 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)" 2024-09-21T04:26:28.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:28 smithi148 ceph-mon[78701]: pgmap v321: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:29.525 DEBUG:teuthology.parallel:result is None 2024-09-21T04:26:29.525 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:26:29.537 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:26:29.537 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:26:29.736 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:29.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:29 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2814741864' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:29.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:29 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2814741864' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:26:30.797 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:30 smithi110 ceph-mon[92455]: pgmap v322: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:30.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:30 smithi148 ceph-mon[78701]: pgmap v322: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- 172.21.15.110:0/676509924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 msgr2=0x7f08c40fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/676509924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c40fee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009880 tx=0x7f08b802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- 172.21.15.110:0/676509924 shutdown_connections 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/676509924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c40fee00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/676509924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 0x7f08c4104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- 172.21.15.110:0/676509924 >> 172.21.15.110:0/676509924 conn(0x7f08c40758c0 msgr2=0x7f08c40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- 172.21.15.110:0/676509924 shutdown_connections 2024-09-21T04:26:31.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- 172.21.15.110:0/676509924 wait complete. 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 Processor -- start 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- start start 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 0x7f08c419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c419c380 con 0x7f08c40fe9a0 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.275+0000 7f08cc6b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4195910 con 0x7f08c41049a0 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37116/0 (socket says 172.21.15.110:37116) 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 -- 172.21.15.110:0/1103125708 learned_addr learned my addr 172.21.15.110:0/1103125708 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08c9c4b640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 0x7f08c419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 -- 172.21.15.110:0/1103125708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 msgr2=0x7f08c419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 0x7f08c419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 -- 172.21.15.110:0/1103125708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009560 con 0x7f08c40fe9a0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08ca44c640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00ea30 tx=0x7f08bc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08bc00cdb0 con 0x7f08c40fe9a0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08bc011930 con 0x7f08c40fe9a0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c4195bf0 con 0x7f08c40fe9a0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08bc01aa30 con 0x7f08c40fe9a0 2024-09-21T04:26:31.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.276+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c4196110 con 0x7f08c40fe9a0 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.277+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c41000e0 con 0x7f08c40fe9a0 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.278+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f08bc01aca0 con 0x7f08c40fe9a0 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.278+0000 7f08c37fe640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 0x7f08ac07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.278+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f08bc09ebf0 con 0x7f08c40fe9a0 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.278+0000 7f08c9c4b640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 0x7f08ac07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:31.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.279+0000 7f08c9c4b640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 0x7f08ac07d740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8005a20 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:31.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.283+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f08bc061a30 con 0x7f08c40fe9a0 2024-09-21T04:26:31.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.498+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f08c410bc10 con 0x7f08c40fe9a0 2024-09-21T04:26:31.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.498+0000 7f08c37fe640 1 -- 172.21.15.110:0/1103125708 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1982 (secure 0 0 0) 0x7f08bc064b50 con 0x7f08c40fe9a0 2024-09-21T04:26:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-09-21T04:26:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:btime 2024-09-21T04:22:57:935402+0000 2024-09-21T04:26:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 34 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-21T04:12:39.437314+0000 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-09-21T04:22:56.930115+0000 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-09-21T04:26:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 95 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-09-21T04:26:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.trhyxr{0:34404} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.110:6826/2012701813,v1:172.21.15.110:6827/2012701813] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.faxwju{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/3347525595,v1:172.21.15.110:6829/3347525595] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.tpueww{-1:44293} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/2751562686,v1:172.21.15.148:6825/2751562686] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:26:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi148.fpatxi{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2055628785,v1:172.21.15.148:6827/2055628785] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 msgr2=0x7f08ac07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 0x7f08ac07d740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8005a20 comp rx=0 tx=0).stop 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 msgr2=0x7f08c419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00ea30 tx=0x7f08bc00eef0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 shutdown_connections 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f08ac07b280 0x7f08ac07d740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c41049a0 0x7f08c419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 --2- 172.21.15.110:0/1103125708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08c40fe9a0 0x7f08c419b820 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 >> 172.21.15.110:0/1103125708 conn(0x7f08c40758c0 msgr2=0x7f08c40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 shutdown_connections 2024-09-21T04:26:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:31.501+0000 7f08cc6b0640 1 -- 172.21.15.110:0/1103125708 wait complete. 2024-09-21T04:26:31.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:31 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1103125708' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:26:31.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:31 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1103125708' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:26:32.190 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-21T04:26:32.208 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:26:32.410 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:32.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:32 smithi148 ceph-mon[78701]: pgmap v323: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:32.782 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:32 smithi110 ceph-mon[92455]: pgmap v323: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 -- 172.21.15.110:0/519606857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 msgr2=0x7f1d900fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/519606857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d900fee80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009880 tx=0x7f1d7802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 -- 172.21.15.110:0/519606857 shutdown_connections 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/519606857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d900fee80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/519606857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 0x7f1d90104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 -- 172.21.15.110:0/519606857 >> 172.21.15.110:0/519606857 conn(0x7f1d900fa720 msgr2=0x7f1d900fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 -- 172.21.15.110:0/519606857 shutdown_connections 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.907+0000 7f1d96c6b640 1 -- 172.21.15.110:0/519606857 wait complete. 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 Processor -- start 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 -- start start 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 0x7f1d90196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:33.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90197440 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d96c6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d901975b0 con 0x7f1d90104a20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53296/0 (socket says 172.21.15.110:53296) 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 -- 172.21.15.110:0/1713231654 learned_addr learned my addr 172.21.15.110:0/1713231654 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d8ffff640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 0x7f1d90196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 -- 172.21.15.110:0/1713231654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 msgr2=0x7f1d90196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 0x7f1d90196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.908+0000 7f1d94a07640 1 -- 172.21.15.110:0/1713231654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78009560 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d94a07640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000d6f0 tx=0x7f1d8000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d80009a30 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9019f9a0 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d800054e0 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d80010460 con 0x7f1d900fea20 2024-09-21T04:26:33.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.909+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9019fdb0 con 0x7f1d900fea20 2024-09-21T04:26:33.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.910+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90100160 con 0x7f1d900fea20 2024-09-21T04:26:33.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.911+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f1d80009b90 con 0x7f1d900fea20 2024-09-21T04:26:33.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.911+0000 7f1d8dffb640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 0x7f1d7c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:33.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.911+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f1d8009e080 con 0x7f1d900fea20 2024-09-21T04:26:33.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.911+0000 7f1d8ffff640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 0x7f1d7c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:33.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.912+0000 7f1d8ffff640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 0x7f1d7c07d730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78005a20 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:33.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:33.916+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f1d80060ec0 con 0x7f1d900fea20 2024-09-21T04:26:34.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.128+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d90068a90 con 0x7f1d900fea20 2024-09-21T04:26:34.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.129+0000 7f1d8dffb640 1 -- 172.21.15.110:0/1713231654 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5261 (secure 0 0 0) 0x7f1d80063fe0 con 0x7f1d900fea20 2024-09-21T04:26:34.132 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:34.132 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":35,"btime":"2024-09-21T04:22:57:935402+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44313,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2055628785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2055628785},{"type":"v1","addr":"172.21.15.148:6827","nonce":2055628785}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:56.930115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":0,"incarnation":31,"state":"up:active","state_seq":19,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34404,"qdb_cluster":[34404]},"id":1}]} 2024-09-21T04:26:34.132 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 msgr2=0x7f1d7c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 0x7f1d7c07d730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78005a20 comp rx=0 tx=0).stop 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 msgr2=0x7f1d90196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000d6f0 tx=0x7f1d8000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 shutdown_connections 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f1d7c07b270 0x7f1d7c07d730 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d90104a20 0x7f1d90196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 --2- 172.21.15.110:0/1713231654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d900fea20 0x7f1d90196930 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 >> 172.21.15.110:0/1713231654 conn(0x7f1d900fa720 msgr2=0x7f1d900fb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 shutdown_connections 2024-09-21T04:26:34.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:34.131+0000 7f1d96c6b640 1 -- 172.21.15.110:0/1713231654 wait complete. 2024-09-21T04:26:34.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:34 smithi110 ceph-mon[92455]: pgmap v324: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:34.686 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:34 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1713231654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:26:34.708 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-09-21T04:26:34.708 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 16 2024-09-21T04:26:34.906 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:34.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:34 smithi148 ceph-mon[78701]: pgmap v324: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:34.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:34 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1713231654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:26:36.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.517+0000 7fd87a88e640 1 -- 172.21.15.110:0/4269885138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8741016c0 msgr2=0x7fd874109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:36.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 --2- 172.21.15.110:0/4269885138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8741016c0 0x7fd874109bb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd868009880 tx=0x7fd86802a530 comp rx=0 tx=0).stop 2024-09-21T04:26:36.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 -- 172.21.15.110:0/4269885138 shutdown_connections 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 --2- 172.21.15.110:0/4269885138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8741016c0 0x7fd874109bb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 --2- 172.21.15.110:0/4269885138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd874100da0 0x7fd874101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 -- 172.21.15.110:0/4269885138 >> 172.21.15.110:0/4269885138 conn(0x7fd8740fa720 msgr2=0x7fd8740fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 -- 172.21.15.110:0/4269885138 shutdown_connections 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 -- 172.21.15.110:0/4269885138 wait complete. 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 Processor -- start 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.518+0000 7fd87a88e640 1 -- start start 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 0x7fd87419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741959b0 con 0x7fd874100da0 2024-09-21T04:26:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874195af0 con 0x7fd8741016c0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53322/0 (socket says 172.21.15.110:53322) 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 -- 172.21.15.110:0/2532096520 learned_addr learned my addr 172.21.15.110:0/2532096520 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd8737fe640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 0x7fd87419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 -- 172.21.15.110:0/2532096520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 msgr2=0x7fd87419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 0x7fd87419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 -- 172.21.15.110:0/2532096520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009560 con 0x7fd874100da0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd8737fe640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 0x7fd87419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd873fff640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d6f0 tx=0x7fd86400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864009a30 con 0x7fd874100da0 2024-09-21T04:26:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd874195dd0 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8640054e0 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8741962f0 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.519+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864004e60 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.520+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874068c20 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.521+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fd864009b90 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.522+0000 7fd8717fa640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 0x7fd85c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.522+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fd86409e400 con 0x7fd874100da0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.522+0000 7fd8737fe640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 0x7fd85c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:36.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.522+0000 7fd8737fe640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 0x7fd85c07d730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd868031440 tx=0x7fd868005a20 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.526+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fd864061240 con 0x7fd874100da0 2024-09-21T04:26:36.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.741+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7fd874101180 con 0x7fd874100da0 2024-09-21T04:26:36.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.742+0000 7fd8717fa640 1 -- 172.21.15.110:0/2532096520 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v35) v1 ==== 107+0+4123 (secure 0 0 0) 0x7fd864064360 con 0x7fd874100da0 2024-09-21T04:26:36.745 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:36.745 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":16,"btime":"2024-09-21T04:21:49:425031+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:12:57.204430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"cephfs.smithi110.faxwju","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6829/1332815334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1332815334},{"type":"v1","addr":"172.21.15.110:6829","nonce":1332815334}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:36.745 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 16 2024-09-21T04:26:36.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 msgr2=0x7fd85c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:36.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 0x7fd85c07d730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd868031440 tx=0x7fd868005a20 comp rx=0 tx=0).stop 2024-09-21T04:26:36.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 msgr2=0x7fd87419b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:36.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d6f0 tx=0x7fd86400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 shutdown_connections 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fd85c07b270 0x7fd85c07d730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8741016c0 0x7fd87419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 --2- 172.21.15.110:0/2532096520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd874100da0 0x7fd87419b950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 >> 172.21.15.110:0/2532096520 conn(0x7fd8740fa720 msgr2=0x7fd874102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 shutdown_connections 2024-09-21T04:26:36.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:36.744+0000 7fd87a88e640 1 -- 172.21.15.110:0/2532096520 wait complete. 2024-09-21T04:26:36.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:36 smithi110 ceph-mon[92455]: pgmap v325: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:36.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:36 smithi148 ceph-mon[78701]: pgmap v325: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:37.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 17 2024-09-21T04:26:37.592 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:37.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:37 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2532096520' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 16, "format": "json"} : dispatch 2024-09-21T04:26:37.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:37 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2532096520' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 16, "format": "json"} : dispatch 2024-09-21T04:26:38.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:38 smithi110 ceph-mon[92455]: pgmap v326: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:38.668 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:38 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:38.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:38 smithi148 ceph-mon[78701]: pgmap v326: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:38.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:38 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:39.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 -- 172.21.15.110:0/3360775115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 msgr2=0x7f3ed80fe520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:39.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 --2- 172.21.15.110:0/3360775115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80fe520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009880 tx=0x7f3ec002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 -- 172.21.15.110:0/3360775115 shutdown_connections 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 --2- 172.21.15.110:0/3360775115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80fe520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 --2- 172.21.15.110:0/3360775115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ed8104b90 0x7f3ed8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 -- 172.21.15.110:0/3360775115 >> 172.21.15.110:0/3360775115 conn(0x7f3ed80f9ef0 msgr2=0x7f3ed80fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 -- 172.21.15.110:0/3360775115 shutdown_connections 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.165+0000 7f3edcf62640 1 -- 172.21.15.110:0/3360775115 wait complete. 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 Processor -- start 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 -- start start 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ed8104b90 0x7f3ed80ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80ffd20 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3edcf62640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80ffe90 con 0x7f3ed8104b90 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53348/0 (socket says 172.21.15.110:53348) 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 -- 172.21.15.110:0/546412068 learned_addr learned my addr 172.21.15.110:0/546412068 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 -- 172.21.15.110:0/546412068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ed8104b90 msgr2=0x7f3ed80ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ed8104b90 0x7f3ed80ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.166+0000 7f3ed659c640 1 -- 172.21.15.110:0/546412068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0009560 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3ed659c640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c8f0 tx=0x7f3ec800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec8007b80 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ec80054e0 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed81a1e00 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec800f450 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.167+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed81a2160 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.168+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed81013e0 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.168+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f3ec8004f60 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.169+0000 7f3ecf7fe640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 0x7f3ec407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.169+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f3ec809e2b0 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.169+0000 7f3ed5d9b640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 0x7f3ec407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.170+0000 7f3ed5d9b640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 0x7f3ec407d730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8100a20 tx=0x7f3ec0031400 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:39.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.173+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f3ec8061110 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.387+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f3ed8100050 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.388+0000 7f3ecf7fe640 1 -- 172.21.15.110:0/546412068 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4973 (secure 0 0 0) 0x7f3ec8064230 con 0x7f3ed80fe0c0 2024-09-21T04:26:39.391 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:39.391 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"btime":"2024-09-21T04:21:52:436880+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:12:57.204430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"cephfs.smithi110.faxwju","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.110:6829/1332815334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1332815334},{"type":"v1","addr":"172.21.15.110:6829","nonce":1332815334}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:39.392 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-09-21T04:26:39.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 msgr2=0x7f3ec407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:39.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 0x7f3ec407d730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8100a20 tx=0x7f3ec0031400 comp rx=0 tx=0).stop 2024-09-21T04:26:39.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 msgr2=0x7f3ed80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c8f0 tx=0x7f3ec800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 shutdown_connections 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ec407b270 0x7f3ec407d730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ed8104b90 0x7f3ed80ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 --2- 172.21.15.110:0/546412068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed80fe0c0 0x7f3ed80ff2a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 >> 172.21.15.110:0/546412068 conn(0x7f3ed80f9ef0 msgr2=0x7f3ed80fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 shutdown_connections 2024-09-21T04:26:39.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:39.390+0000 7f3edcf62640 1 -- 172.21.15.110:0/546412068 wait complete. 2024-09-21T04:26:39.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:39 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/546412068' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 17, "format": "json"} : dispatch 2024-09-21T04:26:39.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:39 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/546412068' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 17, "format": "json"} : dispatch 2024-09-21T04:26:39.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 18 2024-09-21T04:26:40.149 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:40.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:40 smithi110 ceph-mon[92455]: pgmap v327: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:40.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:40 smithi148 ceph-mon[78701]: pgmap v327: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:41.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- 172.21.15.110:0/3067496547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c101640 msgr2=0x7f759c109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:41.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3067496547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c101640 0x7f759c109b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7590009880 tx=0x7f759002a530 comp rx=0 tx=0).stop 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- 172.21.15.110:0/3067496547 shutdown_connections 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3067496547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c101640 0x7f759c109b30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3067496547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c100d20 0x7f759c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- 172.21.15.110:0/3067496547 >> 172.21.15.110:0/3067496547 conn(0x7f759c0758c0 msgr2=0x7f759c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- 172.21.15.110:0/3067496547 shutdown_connections 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- 172.21.15.110:0/3067496547 wait complete. 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 Processor -- start 2024-09-21T04:26:41.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.730+0000 7f75a27d9640 1 -- start start 2024-09-21T04:26:41.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f75a27d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:41.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f75a27d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c101640 0x7f759c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:41.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f75a27d9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19c380 con 0x7f759c100d20 2024-09-21T04:26:41.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f75a27d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c195910 con 0x7f759c101640 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53370/0 (socket says 172.21.15.110:53370) 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 -- 172.21.15.110:0/3614739638 learned_addr learned my addr 172.21.15.110:0/3614739638 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759b7fe640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c101640 0x7f759c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 -- 172.21.15.110:0/3614739638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c101640 msgr2=0x7f759c19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c101640 0x7f759c19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 -- 172.21.15.110:0/3614739638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590009560 con 0x7f759c100d20 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.731+0000 7f759bfff640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f758c00d8d0 tx=0x7f758c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.732+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758c00bea0 con 0x7f759c100d20 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.732+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c195bf0 con 0x7f759c100d20 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.732+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f758c009c00 con 0x7f759c100d20 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.732+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758c00f710 con 0x7f759c100d20 2024-09-21T04:26:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.732+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c196140 con 0x7f759c100d20 2024-09-21T04:26:41.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.733+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c068400 con 0x7f759c100d20 2024-09-21T04:26:41.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.734+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f758c00f990 con 0x7f759c100d20 2024-09-21T04:26:41.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.734+0000 7f75997fa640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 0x7f758407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:41.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.734+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f758c09e260 con 0x7f759c100d20 2024-09-21T04:26:41.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.734+0000 7f759b7fe640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 0x7f758407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:41.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.735+0000 7f759b7fe640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 0x7f758407d740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7590000c00 tx=0x7f7590031400 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:41.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.738+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f758c0610a0 con 0x7f759c100d20 2024-09-21T04:26:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.952+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f759c101100 con 0x7f759c100d20 2024-09-21T04:26:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.953+0000 7f75997fa640 1 -- 172.21.15.110:0/3614739638 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4180 (secure 0 0 0) 0x7f758c0641c0 con 0x7f759c100d20 2024-09-21T04:26:41.956 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:41.956 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":18,"btime":"2024-09-21T04:22:04:027467+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:04.027198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:41.956 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 18 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 msgr2=0x7f758407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 0x7f758407d740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7590000c00 tx=0x7f7590031400 comp rx=0 tx=0).stop 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 msgr2=0x7f759c19b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f758c00d8d0 tx=0x7f758c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 shutdown_connections 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f758407b280 0x7f758407d740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c101640 0x7f759c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 --2- 172.21.15.110:0/3614739638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759c100d20 0x7f759c19b820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:41.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 >> 172.21.15.110:0/3614739638 conn(0x7f759c0758c0 msgr2=0x7f759c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:41.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 shutdown_connections 2024-09-21T04:26:41.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:41.955+0000 7f75a27d9640 1 -- 172.21.15.110:0/3614739638 wait complete. 2024-09-21T04:26:42.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 19 2024-09-21T04:26:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:42.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:42 smithi110 ceph-mon[92455]: pgmap v328: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:42.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:42 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3614739638' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-21T04:26:42.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:42 smithi148 ceph-mon[78701]: pgmap v328: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:42.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:42 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3614739638' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-21T04:26:44.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 -- 172.21.15.110:0/3911283755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 msgr2=0x7f76a410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:44.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 --2- 172.21.15.110:0/3911283755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a410be60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7698009880 tx=0x7f769802a490 comp rx=0 tx=0).stop 2024-09-21T04:26:44.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 -- 172.21.15.110:0/3911283755 shutdown_connections 2024-09-21T04:26:44.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 --2- 172.21.15.110:0/3911283755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a410be60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 --2- 172.21.15.110:0/3911283755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 0x7f76a40febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 -- 172.21.15.110:0/3911283755 >> 172.21.15.110:0/3911283755 conn(0x7f76a40758c0 msgr2=0x7f76a40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 -- 172.21.15.110:0/3911283755 shutdown_connections 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 -- 172.21.15.110:0/3911283755 wait complete. 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.396+0000 7f76aaa32640 1 Processor -- start 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76aaa32640 1 -- start start 2024-09-21T04:26:44.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76aaa32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 0x7f76a4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76aaa32640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76aaa32640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a4198930 con 0x7f76a40ff1a0 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76aaa32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a419c6a0 con 0x7f76a40fe7f0 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58572/0 (socket says 172.21.15.110:58572) 2024-09-21T04:26:44.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 -- 172.21.15.110:0/1773199889 learned_addr learned my addr 172.21.15.110:0/1773199889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a3fff640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 0x7f76a4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 -- 172.21.15.110:0/1773199889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 msgr2=0x7f76a4197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 0x7f76a4197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.397+0000 7f76a37fe640 1 -- 172.21.15.110:0/1773199889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009560 con 0x7f76a40ff1a0 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76a37fe640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f769802a9b0 tx=0x7f76980055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:44.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7698004aa0 con 0x7f76a40ff1a0 2024-09-21T04:26:44.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7698004c00 con 0x7f76a40ff1a0 2024-09-21T04:26:44.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a419c920 con 0x7f76a40ff1a0 2024-09-21T04:26:44.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7698003980 con 0x7f76a40ff1a0 2024-09-21T04:26:44.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.398+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a419ce70 con 0x7f76a40ff1a0 2024-09-21T04:26:44.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.399+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a40684c0 con 0x7f76a40ff1a0 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.400+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f7698003b00 con 0x7f76a40ff1a0 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.400+0000 7f76a17fa640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 0x7f768c07d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.400+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f76980bf400 con 0x7f76a40ff1a0 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.400+0000 7f76a3fff640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 0x7f768c07d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.400+0000 7f76a3fff640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 0x7f768c07d740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7694005fd0 tx=0x7f769400b390 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:44.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.404+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f7698082370 con 0x7f76a40ff1a0 2024-09-21T04:26:44.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.620+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f76a4106630 con 0x7f76a40ff1a0 2024-09-21T04:26:44.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.621+0000 7f76a17fa640 1 -- 172.21.15.110:0/1773199889 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4191 (secure 0 0 0) 0x7f7698085490 con 0x7f76a40ff1a0 2024-09-21T04:26:44.624 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:44.624 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"btime":"2024-09-21T04:22:04:031973+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:04.031961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:44.625 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 19 2024-09-21T04:26:44.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 msgr2=0x7f768c07d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:44.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 0x7f768c07d740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7694005fd0 tx=0x7f769400b390 comp rx=0 tx=0).stop 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 msgr2=0x7f76a41982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f769802a9b0 tx=0x7f76980055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 shutdown_connections 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f768c07b280 0x7f768c07d740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76a40ff1a0 0x7f76a41982a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 --2- 172.21.15.110:0/1773199889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a40fe7f0 0x7f76a4197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.623+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 >> 172.21.15.110:0/1773199889 conn(0x7f76a40758c0 msgr2=0x7f76a40fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.624+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 shutdown_connections 2024-09-21T04:26:44.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:44.624+0000 7f76aaa32640 1 -- 172.21.15.110:0/1773199889 wait complete. 2024-09-21T04:26:44.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:44 smithi110 ceph-mon[92455]: pgmap v329: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:44.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:44 smithi148 ceph-mon[78701]: pgmap v329: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:45.304 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 20 2024-09-21T04:26:45.515 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:45.900 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:45 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1773199889' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-21T04:26:45.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:45 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1773199889' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-21T04:26:46.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:46 smithi110 ceph-mon[92455]: pgmap v330: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:46.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:46 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:26:46.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:46 smithi148 ceph-mon[78701]: pgmap v330: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:46.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:46 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:26:47.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 -- 172.21.15.110:0/994504957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 msgr2=0x7fbdb4109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:47.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/994504957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb4109a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009880 tx=0x7fbda402a530 comp rx=0 tx=0).stop 2024-09-21T04:26:47.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 -- 172.21.15.110:0/994504957 shutdown_connections 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/994504957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb4109a70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/994504957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 0x7fbdb4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 -- 172.21.15.110:0/994504957 >> 172.21.15.110:0/994504957 conn(0x7fbdb40758c0 msgr2=0x7fbdb40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 -- 172.21.15.110:0/994504957 shutdown_connections 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.198+0000 7fbdbc4da640 1 -- 172.21.15.110:0/994504957 wait complete. 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 Processor -- start 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 -- start start 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 0x7fbdb419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419c480 con 0x7fbdb4101580 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdbc4da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4195a10 con 0x7fbdb4100c60 2024-09-21T04:26:47.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58588/0 (socket says 172.21.15.110:58588) 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 -- 172.21.15.110:0/411729963 learned_addr learned my addr 172.21.15.110:0/411729963 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdba276640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 0x7fbdb419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 -- 172.21.15.110:0/411729963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 msgr2=0x7fbdb419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 0x7fbdb419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.199+0000 7fbdb9a75640 1 -- 172.21.15.110:0/411729963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4009560 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdb9a75640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009230 tx=0x7fbda40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda40048d0 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbda4031640 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb4195c90 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda4003820 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.200+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb4196180 con 0x7fbdb4101580 2024-09-21T04:26:47.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.201+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb40684c0 con 0x7fbdb4101580 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.201+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fbda402ad50 con 0x7fbdb4101580 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.202+0000 7fbdab7fe640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 0x7fbda007d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.202+0000 7fbdba276640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 0x7fbda007d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.202+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fbda40bf0d0 con 0x7fbdb4101580 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.202+0000 7fbdba276640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 0x7fbda007d790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009bd0 tx=0x7fbdb0005f90 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:47.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.206+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fbda4081fc0 con 0x7fbdb4101580 2024-09-21T04:26:47.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.421+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fbdb4101040 con 0x7fbdb4101580 2024-09-21T04:26:47.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.421+0000 7fbdab7fe640 1 -- 172.21.15.110:0/411729963 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4196 (secure 0 0 0) 0x7fbda40850e0 con 0x7fbdb4101580 2024-09-21T04:26:47.424 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:47.425 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":20,"btime":"2024-09-21T04:22:10:989495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:10.939303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":0,"incarnation":19,"state":"up:reconnect","state_seq":143,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:47.425 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 20 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.423+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 msgr2=0x7fbda007d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.423+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 0x7fbda007d790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009bd0 tx=0x7fbdb0005f90 comp rx=0 tx=0).stop 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 msgr2=0x7fbdb419be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009230 tx=0x7fbda40055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 shutdown_connections 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbda007b2d0 0x7fbda007d790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdb4101580 0x7fbdb419be60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 --2- 172.21.15.110:0/411729963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb4100c60 0x7fbdb419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 >> 172.21.15.110:0/411729963 conn(0x7fbdb40758c0 msgr2=0x7fbdb4101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 shutdown_connections 2024-09-21T04:26:47.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:47.424+0000 7fbdbc4da640 1 -- 172.21.15.110:0/411729963 wait complete. 2024-09-21T04:26:47.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:26:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:26:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:47.941 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:47 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/411729963' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-21T04:26:48.137 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:48.137 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:48.137 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:26:48.138 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:26:48.138 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' 2024-09-21T04:26:48.138 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:47 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/411729963' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-21T04:26:48.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 21 2024-09-21T04:26:48.363 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:49.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:48 smithi110 ceph-mon[92455]: pgmap v331: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:49.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:48 smithi148 ceph-mon[78701]: pgmap v331: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:50.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 -- 172.21.15.110:0/4038013245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 msgr2=0x7f4f201010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:50.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 --2- 172.21.15.110:0/4038013245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f201010c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009c20 tx=0x7f4f1002aa00 comp rx=0 tx=0).stop 2024-09-21T04:26:50.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 -- 172.21.15.110:0/4038013245 shutdown_connections 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 --2- 172.21.15.110:0/4038013245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 0x7f4f20109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 --2- 172.21.15.110:0/4038013245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f201010c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 -- 172.21.15.110:0/4038013245 >> 172.21.15.110:0/4038013245 conn(0x7f4f200fa720 msgr2=0x7f4f200fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 -- 172.21.15.110:0/4038013245 shutdown_connections 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.998+0000 7f4f261a8640 1 -- 172.21.15.110:0/4038013245 wait complete. 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.999+0000 7f4f261a8640 1 Processor -- start 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:49.999+0000 7f4f261a8640 1 -- start start 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f261a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f261a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 0x7f4f2019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f261a8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019c360 con 0x7f4f20100ce0 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f261a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f201958f0 con 0x7f4f20101600 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:50.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58600/0 (socket says 172.21.15.110:58600) 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 -- 172.21.15.110:0/3708273465 learned_addr learned my addr 172.21.15.110:0/3708273465 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1effd640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 0x7f4f2019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 -- 172.21.15.110:0/3708273465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 msgr2=0x7f4f2019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 0x7f4f2019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 -- 172.21.15.110:0/3708273465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10009900 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1f7fe640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f1002af20 tx=0x7f4f1002f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.000+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f1003e070 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.001+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f20195b70 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.001+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f1002fcd0 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.001+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f10043470 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.001+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f20196060 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.002+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f20068ce0 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.002+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f4f1000a040 con 0x7f4f20100ce0 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.003+0000 7f4f1cff9640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 0x7f4f0807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:50.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.003+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f4f100bf790 con 0x7f4f20100ce0 2024-09-21T04:26:50.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.003+0000 7f4f1effd640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 0x7f4f0807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:50.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.003+0000 7f4f1effd640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 0x7f4f0807d730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f20196d60 tx=0x7f4f14005f50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:50.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.007+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f4f10082680 con 0x7f4f20100ce0 2024-09-21T04:26:50.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.220+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f4f201010c0 con 0x7f4f20100ce0 2024-09-21T04:26:50.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.220+0000 7f4f1cff9640 1 -- 172.21.15.110:0/3708273465 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+4193 (secure 0 0 0) 0x7f4f100857a0 con 0x7f4f20100ce0 2024-09-21T04:26:50.223 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:50.223 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"btime":"2024-09-21T04:22:11:990872+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:10.992813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":144,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:50.224 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 msgr2=0x7f4f0807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 0x7f4f0807d730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f20196d60 tx=0x7f4f14005f50 comp rx=0 tx=0).stop 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 msgr2=0x7f4f2019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f1002af20 tx=0x7f4f1002f890 comp rx=0 tx=0).stop 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 shutdown_connections 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f4f0807b270 0x7f4f0807d730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f20101600 0x7f4f2019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 --2- 172.21.15.110:0/3708273465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f20100ce0 0x7f4f2019b800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 >> 172.21.15.110:0/3708273465 conn(0x7f4f200fa720 msgr2=0x7f4f20101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 shutdown_connections 2024-09-21T04:26:50.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:50.223+0000 7f4f261a8640 1 -- 172.21.15.110:0/3708273465 wait complete. 2024-09-21T04:26:50.925 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 22 2024-09-21T04:26:51.128 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:50 smithi110 ceph-mon[92455]: pgmap v332: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:51.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:50 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3708273465' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-21T04:26:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:50 smithi148 ceph-mon[78701]: pgmap v332: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:51.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:50 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3708273465' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-21T04:26:52.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 -- 172.21.15.110:0/2976474378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0120105bf0 msgr2=0x7f0120105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 --2- 172.21.15.110:0/2976474378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0120105bf0 0x7f0120105fd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0110009880 tx=0x7f011002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 -- 172.21.15.110:0/2976474378 shutdown_connections 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 --2- 172.21.15.110:0/2976474378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01200ffb40 0x7f01200fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 --2- 172.21.15.110:0/2976474378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0120105bf0 0x7f0120105fd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 -- 172.21.15.110:0/2976474378 >> 172.21.15.110:0/2976474378 conn(0x7f01200fb840 msgr2=0x7f01200fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 -- 172.21.15.110:0/2976474378 shutdown_connections 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 -- 172.21.15.110:0/2976474378 wait complete. 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.732+0000 7f0125b11640 1 Processor -- start 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f0125b11640 1 -- start start 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f0125b11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f0125b11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0120105bf0 0x7f012019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f0125b11640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012019c350 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f0125b11640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0120195930 con 0x7f0120105bf0 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58578/0 (socket says 172.21.15.110:58578) 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 -- 172.21.15.110:0/2897188542 learned_addr learned my addr 172.21.15.110:0/2897188542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011effd640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0120105bf0 0x7f012019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 -- 172.21.15.110:0/2897188542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0120105bf0 msgr2=0x7f012019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0120105bf0 0x7f012019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.733+0000 7f011f7fe640 1 -- 172.21.15.110:0/2897188542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110009560 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f011f7fe640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f011002a9c0 tx=0x7f01100055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0110004aa0 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0110004c00 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0120195bb0 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0110003980 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.734+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01201960a0 con 0x7f01200ffb40 2024-09-21T04:26:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.735+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01201011f0 con 0x7f01200ffb40 2024-09-21T04:26:52.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.736+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f0110003b00 con 0x7f01200ffb40 2024-09-21T04:26:52.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.736+0000 7f011cff9640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 0x7f010c07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:52.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.736+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f01100bf490 con 0x7f01200ffb40 2024-09-21T04:26:52.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.736+0000 7f011effd640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 0x7f010c07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:52.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.737+0000 7f011effd640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 0x7f010c07d730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f0108005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.741+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f0110082350 con 0x7f01200ffb40 2024-09-21T04:26:52.940 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:52 smithi148 ceph-mon[78701]: pgmap v333: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:52.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.955+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f0120108010 con 0x7f01200ffb40 2024-09-21T04:26:52.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.956+0000 7f011cff9640 1 -- 172.21.15.110:0/2897188542 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4202 (secure 0 0 0) 0x7f0110085470 con 0x7f01200ffb40 2024-09-21T04:26:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:52.959 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":22,"btime":"2024-09-21T04:22:12:995971+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:12.995969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":0,"incarnation":19,"state":"up:active","state_seq":145,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24295,"qdb_cluster":[24295]},"id":1}]} 2024-09-21T04:26:52.959 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 22 2024-09-21T04:26:52.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 msgr2=0x7f010c07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 0x7f010c07d730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f0108005850 comp rx=0 tx=0).stop 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 msgr2=0x7f012019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f011002a9c0 tx=0x7f01100055a0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 shutdown_connections 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f010c07b270 0x7f010c07d730 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0120105bf0 0x7f012019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.958+0000 7f0125b11640 1 --2- 172.21.15.110:0/2897188542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01200ffb40 0x7f012019b7f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.959+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 >> 172.21.15.110:0/2897188542 conn(0x7f01200fb840 msgr2=0x7f01200fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.959+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 shutdown_connections 2024-09-21T04:26:52.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:52.959+0000 7f0125b11640 1 -- 172.21.15.110:0/2897188542 wait complete. 2024-09-21T04:26:53.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:52 smithi110 ceph-mon[92455]: pgmap v333: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:53.647 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 23 2024-09-21T04:26:53.845 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:53 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2897188542' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-21T04:26:53.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:53 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:53 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2897188542' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-21T04:26:54.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:53 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:26:54.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:54 smithi110 ceph-mon[92455]: pgmap v334: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:55.091 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:54 smithi148 ceph-mon[78701]: pgmap v334: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 -- 172.21.15.110:0/3310884242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128100d10 msgr2=0x7ff1281010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3310884242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128100d10 0x7ff1281010f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098b0 tx=0x7ff11802a560 comp rx=0 tx=0).stop 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 -- 172.21.15.110:0/3310884242 shutdown_connections 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3310884242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128101630 0x7ff128109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3310884242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128100d10 0x7ff1281010f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 -- 172.21.15.110:0/3310884242 >> 172.21.15.110:0/3310884242 conn(0x7ff1280fa6d0 msgr2=0x7ff1280fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:55.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 -- 172.21.15.110:0/3310884242 shutdown_connections 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.396+0000 7ff12d67d640 1 -- 172.21.15.110:0/3310884242 wait complete. 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 Processor -- start 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 -- start start 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 0x7ff128197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281989c0 con 0x7ff128101630 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff12d67d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128198b30 con 0x7ff128100d10 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff11e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff11e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58602/0 (socket says 172.21.15.110:58602) 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.397+0000 7ff11e7fc640 1 -- 172.21.15.110:0/3313744678 learned_addr learned my addr 172.21.15.110:0/3313744678 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff126ffd640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 0x7ff128197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff11e7fc640 1 -- 172.21.15.110:0/3313744678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 msgr2=0x7ff128197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff11e7fc640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 0x7ff128197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff11e7fc640 1 -- 172.21.15.110:0/3313744678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009560 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff126ffd640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 0x7ff128197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff11e7fc640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c8a0 tx=0x7ff11000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff110007b80 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff110007ce0 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12819c910 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff110018810 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.398+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12819ce60 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.399+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff128068c80 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.400+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7ff110018a80 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.400+0000 7ff124ff9640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 0x7ff11407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.400+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7ff11009e710 con 0x7ff128101630 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.400+0000 7ff126ffd640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 0x7ff11407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.401+0000 7ff126ffd640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 0x7ff11407d740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff118006010 tx=0x7ff118005e60 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:55.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.404+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7ff110061550 con 0x7ff128101630 2024-09-21T04:26:55.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.619+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7ff128109580 con 0x7ff128101630 2024-09-21T04:26:55.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.620+0000 7ff124ff9640 1 -- 172.21.15.110:0/3313744678 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+5052 (secure 0 0 0) 0x7ff110064670 con 0x7ff128101630 2024-09-21T04:26:55.623 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:55.623 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"btime":"2024-09-21T04:22:15:001098+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:12.995969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi148.tpueww","rank":0,"incarnation":19,"state":"up:active","state_seq":145,"addr":"172.21.15.148:6825/509765299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":509765299},{"type":"v1","addr":"172.21.15.148:6825","nonce":509765299}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24295,"qdb_cluster":[24295]},"id":1}]} 2024-09-21T04:26:55.623 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 23 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 msgr2=0x7ff11407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 0x7ff11407d740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff118006010 tx=0x7ff118005e60 comp rx=0 tx=0).stop 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 msgr2=0x7ff1281983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c8a0 tx=0x7ff11000cd60 comp rx=0 tx=0).stop 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 shutdown_connections 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7ff11407b280 0x7ff11407d740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff128101630 0x7ff1281983c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 --2- 172.21.15.110:0/3313744678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff128100d10 0x7ff128197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:55.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 >> 172.21.15.110:0/3313744678 conn(0x7ff1280fa6d0 msgr2=0x7ff1280fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:55.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 shutdown_connections 2024-09-21T04:26:55.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:55.622+0000 7ff12d67d640 1 -- 172.21.15.110:0/3313744678 wait complete. 2024-09-21T04:26:55.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:55 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3313744678' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-21T04:26:56.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:55 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3313744678' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-21T04:26:56.303 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 24 2024-09-21T04:26:56.514 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:56.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:56 smithi110 ceph-mon[92455]: pgmap v335: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:57.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:56 smithi148 ceph-mon[78701]: pgmap v335: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 -- 172.21.15.110:0/3517146287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 msgr2=0x7f10280fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 --2- 172.21.15.110:0/3517146287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280fee50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1010009880 tx=0x7f101002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 -- 172.21.15.110:0/3517146287 shutdown_connections 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 --2- 172.21.15.110:0/3517146287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280fee50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 --2- 172.21.15.110:0/3517146287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 0x7f1028104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.051+0000 7f102d228640 1 -- 172.21.15.110:0/3517146287 >> 172.21.15.110:0/3517146287 conn(0x7f10280fa6d0 msgr2=0x7f10280fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.052+0000 7f102d228640 1 -- 172.21.15.110:0/3517146287 shutdown_connections 2024-09-21T04:26:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.052+0000 7f102d228640 1 -- 172.21.15.110:0/3517146287 wait complete. 2024-09-21T04:26:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.052+0000 7f102d228640 1 Processor -- start 2024-09-21T04:26:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.052+0000 7f102d228640 1 -- start start 2024-09-21T04:26:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f102d228640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f102d228640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 0x7f10280fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f102d228640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280fba00 con 0x7f10280fe9f0 2024-09-21T04:26:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f102d228640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280fbb70 con 0x7f10281049f0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58616/0 (socket says 172.21.15.110:58616) 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 -- 172.21.15.110:0/2327960886 learned_addr learned my addr 172.21.15.110:0/2327960886 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f102659c640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 0x7f10280fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 -- 172.21.15.110:0/2327960886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 msgr2=0x7f10280fb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 0x7f10280fb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 -- 172.21.15.110:0/2327960886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009560 con 0x7f10280fe9f0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f1026d9d640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f102000cc60 tx=0x7f102000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.053+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1020004b20 con 0x7f10280fe9f0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.054+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10281a1c60 con 0x7f10280fe9f0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.054+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1020004c80 con 0x7f10280fe9f0 2024-09-21T04:26:58.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.054+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102000f750 con 0x7f10280fe9f0 2024-09-21T04:26:58.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.054+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a1fc0 con 0x7f10280fe9f0 2024-09-21T04:26:58.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.055+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028100130 con 0x7f10280fe9f0 2024-09-21T04:26:58.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.055+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f102000f9c0 con 0x7f10280fe9f0 2024-09-21T04:26:58.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.056+0000 7f101ffff640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 0x7f101407d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:26:58.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.056+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f102009e600 con 0x7f10280fe9f0 2024-09-21T04:26:58.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.056+0000 7f102659c640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 0x7f101407d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:26:58.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.056+0000 7f102659c640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 0x7f101407d740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1010009230 tx=0x7f10100058d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:26:58.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.061+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f10200614f0 con 0x7f10280fe9f0 2024-09-21T04:26:58.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.279+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f10280fc4b0 con 0x7f10280fe9f0 2024-09-21T04:26:58.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.280+0000 7f101ffff640 1 -- 172.21.15.110:0/2327960886 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+4251 (secure 0 0 0) 0x7f1020064610 con 0x7f10280fe9f0 2024-09-21T04:26:58.283 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:26:58.283 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":24,"btime":"2024-09-21T04:22:26:430954+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:26.430950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:26:58.284 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 24 2024-09-21T04:26:58.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 msgr2=0x7f101407d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 0x7f101407d740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1010009230 tx=0x7f10100058d0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 msgr2=0x7f10280faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f102000cc60 tx=0x7f102000a620 comp rx=0 tx=0).stop 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 shutdown_connections 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f101407b280 0x7f101407d740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10281049f0 0x7f10280fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 --2- 172.21.15.110:0/2327960886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10280fe9f0 0x7f10280faf80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 >> 172.21.15.110:0/2327960886 conn(0x7f10280fa6d0 msgr2=0x7f1028068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 shutdown_connections 2024-09-21T04:26:58.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:26:58.282+0000 7f102d228640 1 -- 172.21.15.110:0/2327960886 wait complete. 2024-09-21T04:26:58.986 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 25 2024-09-21T04:26:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:58 smithi110 ceph-mon[92455]: pgmap v336: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:59.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:26:58 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2327960886' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-21T04:26:59.186 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:26:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:58 smithi148 ceph-mon[78701]: pgmap v336: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:26:59.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:26:58 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2327960886' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-21T04:27:00.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:00 smithi110 ceph-mon[92455]: pgmap v337: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 -- 172.21.15.110:0/3238533259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10073170 msgr2=0x7f2b101105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 --2- 172.21.15.110:0/3238533259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10073170 0x7f2b101105f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009880 tx=0x7f2b0002a530 comp rx=0 tx=0).stop 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 -- 172.21.15.110:0/3238533259 shutdown_connections 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 --2- 172.21.15.110:0/3238533259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10073170 0x7f2b101105f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 --2- 172.21.15.110:0/3238533259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10072850 0x7f2b10072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 -- 172.21.15.110:0/3238533259 >> 172.21.15.110:0/3238533259 conn(0x7f2b100fb840 msgr2=0x7f2b100fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 -- 172.21.15.110:0/3238533259 shutdown_connections 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.926+0000 7f2b181d1640 1 -- 172.21.15.110:0/3238533259 wait complete. 2024-09-21T04:27:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 Processor -- start 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 -- start start 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 0x7f2b1019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1019cd80 con 0x7f2b10072850 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b181d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b101a0af0 con 0x7f2b10073170 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58622/0 (socket says 172.21.15.110:58622) 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 -- 172.21.15.110:0/2875931634 learned_addr learned my addr 172.21.15.110:0/2875931634 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b1576c640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 0x7f2b1019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 -- 172.21.15.110:0/2875931634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 msgr2=0x7f2b1019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 0x7f2b1019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b15f6d640 1 -- 172.21.15.110:0/2875931634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00009560 con 0x7f2b10072850 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.927+0000 7f2b1576c640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 0x7f2b1019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b15f6d640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c810 tx=0x7f2b0c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:00.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c0041f0 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b0c007e70 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b101a0dd0 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c0049d0 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.928+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b101a1320 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.929+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b10073f90 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.929+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f2b0c0079b0 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.930+0000 7f2b06ffd640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 0x7f2afc07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.930+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f2b0c09e320 con 0x7f2b10072850 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.930+0000 7f2b1576c640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 0x7f2afc07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.930+0000 7f2b1576c640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 0x7f2afc07d730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006010 tx=0x7f2b00031400 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:00.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:00.934+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f2b0c061160 con 0x7f2b10072850 2024-09-21T04:27:01.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.148+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f2b10072c30 con 0x7f2b10072850 2024-09-21T04:27:01.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.149+0000 7f2b06ffd640 1 -- 172.21.15.110:0/2875931634 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4262 (secure 0 0 0) 0x7f2b0c064280 con 0x7f2b10072850 2024-09-21T04:27:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:01.152 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":25,"btime":"2024-09-21T04:22:26:435862+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:26.435849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24303":{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:01.152 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 25 2024-09-21T04:27:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 msgr2=0x7f2afc07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 0x7f2afc07d730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006010 tx=0x7f2b00031400 comp rx=0 tx=0).stop 2024-09-21T04:27:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 msgr2=0x7f2b1019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c810 tx=0x7f2b0c00ccd0 comp rx=0 tx=0).stop 2024-09-21T04:27:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 shutdown_connections 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f2afc07b270 0x7f2afc07d730 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b10073170 0x7f2b1019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 --2- 172.21.15.110:0/2875931634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b10072850 0x7f2b1019c1b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 >> 172.21.15.110:0/2875931634 conn(0x7f2b100fb840 msgr2=0x7f2b100ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 shutdown_connections 2024-09-21T04:27:01.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:01.151+0000 7f2b181d1640 1 -- 172.21.15.110:0/2875931634 wait complete. 2024-09-21T04:27:01.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:00 smithi148 ceph-mon[78701]: pgmap v337: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:01.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 26 2024-09-21T04:27:02.019 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:02.139 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:01 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2875931634' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-21T04:27:02.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:01 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2875931634' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-21T04:27:03.149 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:02 smithi110 ceph-mon[92455]: pgmap v338: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:03.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:02 smithi148 ceph-mon[78701]: pgmap v338: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:03.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- 172.21.15.110:0/3542807667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098101600 msgr2=0x7fc098109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 --2- 172.21.15.110:0/3542807667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098101600 0x7fc098109af0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc080009880 tx=0x7fc08002a530 comp rx=0 tx=0).stop 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- 172.21.15.110:0/3542807667 shutdown_connections 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 --2- 172.21.15.110:0/3542807667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098101600 0x7fc098109af0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 --2- 172.21.15.110:0/3542807667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098100ce0 0x7fc0981010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- 172.21.15.110:0/3542807667 >> 172.21.15.110:0/3542807667 conn(0x7fc0980758c0 msgr2=0x7fc0980fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- 172.21.15.110:0/3542807667 shutdown_connections 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- 172.21.15.110:0/3542807667 wait complete. 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 Processor -- start 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.774+0000 7fc09dc24640 1 -- start start 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc09dc24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:03.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc09dc24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 0x7fc0980fee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc09dc24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980ff3c0 con 0x7fc098100ce0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc09dc24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980ff530 con 0x7fc098101600 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35258/0 (socket says 172.21.15.110:35258) 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 -- 172.21.15.110:0/257900936 learned_addr learned my addr 172.21.15.110:0/257900936 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc08ffff640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 0x7fc0980fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 -- 172.21.15.110:0/257900936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 msgr2=0x7fc0980fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 0x7fc0980fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 -- 172.21.15.110:0/257900936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009560 con 0x7fc098100ce0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc08ffff640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 0x7fc0980fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0977fe640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc08800d6f0 tx=0x7fc08800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc088009a30 con 0x7fc098100ce0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0981a1cb0 con 0x7fc098100ce0 2024-09-21T04:27:03.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0880054e0 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.775+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc088004e40 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.776+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0981a2010 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.776+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098068440 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.777+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fc088009b90 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.777+0000 7fc0957fa640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 0x7fc08407d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.778+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fc08809e040 con 0x7fc098100ce0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.778+0000 7fc08ffff640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 0x7fc08407d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.778+0000 7fc08ffff640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 0x7fc08407d6e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc080031440 tx=0x7fc080005d50 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:03.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.782+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fc088060e80 con 0x7fc098100ce0 2024-09-21T04:27:03.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.996+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fc09810cd30 con 0x7fc098100ce0 2024-09-21T04:27:04.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.997+0000 7fc0957fa640 1 -- 172.21.15.110:0/257900936 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4267 (secure 0 0 0) 0x7fc088063fa0 con 0x7fc098100ce0 2024-09-21T04:27:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:04.000 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":26,"btime":"2024-09-21T04:22:33:982187+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:33.669280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24303":{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":0,"incarnation":25,"state":"up:reconnect","state_seq":147,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:04.000 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 26 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 msgr2=0x7fc08407d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 0x7fc08407d6e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc080031440 tx=0x7fc080005d50 comp rx=0 tx=0).stop 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 msgr2=0x7fc0980fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc08800d6f0 tx=0x7fc08800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 shutdown_connections 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fc08407b220 0x7fc08407d6e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:04.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098101600 0x7fc0980fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:04.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 --2- 172.21.15.110:0/257900936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc098100ce0 0x7fc0980fe940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:04.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 >> 172.21.15.110:0/257900936 conn(0x7fc0980758c0 msgr2=0x7fc098101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:04.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 shutdown_connections 2024-09-21T04:27:04.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:03.999+0000 7fc09dc24640 1 -- 172.21.15.110:0/257900936 wait complete. 2024-09-21T04:27:04.591 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 27 2024-09-21T04:27:04.793 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:05.151 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:04 smithi110 ceph-mon[92455]: pgmap v339: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:05.151 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:04 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/257900936' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-21T04:27:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:04 smithi148 ceph-mon[78701]: pgmap v339: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:05.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:04 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/257900936' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-21T04:27:06.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 -- 172.21.15.110:0/1663560081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 msgr2=0x7fb908101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:06.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 --2- 172.21.15.110:0/1663560081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908101130 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb904009880 tx=0x7fb90402a530 comp rx=0 tx=0).stop 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 -- 172.21.15.110:0/1663560081 shutdown_connections 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 --2- 172.21.15.110:0/1663560081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb908109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 --2- 172.21.15.110:0/1663560081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908101130 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 -- 172.21.15.110:0/1663560081 >> 172.21.15.110:0/1663560081 conn(0x7fb9080fa7b0 msgr2=0x7fb9080fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 -- 172.21.15.110:0/1663560081 shutdown_connections 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.389+0000 7fb90f619640 1 -- 172.21.15.110:0/1663560081 wait complete. 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 Processor -- start 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 -- start start 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb9081982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908198930 con 0x7fb908100d50 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90f619640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90819c6a0 con 0x7fb908101670 2024-09-21T04:27:06.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35286/0 (socket says 172.21.15.110:35286) 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 -- 172.21.15.110:0/1563335052 learned_addr learned my addr 172.21.15.110:0/1563335052 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 -- 172.21.15.110:0/1563335052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 msgr2=0x7fb9081982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90cbb4640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb9081982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb9081982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 -- 172.21.15.110:0/1563335052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904009560 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90cbb4640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb9081982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.390+0000 7fb90d3b5640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb90402aa50 tx=0x7fb9040055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.391+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb904004aa0 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.391+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb904004c00 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.391+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90819c920 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.391+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb904003ab0 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.391+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90819ce10 con 0x7fb908100d50 2024-09-21T04:27:06.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.392+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb908068ce0 con 0x7fb908100d50 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.392+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fb90402ad50 con 0x7fb908100d50 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.393+0000 7fb8fa7fc640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 0x7fb8f407d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.393+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fb9040bf360 con 0x7fb908100d50 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.393+0000 7fb90cbb4640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 0x7fb8f407d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.393+0000 7fb90cbb4640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 0x7fb8f407d730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb908199310 tx=0x7fb8fc005dd0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:06.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.398+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fb9040821a0 con 0x7fb908100d50 2024-09-21T04:27:06.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.613+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fb9081990c0 con 0x7fb908100d50 2024-09-21T04:27:06.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.614+0000 7fb8fa7fc640 1 -- 172.21.15.110:0/1563335052 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+4264 (secure 0 0 0) 0x7fb9040852c0 con 0x7fb908100d50 2024-09-21T04:27:06.617 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:06.617 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-21T04:22:34:984146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:33.988109+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24303":{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":0,"incarnation":25,"state":"up:rejoin","state_seq":148,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:06.618 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-21T04:27:06.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.616+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 msgr2=0x7fb8f407d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:06.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.616+0000 7fb90f619640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 0x7fb8f407d730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb908199310 tx=0x7fb8fc005dd0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.616+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 msgr2=0x7fb908197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.616+0000 7fb90f619640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb90402aa50 tx=0x7fb9040055a0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 shutdown_connections 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fb8f407b270 0x7fb8f407d730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb908101670 0x7fb9081982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 --2- 172.21.15.110:0/1563335052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb908100d50 0x7fb908197d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 >> 172.21.15.110:0/1563335052 conn(0x7fb9080fa7b0 msgr2=0x7fb9081095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 shutdown_connections 2024-09-21T04:27:06.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:06.617+0000 7fb90f619640 1 -- 172.21.15.110:0/1563335052 wait complete. 2024-09-21T04:27:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:06 smithi110 ceph-mon[92455]: pgmap v340: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:06.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:06 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/1563335052' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-21T04:27:07.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:06 smithi148 ceph-mon[78701]: pgmap v340: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:07.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:06 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/1563335052' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-21T04:27:07.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 28 2024-09-21T04:27:07.428 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 -- 172.21.15.110:0/2452629842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 msgr2=0x7f3bc0101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 --2- 172.21.15.110:0/2452629842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc0101040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c20 tx=0x7f3bb002aa00 comp rx=0 tx=0).stop 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 -- 172.21.15.110:0/2452629842 shutdown_connections 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 --2- 172.21.15.110:0/2452629842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 0x7f3bc0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 --2- 172.21.15.110:0/2452629842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc0101040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 -- 172.21.15.110:0/2452629842 >> 172.21.15.110:0/2452629842 conn(0x7f3bc00758c0 msgr2=0x7f3bc00fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 -- 172.21.15.110:0/2452629842 shutdown_connections 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 -- 172.21.15.110:0/2452629842 wait complete. 2024-09-21T04:27:09.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.013+0000 7f3bc8521640 1 Processor -- start 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 -- start start 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 0x7f3bc019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc019c3a0 con 0x7f3bc0100c60 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0195930 con 0x7f3bc0101580 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35296/0 (socket says 172.21.15.110:35296) 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 -- 172.21.15.110:0/3127907590 learned_addr learned my addr 172.21.15.110:0/3127907590 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc5abc640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 0x7f3bc019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 -- 172.21.15.110:0/3127907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 msgr2=0x7f3bc019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 0x7f3bc019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 -- 172.21.15.110:0/3127907590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb0009900 con 0x7f3bc0100c60 2024-09-21T04:27:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc62bd640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0006010 tx=0x7f3bb002f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb003e070 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb002e870 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb0004880 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.014+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc0195bb0 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.015+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc01960a0 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.015+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc00684c0 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.016+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f3bb000a040 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.017+0000 7f3bbf7fe640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 0x7f3ba807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.017+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f3bb00bf830 con 0x7f3bc0100c60 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.017+0000 7f3bc5abc640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 0x7f3ba807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.017+0000 7f3bc5abc640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 0x7f3ba807d740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005fd0 tx=0x7f3bb4005d00 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.021+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f3bb00826f0 con 0x7f3bc0100c60 2024-09-21T04:27:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:08 smithi110 ceph-mon[92455]: pgmap v341: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:09.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:08 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:27:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:08 smithi148 ceph-mon[78701]: pgmap v341: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:09.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:08 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:27:09.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.234+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f3bc0101040 con 0x7f3bc0100c60 2024-09-21T04:27:09.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.235+0000 7f3bbf7fe640 1 -- 172.21.15.110:0/3127907590 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+4273 (secure 0 0 0) 0x7f3bb0085810 con 0x7f3bc0100c60 2024-09-21T04:27:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:09.238 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":28,"btime":"2024-09-21T04:22:35:989012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:35.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24303":{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":0,"incarnation":25,"state":"up:active","state_seq":149,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24303,"qdb_cluster":[24303]},"id":1}]} 2024-09-21T04:27:09.238 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 28 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 msgr2=0x7f3ba807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 0x7f3ba807d740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4005fd0 tx=0x7f3bb4005d00 comp rx=0 tx=0).stop 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 msgr2=0x7f3bc019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0006010 tx=0x7f3bb002f870 comp rx=0 tx=0).stop 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 shutdown_connections 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f3ba807b280 0x7f3ba807d740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc0101580 0x7f3bc019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 --2- 172.21.15.110:0/3127907590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bc0100c60 0x7f3bc019b840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 >> 172.21.15.110:0/3127907590 conn(0x7f3bc00758c0 msgr2=0x7f3bc0101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:09.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 shutdown_connections 2024-09-21T04:27:09.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:09.237+0000 7f3bc8521640 1 -- 172.21.15.110:0/3127907590 wait complete. 2024-09-21T04:27:09.884 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 29 2024-09-21T04:27:10.083 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:10.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:09 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3127907590' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-21T04:27:10.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:09 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3127907590' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-21T04:27:11.155 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:10 smithi110 ceph-mon[92455]: pgmap v342: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:11.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:10 smithi148 ceph-mon[78701]: pgmap v342: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 -- 172.21.15.110:0/3382050941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 msgr2=0x7f8888109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 --2- 172.21.15.110:0/3382050941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f8888109a70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f887c009880 tx=0x7f887c02a530 comp rx=0 tx=0).stop 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 -- 172.21.15.110:0/3382050941 shutdown_connections 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 --2- 172.21.15.110:0/3382050941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f8888109a70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 --2- 172.21.15.110:0/3382050941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 0x7f8888101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 -- 172.21.15.110:0/3382050941 >> 172.21.15.110:0/3382050941 conn(0x7f88880758c0 msgr2=0x7f88880fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 -- 172.21.15.110:0/3382050941 shutdown_connections 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.659+0000 7f888e611640 1 -- 172.21.15.110:0/3382050941 wait complete. 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 Processor -- start 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 -- start start 2024-09-21T04:27:11.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 0x7f88880fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880ff340 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f888e611640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880ff4b0 con 0x7f8888100c60 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35312/0 (socket says 172.21.15.110:35312) 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 -- 172.21.15.110:0/3505313417 learned_addr learned my addr 172.21.15.110:0/3505313417 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f8887fff640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 0x7f88880fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 -- 172.21.15.110:0/3505313417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 msgr2=0x7f88880fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 0x7f88880fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.660+0000 7f88877fe640 1 -- 172.21.15.110:0/3505313417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c009560 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f88877fe640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f887c02aa50 tx=0x7f887c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c0048d0 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f887c031640 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88880ff730 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c0038d0 con 0x7f8888101580 2024-09-21T04:27:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.661+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88881a2010 con 0x7f8888101580 2024-09-21T04:27:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.662+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88880684c0 con 0x7f8888101580 2024-09-21T04:27:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.662+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f887c02ad50 con 0x7f8888101580 2024-09-21T04:27:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.663+0000 7f88857fa640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 0x7f887007d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.663+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f887c0bf010 con 0x7f8888101580 2024-09-21T04:27:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.663+0000 7f8887fff640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 0x7f887007d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.663+0000 7f8887fff640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 0x7f887007d740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8878005fd0 tx=0x7f8878005850 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:11.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.667+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f887c081ed0 con 0x7f8888101580 2024-09-21T04:27:11.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.879+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f88880ff730 con 0x7f8888101580 2024-09-21T04:27:11.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.880+0000 7f88857fa640 1 -- 172.21.15.110:0/3505313417 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+5123 (secure 0 0 0) 0x7f88880ff730 con 0x7f8888101580 2024-09-21T04:27:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:11.883 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":29,"btime":"2024-09-21T04:22:38:000614+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:35.989010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24303":{"gid":24303,"name":"cephfs.smithi148.fpatxi","rank":0,"incarnation":25,"state":"up:active","state_seq":149,"addr":"172.21.15.148:6827/475239845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":475239845},{"type":"v1","addr":"172.21.15.148:6827","nonce":475239845}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24303,"qdb_cluster":[24303]},"id":1}]} 2024-09-21T04:27:11.883 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 29 2024-09-21T04:27:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 msgr2=0x7f887007d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 0x7f887007d740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8878005fd0 tx=0x7f8878005850 comp rx=0 tx=0).stop 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 msgr2=0x7f88880fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f887c02aa50 tx=0x7f887c0055a0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 shutdown_connections 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f887007b280 0x7f887007d740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888101580 0x7f88880fee00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 --2- 172.21.15.110:0/3505313417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8888100c60 0x7f88880fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.882+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 >> 172.21.15.110:0/3505313417 conn(0x7f88880758c0 msgr2=0x7f8888101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.883+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 shutdown_connections 2024-09-21T04:27:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:11.883+0000 7f888e611640 1 -- 172.21.15.110:0/3505313417 wait complete. 2024-09-21T04:27:12.534 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 30 2024-09-21T04:27:12.734 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:13.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:12 smithi110 ceph-mon[92455]: pgmap v343: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:13.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:12 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3505313417' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-21T04:27:13.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:12 smithi148 ceph-mon[78701]: pgmap v343: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:13.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:12 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3505313417' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 -- 172.21.15.110:0/3099334708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 msgr2=0x7f45d00fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 --2- 172.21.15.110:0/3099334708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d00fee80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009880 tx=0x7f45c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 -- 172.21.15.110:0/3099334708 shutdown_connections 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 --2- 172.21.15.110:0/3099334708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d00fee80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 --2- 172.21.15.110:0/3099334708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 0x7f45d0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 -- 172.21.15.110:0/3099334708 >> 172.21.15.110:0/3099334708 conn(0x7f45d00fa720 msgr2=0x7f45d00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 -- 172.21.15.110:0/3099334708 shutdown_connections 2024-09-21T04:27:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.334+0000 7f45d7743640 1 -- 172.21.15.110:0/3099334708 wait complete. 2024-09-21T04:27:14.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.335+0000 7f45d7743640 1 Processor -- start 2024-09-21T04:27:14.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.335+0000 7f45d7743640 1 -- start start 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 0x7f45d0194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01955e0 con 0x7f45d00fea20 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0199330 con 0x7f45d0104a20 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:14.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47400/0 (socket says 172.21.15.110:47400) 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 -- 172.21.15.110:0/3401070036 learned_addr learned my addr 172.21.15.110:0/3401070036 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d4cde640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 0x7f45d0194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 -- 172.21.15.110:0/3401070036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 msgr2=0x7f45d0194f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 0x7f45d0194f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 -- 172.21.15.110:0/3401070036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009560 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d54df640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6f0 tx=0x7f45cc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45cc009a30 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45cc0054e0 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45cc010460 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d0199610 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.336+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0199b30 con 0x7f45d00fea20 2024-09-21T04:27:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.338+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d00fce20 con 0x7f45d00fea20 2024-09-21T04:27:14.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.338+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f45cc009b90 con 0x7f45d00fea20 2024-09-21T04:27:14.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.339+0000 7f45c67fc640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 0x7f45bc07d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:14.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.339+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f45cc09df20 con 0x7f45d00fea20 2024-09-21T04:27:14.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.339+0000 7f45d4cde640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 0x7f45bc07d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:14.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.339+0000 7f45d4cde640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 0x7f45bc07d730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009230 tx=0x7f45c00058d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:14.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.343+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f45cc0d5710 con 0x7f45d00fea20 2024-09-21T04:27:14.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.559+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f45d0100160 con 0x7f45d00fea20 2024-09-21T04:27:14.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.560+0000 7f45c67fc640 1 -- 172.21.15.110:0/3401070036 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+4322 (secure 0 0 0) 0x7f45cc060e90 con 0x7f45d00fea20 2024-09-21T04:27:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:14.563 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":30,"btime":"2024-09-21T04:22:47:369388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:47.369385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:14.563 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 30 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 msgr2=0x7f45bc07d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 0x7f45bc07d730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009230 tx=0x7f45c00058d0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 msgr2=0x7f45d01949c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6f0 tx=0x7f45cc00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 shutdown_connections 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f45bc07b270 0x7f45bc07d730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45d0104a20 0x7f45d0194f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 --2- 172.21.15.110:0/3401070036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45d00fea20 0x7f45d01949c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 >> 172.21.15.110:0/3401070036 conn(0x7f45d00fa720 msgr2=0x7f45d00fb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 shutdown_connections 2024-09-21T04:27:14.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:14.562+0000 7f45d7743640 1 -- 172.21.15.110:0/3401070036 wait complete. 2024-09-21T04:27:15.151 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:14 smithi110 ceph-mon[92455]: pgmap v344: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:15.152 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:14 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3401070036' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-21T04:27:15.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 31 2024-09-21T04:27:15.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:14 smithi148 ceph-mon[78701]: pgmap v344: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:15.191 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:14 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3401070036' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-21T04:27:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:16.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:16 smithi110 ceph-mon[92455]: pgmap v345: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:17.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 -- 172.21.15.110:0/4290857589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 msgr2=0x7f6c540fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:17.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/4290857589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c540fee80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009880 tx=0x7f6c4402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 -- 172.21.15.110:0/4290857589 shutdown_connections 2024-09-21T04:27:17.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/4290857589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c540fee80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/4290857589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c54104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 -- 172.21.15.110:0/4290857589 >> 172.21.15.110:0/4290857589 conn(0x7f6c540fa720 msgr2=0x7f6c540fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:17.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 -- 172.21.15.110:0/4290857589 shutdown_connections 2024-09-21T04:27:17.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.049+0000 7f6c5bc06640 1 -- 172.21.15.110:0/4290857589 wait complete. 2024-09-21T04:27:17.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 Processor -- start 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 -- start start 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c5419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54194930 con 0x7f6c540fea20 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c5bc06640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54194a70 con 0x7f6c54104a20 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c599a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c599a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47412/0 (socket says 172.21.15.110:47412) 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c599a2640 1 -- 172.21.15.110:0/2114210300 learned_addr learned my addr 172.21.15.110:0/2114210300 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.050+0000 7f6c591a1640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c5419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c599a2640 1 -- 172.21.15.110:0/2114210300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 msgr2=0x7f6c5419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c599a2640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c5419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c599a2640 1 -- 172.21.15.110:0/2114210300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44009560 con 0x7f6c540fea20 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c591a1640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c5419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:27:17.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c599a2640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000b420 tx=0x7f6c5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c50011800 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54194d50 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c50011e20 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5000f2e0 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.051+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c54195270 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.052+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c54100160 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.053+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f6c50010380 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.053+0000 7f6c4affd640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 0x7f6c4007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.053+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f6c5009dd40 con 0x7f6c540fea20 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.053+0000 7f6c591a1640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 0x7f6c4007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.054+0000 7f6c591a1640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 0x7f6c4007d730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44005a20 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:17.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.058+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f6c50060c30 con 0x7f6c540fea20 2024-09-21T04:27:17.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:16 smithi148 ceph-mon[78701]: pgmap v345: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:17.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.268+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f6c54106fe0 con 0x7f6c540fea20 2024-09-21T04:27:17.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.269+0000 7f6c4affd640 1 -- 172.21.15.110:0/2114210300 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+4401 (secure 0 0 0) 0x7f6c50063d50 con 0x7f6c540fea20 2024-09-21T04:27:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:17.272 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":31,"btime":"2024-09-21T04:22:47:375015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:47.374997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:17.272 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 31 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 msgr2=0x7f6c4007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 0x7f6c4007d730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44005a20 comp rx=0 tx=0).stop 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 msgr2=0x7f6c5419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000b420 tx=0x7f6c5000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 shutdown_connections 2024-09-21T04:27:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f6c4007b270 0x7f6c4007d730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c54104a20 0x7f6c5419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 --2- 172.21.15.110:0/2114210300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c540fea20 0x7f6c5419a8d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 >> 172.21.15.110:0/2114210300 conn(0x7f6c540fa720 msgr2=0x7f6c540fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 shutdown_connections 2024-09-21T04:27:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:17.271+0000 7f6c5bc06640 1 -- 172.21.15.110:0/2114210300 wait complete. 2024-09-21T04:27:17.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 32 2024-09-21T04:27:18.107 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:18.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:17 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2114210300' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-21T04:27:18.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:17 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2114210300' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-21T04:27:19.149 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:18 smithi110 ceph-mon[92455]: pgmap v346: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:19.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:18 smithi148 ceph-mon[78701]: pgmap v346: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 -- 172.21.15.110:0/1790543863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 msgr2=0x7f7db4104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 --2- 172.21.15.110:0/1790543863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db4104e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009880 tx=0x7f7da402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 -- 172.21.15.110:0/1790543863 shutdown_connections 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 --2- 172.21.15.110:0/1790543863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 0x7f7db40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 --2- 172.21.15.110:0/1790543863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db4104e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 -- 172.21.15.110:0/1790543863 >> 172.21.15.110:0/1790543863 conn(0x7f7db40fa720 msgr2=0x7f7db40fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 -- 172.21.15.110:0/1790543863 shutdown_connections 2024-09-21T04:27:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.661+0000 7f7db8a60640 1 -- 172.21.15.110:0/1790543863 wait complete. 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.662+0000 7f7db8a60640 1 Processor -- start 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.662+0000 7f7db8a60640 1 -- start start 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.662+0000 7f7db8a60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 0x7f7db419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.662+0000 7f7db8a60640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7db8a60640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419b380 con 0x7f7db4104a20 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7db8a60640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4194910 con 0x7f7db40fea20 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47436/0 (socket says 172.21.15.110:47436) 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 -- 172.21.15.110:0/514619691 learned_addr learned my addr 172.21.15.110:0/514619691 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7db259c640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 0x7f7db419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 -- 172.21.15.110:0/514619691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 msgr2=0x7f7db419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 0x7f7db419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 -- 172.21.15.110:0/514619691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da4009560 con 0x7f7db4104a20 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7da9d9b640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d910 tx=0x7f7d9c00ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c00ed40 con 0x7f7db4104a20 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4194b90 con 0x7f7db4104a20 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d9c016900 con 0x7f7db4104a20 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c014990 con 0x7f7db4104a20 2024-09-21T04:27:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.663+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db41950e0 con 0x7f7db4104a20 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.664+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db4100160 con 0x7f7db4104a20 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.665+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f7d9c016440 con 0x7f7db4104a20 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.665+0000 7f7dabfff640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 0x7f7da007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.666+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f7d9c09fe50 con 0x7f7db4104a20 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.666+0000 7f7db259c640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 0x7f7da007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.666+0000 7f7db259c640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 0x7f7da007d730 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009230 tx=0x7f7da40058d0 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:19.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.670+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f7d9c062d40 con 0x7f7db4104a20 2024-09-21T04:27:19.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.883+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f7db410bd10 con 0x7f7db4104a20 2024-09-21T04:27:19.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.883+0000 7f7dabfff640 1 -- 172.21.15.110:0/514619691 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+4405 (secure 0 0 0) 0x7f7d9c065e60 con 0x7f7db4104a20 2024-09-21T04:27:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:19.887 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":32,"btime":"2024-09-21T04:22:54:919331+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:54.593225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:19.887 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 32 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 msgr2=0x7f7da007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 0x7f7da007d730 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009230 tx=0x7f7da40058d0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 msgr2=0x7f7db419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d910 tx=0x7f7d9c00ddd0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 shutdown_connections 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f7da007b270 0x7f7da007d730 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7db4104a20 0x7f7db419ad60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 --2- 172.21.15.110:0/514619691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db40fea20 0x7f7db419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:19.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 >> 172.21.15.110:0/514619691 conn(0x7f7db40fa720 msgr2=0x7f7db40fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:19.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 shutdown_connections 2024-09-21T04:27:19.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:19.886+0000 7f7db8a60640 1 -- 172.21.15.110:0/514619691 wait complete. 2024-09-21T04:27:20.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 33 2024-09-21T04:27:21.036 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:21.158 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:20 smithi110 ceph-mon[92455]: pgmap v347: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:21.158 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:20 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/514619691' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-21T04:27:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:20 smithi148 ceph-mon[78701]: pgmap v347: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:21.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:20 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/514619691' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-21T04:27:22.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.670+0000 7fbb56445640 1 -- 172.21.15.110:0/3095033812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 msgr2=0x7fbb501010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:22.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.670+0000 7fbb56445640 1 --2- 172.21.15.110:0/3095033812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb501010c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009880 tx=0x7fbb4002a530 comp rx=0 tx=0).stop 2024-09-21T04:27:22.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- 172.21.15.110:0/3095033812 shutdown_connections 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 --2- 172.21.15.110:0/3095033812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 --2- 172.21.15.110:0/3095033812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb501010c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- 172.21.15.110:0/3095033812 >> 172.21.15.110:0/3095033812 conn(0x7fbb500fa720 msgr2=0x7fbb500fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- 172.21.15.110:0/3095033812 shutdown_connections 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- 172.21.15.110:0/3095033812 wait complete. 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 Processor -- start 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- start start 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501988f0 con 0x7fbb50100ce0 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.671+0000 7fbb56445640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5019b560 con 0x7fbb50101600 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:22.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50670/0 (socket says 172.21.15.110:50670) 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 -- 172.21.15.110:0/2982474147 learned_addr learned my addr 172.21.15.110:0/2982474147 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4f7fe640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 -- 172.21.15.110:0/2982474147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 msgr2=0x7fbb50198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 -- 172.21.15.110:0/2982474147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40009560 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4f7fe640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4ffff640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb40006010 tx=0x7fbb400055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb40004aa0 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5019b7e0 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb40031470 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb40003940 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.672+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5019bcd0 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.673+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50106360 con 0x7fbb50100ce0 2024-09-21T04:27:22.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.674+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fbb4002ad50 con 0x7fbb50100ce0 2024-09-21T04:27:22.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.675+0000 7fbb4d7fa640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 0x7fbb3807d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:22.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.675+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fbb400bf3c0 con 0x7fbb50100ce0 2024-09-21T04:27:22.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.675+0000 7fbb4f7fe640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 0x7fbb3807d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:22.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.675+0000 7fbb4f7fe640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 0x7fbb3807d730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb50199240 tx=0x7fbb44009670 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:22.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.679+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fbb40082200 con 0x7fbb50100ce0 2024-09-21T04:27:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.894+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fbb500ff9d0 con 0x7fbb50100ce0 2024-09-21T04:27:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.895+0000 7fbb4d7fa640 1 -- 172.21.15.110:0/2982474147 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+4402 (secure 0 0 0) 0x7fbb40085320 con 0x7fbb50100ce0 2024-09-21T04:27:22.898 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T04:27:22.898 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":33,"btime":"2024-09-21T04:22:55:923439+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi110.faxwju","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/3347525595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":3347525595},{"type":"v1","addr":"172.21.15.110:6829","nonce":3347525595}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44293,"name":"cephfs.smithi148.tpueww","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/2751562686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2751562686},{"type":"v1","addr":"172.21.15.148:6825","nonce":2751562686}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T04:12:39.437314+0000","modified":"2024-09-21T04:22:54.928001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.trhyxr","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.110:6827/2012701813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2012701813},{"type":"v1","addr":"172.21.15.110:6827","nonce":2012701813}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T04:27:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 33 2024-09-21T04:27:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 msgr2=0x7fbb3807d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 0x7fbb3807d730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb50199240 tx=0x7fbb44009670 comp rx=0 tx=0).stop 2024-09-21T04:27:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 msgr2=0x7fbb50197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb40006010 tx=0x7fbb400055a0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 shutdown_connections 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fbb3807b270 0x7fbb3807d730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb50101600 0x7fbb50198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 --2- 172.21.15.110:0/2982474147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50100ce0 0x7fbb50197cd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 >> 172.21.15.110:0/2982474147 conn(0x7fbb500fa720 msgr2=0x7fbb501090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 shutdown_connections 2024-09-21T04:27:22.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:22.897+0000 7fbb56445640 1 -- 172.21.15.110:0/2982474147 wait complete. 2024-09-21T04:27:23.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:22 smithi110 ceph-mon[92455]: pgmap v348: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:23.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:22 smithi148 ceph-mon[78701]: pgmap v348: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:23.613 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T04:27:23.626 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T04:27:23.626 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:27:23.626 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:27:23.644 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T04:27:23.644 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:27:23.701 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T04:27:23.952 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:23 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/2982474147' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-21T04:27:24.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:23 smithi110 ceph-mon[92455]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:27:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:23 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/2982474147' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-21T04:27:24.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:23 smithi148 ceph-mon[78701]: from='mgr.24563 172.21.15.110:0/690535084' entity='mgr.smithi110.axmqzb' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:27:25.133 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:24 smithi110 ceph-mon[92455]: pgmap v349: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:25.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:24 smithi148 ceph-mon[78701]: pgmap v349: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 -- 172.21.15.110:0/2148728432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c1049f0 msgr2=0x7fe53c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 --2- 172.21.15.110:0/2148728432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c1049f0 0x7fe53c104dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009c20 tx=0x7fe52c02a970 comp rx=0 tx=0).stop 2024-09-21T04:27:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 -- 172.21.15.110:0/2148728432 shutdown_connections 2024-09-21T04:27:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 --2- 172.21.15.110:0/2148728432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 --2- 172.21.15.110:0/2148728432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c1049f0 0x7fe53c104dd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 -- 172.21.15.110:0/2148728432 >> 172.21.15.110:0/2148728432 conn(0x7fe53c0fa6d0 msgr2=0x7fe53c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 -- 172.21.15.110:0/2148728432 shutdown_connections 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 -- 172.21.15.110:0/2148728432 wait complete. 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.480+0000 7fe5410bc640 1 Processor -- start 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe5410bc640 1 -- start start 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe5410bc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe5410bc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 0x7fe53c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe5410bc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1989a0 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe5410bc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c19c710 con 0x7fe53c1049f0 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50694/0 (socket says 172.21.15.110:50694) 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 -- 172.21.15.110:0/3516196558 learned_addr learned my addr 172.21.15.110:0/3516196558 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53a59c640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 0x7fe53c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:25.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 -- 172.21.15.110:0/3516196558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 msgr2=0x7fe53c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 0x7fe53c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53ad9d640 1 -- 172.21.15.110:0/3516196558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009900 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.481+0000 7fe53a59c640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 0x7fe53c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe53ad9d640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe52c02ae90 tx=0x7fe52c0308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52c02f590 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe52c030ca0 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c19c990 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52c039680 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.482+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c19ce80 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.483+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c100130 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.483+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7fe52c00a040 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.484+0000 7fe533fff640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 0x7fe52807d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.484+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe52c02c040 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.484+0000 7fe53a59c640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 0x7fe52807d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:25.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.484+0000 7fe53a59c640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 0x7fe52807d740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:25.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.488+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7fe52c082730 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.680+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe53c0feed0 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.680+0000 7fe533fff640 1 -- 172.21.15.110:0/3516196558 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 45 entries v95) v1 ==== 81+0+2699 (secure 0 0 0) 0x7fe52c085850 con 0x7fe53c0fe9f0 2024-09-21T04:27:25.683 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6826/475239845 2024-09-22T04:22:47.369054+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6829/1332815334 2024-09-22T04:22:04.027184+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1287159197 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1516382105 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/48669527 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6825/509765299 2024-09-22T04:22:26.430665+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/4004770870 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2502785588 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6829/2999382373 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2497023553 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1362067387 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/539492599 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/602456430 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/1166407618 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.684 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/539492599 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/833735978 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/4128248672 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/491848237 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/922271321 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/278566480 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6828/2999382373 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/521148859 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6827/475239845 2024-09-22T04:22:47.369054+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1516382105 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2874227236 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/833735978 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/2990821192 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6826/1662909837 2024-09-22T04:12:54.194977+0000 2024-09-21T04:27:25.685 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2087016304 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1919392720 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6827/1662909837 2024-09-22T04:12:54.194977+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/3127652240 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/598619296 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3078336191 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3973251001 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1589768070 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2973624819 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6824/509765299 2024-09-22T04:22:26.430665+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3142935389 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2783964987 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1362067387 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1919392720 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1940237520 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:25.686 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6828/1332815334 2024-09-22T04:22:04.027184+0000 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1485849066 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:listed 45 entries 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 msgr2=0x7fe52807d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 0x7fe52807d740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).stop 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 msgr2=0x7fe53c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe52c02ae90 tx=0x7fe52c0308a0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 shutdown_connections 2024-09-21T04:27:25.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7fe52807b280 0x7fe52807d740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe53c1049f0 0x7fe53c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 --2- 172.21.15.110:0/3516196558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe53c0fe9f0 0x7fe53c197dd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:25.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.682+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 >> 172.21.15.110:0/3516196558 conn(0x7fe53c0fa6d0 msgr2=0x7fe53c0fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:25.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.683+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 shutdown_connections 2024-09-21T04:27:25.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:25.683+0000 7fe5410bc640 1 -- 172.21.15.110:0/3516196558 wait complete. 2024-09-21T04:27:26.055 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:25 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/3516196558' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:27:26.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:25 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/3516196558' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:27:26.320 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T04:27:26.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:26.320 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T04:27:26.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:27:26.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:26.349 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T04:27:26.416 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:27:26.416 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:27:26.436 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T04:27:26.436 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:27:26.496 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 77422328-77ce-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T04:27:26.697 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/mon.smithi110/config 2024-09-21T04:27:27.068 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:26 smithi110 ceph-mon[92455]: pgmap v350: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:27.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:26 smithi148 ceph-mon[78701]: pgmap v350: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 -- 172.21.15.110:0/2934017019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 msgr2=0x7f0924105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 --2- 172.21.15.110:0/2934017019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f0924105fd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0914009880 tx=0x7f091402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 -- 172.21.15.110:0/2934017019 shutdown_connections 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 --2- 172.21.15.110:0/2934017019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 0x7f09240fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 --2- 172.21.15.110:0/2934017019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f0924105fd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 -- 172.21.15.110:0/2934017019 >> 172.21.15.110:0/2934017019 conn(0x7f09240fb840 msgr2=0x7f09240fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 -- 172.21.15.110:0/2934017019 shutdown_connections 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.137+0000 7f092ad94640 1 -- 172.21.15.110:0/2934017019 wait complete. 2024-09-21T04:27:28.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 Processor -- start 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 -- start start 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 0x7f092419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092419c350 con 0x7f0924105bf0 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f092ad94640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0924195930 con 0x7f09240ffb40 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50710/0 (socket says 172.21.15.110:50710) 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 -- 172.21.15.110:0/4021116757 learned_addr learned my addr 172.21.15.110:0/4021116757 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0928b30640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 0x7f092419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 -- 172.21.15.110:0/4021116757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 msgr2=0x7f092419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 0x7f092419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.138+0000 7f0923fff640 1 -- 172.21.15.110:0/4021116757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009560 con 0x7f0924105bf0 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f0923fff640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c8f0 tx=0x7f090c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:28.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090c007b80 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924195c10 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f090c0054e0 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090c00f450 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.139+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0924196160 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.140+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09241011f0 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.141+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 39) v1 ==== 103322+0+0 (secure 0 0 0) 0x7f090c004f60 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.141+0000 7f0921ffb640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 0x7f091007d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.141+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f090c09e380 con 0x7f0924105bf0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.141+0000 7f0928b30640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 0x7f091007d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.142+0000 7f0928b30640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 0x7f091007d730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0914000c00 tx=0x7f0914031420 comp rx=0 tx=0).ready entity=mgr.24563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:27:28.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.146+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+198400 (secure 0 0 0) 0x7f090c061160 con 0x7f0924105bf0 2024-09-21T04:27:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.336+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f09240fffa0 con 0x7f0924105bf0 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.337+0000 7f0921ffb640 1 -- 172.21.15.110:0/4021116757 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 45 entries v95) v1 ==== 81+0+2699 (secure 0 0 0) 0x7f090c064280 con 0x7f0924105bf0 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6826/475239845 2024-09-22T04:22:47.369054+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6829/1332815334 2024-09-22T04:22:04.027184+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1287159197 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1516382105 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/48669527 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6825/509765299 2024-09-22T04:22:26.430665+0000 2024-09-21T04:27:28.340 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/4004770870 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2502785588 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6829/2999382373 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2497023553 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1362067387 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/539492599 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/602456430 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/1166407618 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/539492599 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/833735978 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/4128248672 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/491848237 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/922271321 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/278566480 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.341 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6828/2999382373 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/521148859 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6827/475239845 2024-09-22T04:22:47.369054+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1516382105 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2874227236 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/833735978 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/2990821192 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6826/1662909837 2024-09-22T04:12:54.194977+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2087016304 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1919392720 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6827/1662909837 2024-09-22T04:12:54.194977+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:0/3127652240 2024-09-22T04:16:30.376875+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/598619296 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3078336191 2024-09-22T04:17:22.825122+0000 2024-09-21T04:27:28.342 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3973251001 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1589768070 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2973624819 2024-09-22T04:05:43.998468+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.148:6824/509765299 2024-09-22T04:22:26.430665+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3142935389 2024-09-22T04:07:18.131661+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2783964987 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1362067387 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1919392720 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1940237520 2024-09-22T04:15:11.260280+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6828/1332815334 2024-09-22T04:22:04.027184+0000 2024-09-21T04:27:28.343 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1485849066 2024-09-22T04:06:13.993411+0000 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:listed 45 entries 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 msgr2=0x7f091007d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 0x7f091007d730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0914000c00 tx=0x7f0914031420 comp rx=0 tx=0).stop 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 msgr2=0x7f092419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c8f0 tx=0x7f090c00cdb0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 shutdown_connections 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:6800/1486080047,v1:172.21.15.110:6801/1486080047] conn(0x7f091007b270 0x7f091007d730 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0924105bf0 0x7f092419bd30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 --2- 172.21.15.110:0/4021116757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240ffb40 0x7f092419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:27:28.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 >> 172.21.15.110:0/4021116757 conn(0x7f09240fb840 msgr2=0x7f09240fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:27:28.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 shutdown_connections 2024-09-21T04:27:28.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T04:27:28.339+0000 7f092ad94640 1 -- 172.21.15.110:0/4021116757 wait complete. 2024-09-21T04:27:29.003 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T04:27:29.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T04:27:29.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T04:27:29.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.039 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T04:27:29.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.115 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-21T04:27:29.131 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T04:27:29.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.132 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:27:29.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:28 smithi110 ceph-mon[92455]: pgmap v351: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:27:29.167 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:28 smithi110 ceph-mon[92455]: from='client.? 172.21.15.110:0/4021116757' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:27:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:28 smithi148 ceph-mon[78701]: pgmap v351: 65 pgs: 65 active+clean; 195 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-21T04:27:29.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:28 smithi148 ceph-mon[78701]: from='client.? 172.21.15.110:0/4021116757' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:27:29.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.193 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-21T04:27:29.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.313 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T04:27:29.331 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T04:27:29.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:27:29.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:27:29.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T04:27:29.499 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T04:27:29.513 INFO:tasks.cephadm:Teardown begin 2024-09-21T04:27:29.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:27:29.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:27:29.567 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T04:27:29.567 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:27:29.596 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:27:29.623 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T04:27:29.623 INFO:tasks.cephadm.mon.smithi110:Stopping mon.smithi110... 2024-09-21T04:27:29.623 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110 2024-09-21T04:27:29.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:29 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:30.260 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:29 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[92451]: 2024-09-21T04:27:29.940+0000 7f34ada97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:30.260 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:29 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110[92451]: 2024-09-21T04:27:29.940+0000 7f34ada97640 -1 mon.smithi110@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T04:27:30.260 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:29 smithi110 podman[151352]: 2024-09-21 04:27:29.96205576 +0000 UTC m=+0.145079914 container died 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T04:27:30.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:30 smithi110 podman[151352]: 2024-09-21 04:27:30.325283345 +0000 UTC m=+0.508307564 container cleanup 76f20187adf7c9d0c72d6b25f5080656b4b21bdf95f38d4dbe8b049c744ce963 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True) 2024-09-21T04:27:30.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Sep 21 04:27:30 smithi110 bash[151352]: ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi110 2024-09-21T04:27:30.754 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi110.service' 2024-09-21T04:27:30.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:27:30.808 INFO:tasks.cephadm.mon.smithi110:Stopped mon.smithi110 2024-09-21T04:27:30.808 INFO:tasks.cephadm.mon.smithi148:Stopping mon.smithi148... 2024-09-21T04:27:30.808 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148 2024-09-21T04:27:31.132 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:30 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:31.409 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148[78697]: 2024-09-21T04:27:31.129+0000 7f326c5f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:31.409 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148[78697]: 2024-09-21T04:27:31.129+0000 7f326c5f8640 -1 mon.smithi148@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T04:27:31.409 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 podman[115901]: 2024-09-21 04:27:31.181432875 +0000 UTC m=+0.186332748 container died 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T04:27:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 podman[115901]: 2024-09-21 04:27:31.638729774 +0000 UTC m=+0.643629639 container cleanup 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:27:31.690 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 bash[115901]: ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148 2024-09-21T04:27:32.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service' 2024-09-21T04:27:32.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:31 smithi148 podman[115914]: 2024-09-21 04:27:31.948945984 +0000 UTC m=+0.759319484 container remove 62be042bab1c1c8545004f0609c4d07c1b6a8dadffd39a7c5957eeea00e5bc93 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-mon-smithi148, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:27:32.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:32 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service: Deactivated successfully. 2024-09-21T04:27:32.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:32 smithi148 systemd[1]: Stopped Ceph mon.smithi148 for 77422328-77ce-11ef-baf5-efdc52797490. 2024-09-21T04:27:32.144 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Sep 21 04:27:32 smithi148 systemd[1]: ceph-77422328-77ce-11ef-baf5-efdc52797490@mon.smithi148.service: Consumed 5.812s CPU time. 2024-09-21T04:27:32.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:27:32.169 INFO:tasks.cephadm.mon.smithi148:Stopped mon.smithi148 2024-09-21T04:27:32.169 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T04:27:32.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.0 2024-09-21T04:27:32.457 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:32 smithi110 systemd[1]: Stopping Ceph osd.0 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:32.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:32 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:27:32.453+0000 7ffa67f5b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:32.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:32 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:27:32.453+0000 7ffa67f5b640 -1 osd.0 95 *** Got signal Terminated *** 2024-09-21T04:27:32.917 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:32 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0[100335]: 2024-09-21T04:27:32.453+0000 7ffa67f5b640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:27:38.083 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:37 smithi110 podman[151483]: 2024-09-21 04:27:37.80946444 +0000 UTC m=+5.450357307 container died a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T04:27:38.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:38 smithi110 podman[151483]: 2024-09-21 04:27:38.180291327 +0000 UTC m=+5.821184193 container cleanup a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T04:27:38.417 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:38 smithi110 bash[151483]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0 2024-09-21T04:27:38.807 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:38 smithi110 podman[151498]: 2024-09-21 04:27:38.452105857 +0000 UTC m=+0.645313476 container remove a67c7fde8c6b4c201f3aaf46fff91bb7bbe54f3592bc392cb3ea33e41b58c412 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:27:39.071 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:38 smithi110 podman[151590]: 2024-09-21 04:27:38.708086749 +0000 UTC m=+0.016508791 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:27:39.071 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:38 smithi110 podman[151590]: 2024-09-21 04:27:38.824705734 +0000 UTC m=+0.133127811 container create a53db7c28b8da1840beffde8be556ca37ff777aced08114d1482a012d7cef472 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T04:27:39.071 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:39 smithi110 podman[151590]: 2024-09-21 04:27:39.069150201 +0000 UTC m=+0.377572238 container init a53db7c28b8da1840beffde8be556ca37ff777aced08114d1482a012d7cef472 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T04:27:39.343 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:39 smithi110 podman[151590]: 2024-09-21 04:27:39.073014993 +0000 UTC m=+0.381437033 container start a53db7c28b8da1840beffde8be556ca37ff777aced08114d1482a012d7cef472 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T04:27:39.343 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:39 smithi110 podman[151590]: 2024-09-21 04:27:39.144002351 +0000 UTC m=+0.452424388 container attach a53db7c28b8da1840beffde8be556ca37ff777aced08114d1482a012d7cef472 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T04:27:39.343 INFO:journalctl@ceph.osd.0.smithi110.stdout:Sep 21 04:27:39 smithi110 podman[151590]: 2024-09-21 04:27:39.261935218 +0000 UTC m=+0.570357254 container died a53db7c28b8da1840beffde8be556ca37ff777aced08114d1482a012d7cef472 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2) 2024-09-21T04:27:39.934 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T04:27:39.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:27:39.977 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T04:27:39.977 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T04:27:39.977 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.1 2024-09-21T04:27:40.320 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:40 smithi110 systemd[1]: Stopping Ceph osd.1 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:40.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:40 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:27:40.317+0000 7fdb4ff96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:40.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:40 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:27:40.317+0000 7fdb4ff96640 -1 osd.1 95 *** Got signal Terminated *** 2024-09-21T04:27:40.667 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:40 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1[106006]: 2024-09-21T04:27:40.317+0000 7fdb4ff96640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:27:45.917 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:45 smithi110 podman[151694]: 2024-09-21 04:27:45.64856788 +0000 UTC m=+5.453824480 container died f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:27:46.288 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:45 smithi110 podman[151694]: 2024-09-21 04:27:45.99297354 +0000 UTC m=+5.798230139 container cleanup f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T04:27:46.288 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:45 smithi110 bash[151694]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1 2024-09-21T04:27:46.605 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151722]: 2024-09-21 04:27:46.28617052 +0000 UTC m=+0.637368526 container remove f054d20509ea502a92b8457c2bff795df23da6aaab0d3ddff124b728877cbc47 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T04:27:46.900 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151812]: 2024-09-21 04:27:46.506447572 +0000 UTC m=+0.016307042 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:27:46.900 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151812]: 2024-09-21 04:27:46.604106222 +0000 UTC m=+0.113965681 container create 00aa5bbee0c3e2fd8d583df5da1dc36a115b6abef57607cbcb97333f2b30b237 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T04:27:46.900 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151812]: 2024-09-21 04:27:46.815126387 +0000 UTC m=+0.324985855 container init 00aa5bbee0c3e2fd8d583df5da1dc36a115b6abef57607cbcb97333f2b30b237 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:27:46.900 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151812]: 2024-09-21 04:27:46.818577608 +0000 UTC m=+0.328437065 container start 00aa5bbee0c3e2fd8d583df5da1dc36a115b6abef57607cbcb97333f2b30b237 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T04:27:46.901 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:46 smithi110 podman[151812]: 2024-09-21 04:27:46.89830758 +0000 UTC m=+0.408167048 container attach 00aa5bbee0c3e2fd8d583df5da1dc36a115b6abef57607cbcb97333f2b30b237 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T04:27:47.167 INFO:journalctl@ceph.osd.1.smithi110.stdout:Sep 21 04:27:47 smithi110 podman[151812]: 2024-09-21 04:27:47.003777186 +0000 UTC m=+0.513636649 container died 00aa5bbee0c3e2fd8d583df5da1dc36a115b6abef57607cbcb97333f2b30b237 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:27:47.643 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T04:27:47.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:27:47.683 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T04:27:47.683 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T04:27:47.683 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.2 2024-09-21T04:27:47.917 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:47 smithi110 systemd[1]: Stopping Ceph osd.2 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:48.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:48 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:27:48.039+0000 7f82e20c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:48.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:48 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:27:48.039+0000 7f82e20c3640 -1 osd.2 95 *** Got signal Terminated *** 2024-09-21T04:27:48.417 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:48 smithi110 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2[111687]: 2024-09-21T04:27:48.039+0000 7f82e20c3640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:27:53.859 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:53 smithi110 podman[151915]: 2024-09-21 04:27:53.572459488 +0000 UTC m=+5.668405311 container died 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:27:54.167 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:53 smithi110 podman[151915]: 2024-09-21 04:27:53.955776347 +0000 UTC m=+6.051722211 container cleanup 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2) 2024-09-21T04:27:54.167 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:53 smithi110 bash[151915]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2 2024-09-21T04:27:54.540 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[151930]: 2024-09-21 04:27:54.198788211 +0000 UTC m=+0.619687195 container remove 72f3c7b658aa91425252fd8f8702f884fce516b0ba44eb004eec313e5685d70c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:27:54.862 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.440804648 +0000 UTC m=+0.016035790 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:27:54.862 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.560947942 +0000 UTC m=+0.136179065 container create 120c907c93e0a28bb7442c4ab84ae1a9d40d3067cb5d9dce1aedddf33b13700a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T04:27:54.863 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.778054905 +0000 UTC m=+0.353286029 container init 120c907c93e0a28bb7442c4ab84ae1a9d40d3067cb5d9dce1aedddf33b13700a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T04:27:54.863 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.781432001 +0000 UTC m=+0.356663134 container start 120c907c93e0a28bb7442c4ab84ae1a9d40d3067cb5d9dce1aedddf33b13700a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:27:54.863 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.860443608 +0000 UTC m=+0.435674737 container attach 120c907c93e0a28bb7442c4ab84ae1a9d40d3067cb5d9dce1aedddf33b13700a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:27:55.150 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 04:27:54 smithi110 podman[152017]: 2024-09-21 04:27:54.97363596 +0000 UTC m=+0.548867093 container died 120c907c93e0a28bb7442c4ab84ae1a9d40d3067cb5d9dce1aedddf33b13700a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:27:55.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T04:27:55.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:27:55.586 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T04:27:55.587 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T04:27:55.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.3 2024-09-21T04:27:55.896 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:27:55 smithi148 systemd[1]: Stopping Ceph osd.3 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:27:56.190 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:27:55 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:27:55.893+0000 7f5993e8b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:27:56.190 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:27:55 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:27:55.893+0000 7f5993e8b640 -1 osd.3 95 *** Got signal Terminated *** 2024-09-21T04:27:56.190 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:27:55 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3[87151]: 2024-09-21T04:27:55.893+0000 7f5993e8b640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:28:00.190 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:27:59 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:27:59.890+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:01.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:00 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:00.983+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:01.191 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:00 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:00.929+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:01.653 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:01 smithi148 podman[116037]: 2024-09-21 04:28:01.221557645 +0000 UTC m=+5.434473612 container died 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T04:28:01.653 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:01 smithi148 podman[116037]: 2024-09-21 04:28:01.651148323 +0000 UTC m=+5.864064322 container cleanup 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:28:01.940 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:01 smithi148 bash[116037]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3 2024-09-21T04:28:01.940 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:01 smithi148 podman[116064]: 2024-09-21 04:28:01.869416805 +0000 UTC m=+0.640620500 container remove 36931325021ee4db6ae9d5dbb9784d38eaa7a465e2e2c089555549b395cc9134 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_CLEAN=True) 2024-09-21T04:28:02.275 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:01 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:01.950+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:02.276 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:01 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:01.953+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:02.583 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.176315149 +0000 UTC m=+0.016171014 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:28:02.583 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.278836269 +0000 UTC m=+0.118692166 container create a7b3cd16ec74839ec6aae6299655bcf9c16576071a25162ffb66dd04898e9b55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T04:28:02.583 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.506167035 +0000 UTC m=+0.346022891 container init a7b3cd16ec74839ec6aae6299655bcf9c16576071a25162ffb66dd04898e9b55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-21T04:28:02.583 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.509478619 +0000 UTC m=+0.349334480 container start a7b3cd16ec74839ec6aae6299655bcf9c16576071a25162ffb66dd04898e9b55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:28:02.583 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.581060595 +0000 UTC m=+0.420916450 container attach a7b3cd16ec74839ec6aae6299655bcf9c16576071a25162ffb66dd04898e9b55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:28:02.935 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 21 04:28:02 smithi148 podman[116151]: 2024-09-21 04:28:02.713637879 +0000 UTC m=+0.553493737 container died a7b3cd16ec74839ec6aae6299655bcf9c16576071a25162ffb66dd04898e9b55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:28:03.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:02.932+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:03.191 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:02 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:02.961+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:03.384 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T04:28:03.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:28:03.428 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T04:28:03.428 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T04:28:03.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.4 2024-09-21T04:28:03.658 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:03 smithi148 systemd[1]: Stopping Ceph osd.4 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:28:03.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:03 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:03.754+0000 7f6dfc6de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:28:03.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:03 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:03.754+0000 7f6dfc6de640 -1 osd.4 95 *** Got signal Terminated *** 2024-09-21T04:28:03.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:03 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:03.754+0000 7f6dfc6de640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:28:03.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:03 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:03.888+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:04.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:03 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:03.987+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:05.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:04 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:04.932+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:05.191 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:04 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:04.994+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:06.440 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:05 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:05.949+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:06.441 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:06 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:06.042+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:07.190 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:06 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:06.932+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:07.190 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:07 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:07.045+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:08.440 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:07 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4[91882]: 2024-09-21T04:28:07.964+0000 7f6df4ceb640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:36.904827+0000 front 2024-09-21T04:27:36.904764+0000 (oldest deadline 2024-09-21T04:28:00.404733+0000) 2024-09-21T04:28:08.441 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:08 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:08.028+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:08.441 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:08 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:08.028+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:09.383 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:09 smithi148 podman[116254]: 2024-09-21 04:28:09.150968574 +0000 UTC m=+5.503905968 container died 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2) 2024-09-21T04:28:09.383 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:09 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:09.004+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:09.383 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:09 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:09.004+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:09.675 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:09 smithi148 podman[116254]: 2024-09-21 04:28:09.488078879 +0000 UTC m=+5.841016274 container cleanup 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T04:28:09.675 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:09 smithi148 bash[116254]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4 2024-09-21T04:28:09.940 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:09 smithi148 podman[116269]: 2024-09-21 04:28:09.672730859 +0000 UTC m=+0.517582891 container remove 0dc491d8eac70aa05db3ed0a370320436f396a8316254b1aff6fe8d5f0f0c7e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T04:28:10.328 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:10 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:10.036+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:10.328 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:10 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:10.036+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:10.328 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:09.985673003 +0000 UTC m=+0.015830650 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:28:10.328 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:10.098545514 +0000 UTC m=+0.128703192 container create 068b87a2297a2fba5ecd088f6b7a350e52999c1ababeb1e13cd1e4d07bdd6656 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T04:28:10.328 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:10.326030713 +0000 UTC m=+0.356188364 container init 068b87a2297a2fba5ecd088f6b7a350e52999c1ababeb1e13cd1e4d07bdd6656 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T04:28:10.690 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:10.329325285 +0000 UTC m=+0.359482922 container start 068b87a2297a2fba5ecd088f6b7a350e52999c1ababeb1e13cd1e4d07bdd6656 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:28:10.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:10.400909763 +0000 UTC m=+0.431067412 container attach 068b87a2297a2fba5ecd088f6b7a350e52999c1ababeb1e13cd1e4d07bdd6656 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T04:28:10.691 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 21 04:28:10 smithi148 podman[116360]: 2024-09-21 04:28:10.51630591 +0000 UTC m=+0.546463551 container died 068b87a2297a2fba5ecd088f6b7a350e52999c1ababeb1e13cd1e4d07bdd6656 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T04:28:11.173 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:10 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:10.998+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:11.173 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:10 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:10.998+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:11.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T04:28:11.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:28:11.231 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T04:28:11.231 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T04:28:11.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.5 2024-09-21T04:28:11.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:11 smithi148 systemd[1]: Stopping Ceph osd.5 for 77422328-77ce-11ef-baf5-efdc52797490... 2024-09-21T04:28:11.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:11 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:11.549+0000 7f003c7f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:28:11.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:11 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:11.549+0000 7f003c7f3640 -1 osd.5 95 *** Got signal Terminated *** 2024-09-21T04:28:11.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:11 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:11.549+0000 7f003c7f3640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:28:12.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:12 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:12.031+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:12.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:12 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:12.031+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:13.432 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:13 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:13.061+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:13.433 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:13 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:13.061+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:14.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:14 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:14.085+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:14.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:14 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:14.085+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:15.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:15 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:15.133+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:15.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:15 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:15.133+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:16.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:16 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:16.162+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-09-21T04:27:34.192093+0000 front 2024-09-21T04:27:34.192003+0000 (oldest deadline 2024-09-21T04:27:59.491931+0000) 2024-09-21T04:28:16.440 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:16 smithi148 ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5[96766]: 2024-09-21T04:28:16.162+0000 7f0038e08640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-09-21T04:27:43.592885+0000 front 2024-09-21T04:27:43.592810+0000 (oldest deadline 2024-09-21T04:28:07.092672+0000) 2024-09-21T04:28:17.285 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:17 smithi148 podman[116462]: 2024-09-21 04:28:17.012435263 +0000 UTC m=+5.557425221 container died c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:28:17.625 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:17 smithi148 podman[116462]: 2024-09-21 04:28:17.44948394 +0000 UTC m=+5.994473897 container cleanup c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T04:28:17.625 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:17 smithi148 bash[116462]: ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5 2024-09-21T04:28:17.940 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:17 smithi148 podman[116479]: 2024-09-21 04:28:17.62302668 +0000 UTC m=+0.608940971 container remove c0f3036b7f8d2afb2bdf193f6c67b38d93df16752be60752e53f7cc5f390ad6e (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T04:28:18.323 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:17.919566739 +0000 UTC m=+0.015785596 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:28:18.323 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:18.043599371 +0000 UTC m=+0.139818202 container create 3b6be4dd113b4349dfe25b4b5d1575fca3b70702e8347bf000d72f9b02d851f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T04:28:18.323 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:18.245698401 +0000 UTC m=+0.341917244 container init 3b6be4dd113b4349dfe25b4b5d1575fca3b70702e8347bf000d72f9b02d851f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:28:18.323 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:18.248960381 +0000 UTC m=+0.345179216 container start 3b6be4dd113b4349dfe25b4b5d1575fca3b70702e8347bf000d72f9b02d851f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T04:28:18.323 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:18.320815304 +0000 UTC m=+0.417034150 container attach 3b6be4dd113b4349dfe25b4b5d1575fca3b70702e8347bf000d72f9b02d851f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T04:28:18.690 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 21 04:28:18 smithi148 podman[116566]: 2024-09-21 04:28:18.432664453 +0000 UTC m=+0.528883294 container died 3b6be4dd113b4349dfe25b4b5d1575fca3b70702e8347bf000d72f9b02d851f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-77422328-77ce-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:28:19.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77422328-77ce-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T04:28:19.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:28:19.273 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T04:28:19.273 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77422328-77ce-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:28:19.428 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:28:43.614 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77422328-77ce-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:28:43.770 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:28:59.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:28:59.298 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:28:59.325 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T04:28:59.327 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi110/crash 2024-09-21T04:28:59.328 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:28:59.366 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:28:59.366 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:28:59.368 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi148/crash 2024-09-21T04:28:59.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:28:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/77422328-77ce-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:28:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:28:59.394 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T04:28:59.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T04:28:59.445 INFO:teuthology.orchestra.run.smithi110.stdout:2024-09-21T04:20:00.000128+0000 mon.smithi110 (mon.0) 370 : cluster [WRN] osd.2 (root=default,host=smithi110) is down 2024-09-21T04:28:59.446 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-21T04:28:59.446 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[SEC\]' /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T04:28:59.521 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]' /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T04:28:59.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[WRN\]' /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T04:28:59.671 INFO:teuthology.orchestra.run.smithi110.stdout:2024-09-21T04:20:00.000128+0000 mon.smithi110 (mon.0) 370 : cluster [WRN] osd.2 (root=default,host=smithi110) is down 2024-09-21T04:28:59.671 INFO:tasks.cephadm:Compressing logs... 2024-09-21T04:28:59.672 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:28:59.714 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:28:59.736 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T04:28:59.737 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T04:28:59.737 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T04:28:59.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi110.log 2024-09-21T04:28:59.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:28:59.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi110.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:28:59.739 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.faxwju.log 2024-09-21T04:28:59.739 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:28:59.740 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-09-21T04:28:59.740 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.fpatxi.log 2024-09-21T04:28:59.740 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T04:28:59.740 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.fpatxi.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:28:59.741 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.tpueww.log 2024-09-21T04:28:59.741 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi148.log 2024-09-21T04:28:59.741 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:28:59.742 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.4.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.tpueww.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi148.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log: 86.0% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:28:59.742 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi148.log 2024-09-21T04:28:59.742 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.trhyxr.log 2024-09-21T04:28:59.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.faxwju.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.trhyxr.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:28:59.744 INFO:teuthology.orchestra.run.smithi110.stderr: 91.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:28:59.744 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi110.log 2024-09-21T04:28:59.745 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T04:28:59.745 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi148.log: /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.5.log: 93.9% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi148.log.gz 2024-09-21T04:28:59.745 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:28:59.745 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi148.zjrahi.log 2024-09-21T04:28:59.746 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log: 91.7% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:28:59.746 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T04:28:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log: 86.9% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:28:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T04:28:59.748 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi110.log: 93.9% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi110.log.gz 2024-09-21T04:28:59.748 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:28:59.749 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi148.zjrahi.log: 88.2% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:28:59.749 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:28:59.761 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi110.axmqzb.log 2024-09-21T04:28:59.765 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log: 88.2% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:28:59.766 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.4% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:28:59.766 INFO:teuthology.orchestra.run.smithi110.stderr: 91.4% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:28:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:28:59.768 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:28:59.801 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi148.zjrahi.log.gz 2024-09-21T04:28:59.816 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi110.axmqzb.log: /var/log/ceph/cephadm.log: 89.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:29:00.141 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi148.log.gz 2024-09-21T04:29:00.157 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mgr.smithi110.axmqzb.log.gz 2024-09-21T04:29:00.838 INFO:teuthology.orchestra.run.smithi110.stderr: 90.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mon.smithi110.log.gz 2024-09-21T04:29:04.882 INFO:teuthology.orchestra.run.smithi148.stderr: 93.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T04:29:04.888 INFO:teuthology.orchestra.run.smithi148.stderr: 94.9% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.fpatxi.log.gz 2024-09-21T04:29:05.321 INFO:teuthology.orchestra.run.smithi110.stderr: 94.8% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.trhyxr.log.gz 2024-09-21T04:29:05.357 INFO:teuthology.orchestra.run.smithi110.stderr: 93.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T04:29:06.084 INFO:teuthology.orchestra.run.smithi110.stderr: 93.5% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T04:29:06.273 INFO:teuthology.orchestra.run.smithi148.stderr: 94.9% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi148.tpueww.log.gz 2024-09-21T04:29:06.832 INFO:teuthology.orchestra.run.smithi148.stderr: 93.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T04:29:07.282 INFO:teuthology.orchestra.run.smithi110.stderr: 93.3% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T04:29:07.788 INFO:teuthology.orchestra.run.smithi148.stderr: 93.4% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T04:29:07.790 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-21T04:29:07.790 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m8.061s 2024-09-21T04:29:07.791 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m31.335s 2024-09-21T04:29:07.791 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m1.067s 2024-09-21T04:30:27.359 INFO:teuthology.orchestra.run.smithi110.stderr: 92.6% -- replaced with /var/log/ceph/77422328-77ce-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi110.faxwju.log.gz 2024-09-21T04:30:27.361 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-21T04:30:27.361 INFO:teuthology.orchestra.run.smithi110.stderr:real 1m27.634s 2024-09-21T04:30:27.361 INFO:teuthology.orchestra.run.smithi110.stderr:user 1m50.577s 2024-09-21T04:30:27.361 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m3.402s 2024-09-21T04:30:27.361 INFO:tasks.cephadm:Archiving logs... 2024-09-21T04:30:27.362 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi110/log 2024-09-21T04:30:27.362 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:30:43.139 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi148/log 2024-09-21T04:30:43.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:30:46.731 INFO:tasks.cephadm:Removing cluster... 2024-09-21T04:30:46.731 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77422328-77ce-11ef-baf5-efdc52797490 --force 2024-09-21T04:30:46.884 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:30:47.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77422328-77ce-11ef-baf5-efdc52797490 --force 2024-09-21T04:30:47.431 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 77422328-77ce-11ef-baf5-efdc52797490 2024-09-21T04:30:47.731 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T04:30:47.731 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:30:47.747 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:30:47.762 INFO:tasks.cephadm:Teardown complete 2024-09-21T04:30:47.763 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T04:30:47.779 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T04:30:47.779 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:30:47.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:30:47.866 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T04:30:47.880 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T04:30:47.880 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:30:47.883 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:30:47.895 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:30:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:30:47.899 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:30:47.899 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T04:30:47.900 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 -473us[ -459us] +/- 36ms 2024-09-21T04:30:47.900 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 +1657us[+1657us] +/- 78ms 2024-09-21T04:30:47.900 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -580us[ -580us] +/- 68ms 2024-09-21T04:30:47.900 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -477us[ -469us] +/- 36ms 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 +1674us[+1674us] +/- 78ms 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -575us[ -569us] +/- 67ms 2024-09-21T04:30:47.901 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:30:47.902 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T04:30:47.913 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T04:30:47.913 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T04:30:47.923 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:30:48.019 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:30:48.091 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:30:48.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:30:48.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:30:48.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:30:48.148 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T04:30:48.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T04:30:48.199 INFO:teuthology.task.internal:Duration was 1982.249313 seconds 2024-09-21T04:30:48.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T04:30:48.211 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T04:30:48.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:30:48.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:30:48.249 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:30:48.252 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:30:48.473 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T04:30:48.473 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T04:30:48.473 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:30:48.520 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T04:30:48.520 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:30:48.548 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T04:30:48.548 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:30:48.563 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:30:48.614 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T04:30:48.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:30:48.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:30:49.400 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T04:30:49.412 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T04:30:49.412 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:30:49.437 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:30:49.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T04:30:49.476 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:30:49.480 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:30:49.500 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-21T04:30:49.526 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-21T04:30:49.542 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:30:49.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:30:49.568 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:30:49.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:30:49.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T04:30:49.609 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T04:30:49.610 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi110 2024-09-21T04:30:49.610 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:30:49.659 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913678/remote/smithi148 2024-09-21T04:30:49.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:30:49.696 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T04:30:49.696 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:30:49.699 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:30:49.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T04:30:49.760 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T04:30:49.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T04:30:49.773 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T04:30:49.773 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:30:49.776 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:30:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:30 /home/ubuntu/cephtest 2024-09-21T04:30:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:30 /home/ubuntu/cephtest 2024-09-21T04:30:49.802 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T04:30:49.820 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T04:30:49.875 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1982.2493126392365 failure_reason: '"2024-09-21T04:20:00.000128+0000 mon.smithi110 (mon.0) 370 : cluster [WRN] osd.2 (root=default,host=smithi110) is down" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-09-21T04:30:49.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T04:30:49.981 INFO:teuthology.run:FAIL